summaryrefslogtreecommitdiff
path: root/src/t_zset.c
Commit message (Collapse)AuthorAgeFilesLines
* optimize zset conversion on large ZRANGESTORE (#10789)Oran Agra2022-06-141-2/+4
| | | | | | when we know the size of the zset we're gonna store in advance, we can check if it's greater than the listpack encoding threshold, in which case we can create a skiplist from the get go, and avoid converting the listpack to skiplist later after it was already populated.
* Fix ZRANGESTORE crash when zset_max_listpack_entries is 0 (#10767)Vitaly2022-05-271-3/+4
| | | | | | | | | When `zrangestore` is called container destination object is created. Before this PR we used to create a listpack based object even if `zset-max-ziplist-entries` or equivalent`zset-max-listpack-entries` was set to 0. This triggered immediate conversion of the listpack into a skiplist in `zrangestore`, which hits an assertion resulting in an engine crash. Added a TCL test that reproduces this issue.
* Fix BZMPOP gets unblocked by non-key args and returns them (#10764)Binbin2022-05-231-1/+1
| | | | | | | | | | | | This bug was introduced in #9484 (7.0.0). It result that BZMPOP blocked on non-key arguments. Like `bzmpop 0 1 myzset min count 10`, this command will additionally block in these keys (except for the first and the last argument) and can return their values: - 0: timeout value - 1: numkeys value - min: min/max token - count: count token
* fix some typos in "t_zset.c" (#10670)Lu JJ2022-05-091-4/+4
| | | | | fix some typo in "t_zset.c". 1. `zzlisinlexrange` the function name mentioned in the comment is misspelled. 2. fix typo in function name`zarndmemberReplyWithListpack` -> `zrandmemberReplyWithListpack`
* Optimize integer zset scores in listpack (converting to string and back) ↵Oran Agra2022-04-171-4/+12
| | | | | | | | | | | | | | | | | | | | | | | (#10486) When the score doesn't have fractional part, and can be stored as an integer, we use the integer capabilities of listpack to store it, rather than convert it to string. This already existed before this PR (lpInsert dose that conversion implicitly). But to do that, we would have first converted the score from double to string (calling `d2string`), then pass the string to `lpAppend` which identified it as being an integer and convert it back to an int. Now, instead of converting it to a string, we store it using lpAppendInteger`. Unrelated: --- * Fix the double2ll range check (negative and positive ranges, and also the comparison operands were slightly off. but also, the range could be made much larger, see comment). * Unify the double to string conversion code in rdb.c with the one in util.c * Small optimization in lpStringToInt64, don't attempt to convert strings that are obviously too long. Benchmark; --- Up to 20% improvement in certain tight loops doing zzlInsert with large integers. (if listpack is pre-allocated to avoid realloc, and insertion is sorted from largest to smaller)
* Fix several document error and function comments (#10580)Wen Hui2022-04-131-3/+3
| | | | | | | | | | | | This PR fix the following minor errors before Redis 7 release: ZRANGEBYLEX command in deprecated in 6.2.0, and could be replaced by ZRANGE with the BYLEX argument, but in the document, the words is written incorrect in " by ZRANGE with the BYSCORE argument" Fix function zpopmaxCommand incorrect comment The comments of function zmpopCommand and bzmpopCommand are not consistent with document description, fix them Co-authored-by: Ubuntu <lucas.guang.yang1@huawei.com>
* introduce MAX_D2STRING_CHARS instead of 128 const (#10487)Oran Agra2022-03-281-1/+1
| | | | | | | | | There are a few places that use a hard coded const of 128 to allocate a buffer for d2string. Replace these with a clear macro. Note that In theory, converting double into string could take as much as nearly 400 chars, but since d2string uses `%g` and not `%f`, it won't pass some 40 chars. unrelated: restore some changes to auto generated commands.c that got accidentally reverted in #10293
* Fix an off by one error in zzlStrtod (#10465)yiyuaner2022-03-221-2/+2
| | | | When vlen = sizeof(buf), the statement buf[vlen] = '\0' accessing the buffer buf is an off by one error.
* A faster and more robust code of zslRandomLevel using RAND_MAX (#5539)Henry2022-03-021-1/+2
| | | | | | 1. since ZSKIPLIST_P is float, using it directly inside the condition used to causes floating point code to be used (gcc/x86) 2. In some operating system(eg.Windows), the largest value returned from random() is 0x7FFF(15bit), so after bitwise AND with 0xFFFF, the probability of the less operation returning true in the while loop's condition is no more equal to ZSKIPLIST_P. 3. In case some library has random() returning int in range [0~ZSKIPLIST_P*65535], the while loop will be an infinite loop. 4. on Linux where RAND_MAX is higher than 0xFFFF, this change actually improves precision (despite not matching the result against a float value)
* Optimization: Avoid deferred array reply on ZRANGE commands BYRANK (#10337)filipe oliveira2022-02-241-8/+27
| | | | | | | Avoid deferred array reply on genericZrangebyrankCommand() when consumer type is client. I.e. any ZRANGE / ZREVRNGE (when tank is used). This was a performance regression introduced in #7844 (v 6.2) mainly affecting pipelined workloads. Co-authored-by: Oran Agra <oran@redislabs.com>
* sub-command support for ACL CAT and COMMAND LIST. redisCommand always stores ↵Binbin2022-01-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | fullname (#10127) Summary of changes: 1. Rename `redisCommand->name` to `redisCommand->declared_name`, it is a const char * for native commands and SDS for module commands. 2. Store the [sub]command fullname in `redisCommand->fullname` (sds). 3. List subcommands in `ACL CAT` 4. List subcommands in `COMMAND LIST` 5. `moduleUnregisterCommands` now will also free the module subcommands. 6. RM_GetCurrentCommandName returns full command name Other changes: 1. Add `addReplyErrorArity` and `addReplyErrorExpireTime` 2. Remove `getFullCommandName` function that now is useless. 3. Some cleanups about `fullname` since now it is SDS. 4. Delete `populateSingleCommand` function from server.h that is useless. 5. Added tests to cover this change. 6. Add some module unload tests and fix the leaks 7. Make error messages uniform, make sure they always contain the full command name and that it's quoted. 7. Fixes some typos see the history in #9504, fixes #10124 Co-authored-by: Oran Agra <oran@redislabs.com> Co-authored-by: guybe7 <guy.benoish@redislabs.com>
* Sort out the mess around writable replicas and lookupKeyRead/Write (#9572)Viktor Söderqvist2021-11-281-6/+2
| | | | | | | | | | | | | | | | | | | | | | | | Writable replicas now no longer use the values of expired keys. Expired keys are deleted when lookupKeyWrite() is used, even on a writable replica. Previously, writable replicas could use the value of an expired key in write commands such as INCR, SUNIONSTORE, etc.. This commit also sorts out the mess around the functions lookupKeyRead() and lookupKeyWrite() so they now indicate what we intend to do with the key and are not affected by the command calling them. Multi-key commands like SUNIONSTORE, ZUNIONSTORE, COPY and SORT with the store option now use lookupKeyRead() for the keys they're reading from (which will not allow reading from logically expired keys). This commit also fixes a bug where PFCOUNT could return a value of an expired key. Test modules commands have their readonly and write flags updated to correctly reflect their lookups for reading or writing. Modules are not required to correctly reflect this in their command flags, but this change is made for consistency since the tests serve as usage examples. Fixes #6842. Fixes #7475.
* Replace ziplist with listpack in quicklist (#9740)sundb2021-11-241-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Part three of implementing #8702, following #8887 and #9366 . ## Description of the feature 1. Replace the ziplist container of quicklist with listpack. 2. Convert existing quicklist ziplists on RDB loading time. an O(n) operation. ## Interface changes 1. New `list-max-listpack-size` config is an alias for `list-max-ziplist-size`. 2. Replace `debug ziplist` command with `debug listpack`. ## Internal changes 1. Add `lpMerge` to merge two listpacks . (same as `ziplistMerge`) 2. Add `lpRepr` to print info of listpack which is used in debugCommand and `quicklistRepr`. (same as `ziplistRepr`) 3. Replace `QUICKLIST_NODE_CONTAINER_ZIPLIST` with `QUICKLIST_NODE_CONTAINER_PACKED`(following #9357 ). It represent that a quicklistNode is a packed node, as opposed to a plain node. 4. Remove `createZiplistObject` method, which is never used. 5. Calculate listpack entry size using overhead overestimation in `quicklistAllowInsert`. We prefer an overestimation, which would at worse lead to a few bytes below the lowest limit of 4k. ## Improvements 1. Calling `lpShrinkToFit` after converting Ziplist to listpack, which was missed at #9366. 2. Optimize `quicklistAppendPlainNode` to avoid memcpy data. ## Bugfix 1. Fix crash in `quicklistRepr` when ziplist is compressed, introduced from #9366. ## Test 1. Add unittest for `lpMerge`. 2. Modify the old quicklist ziplist corrupt dump test. Co-authored-by: Oran Agra <oran@redislabs.com>
* Fixes ZPOPMIN/ZPOPMAX wrong replies when count is 0 with non-zset (#9711)Binbin2021-11-181-22/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Moves ZPOP ... 0 fast exit path after type check to reply with WRONGTYPE. In the past it will return an empty array. Also now count is not allowed to be negative. see #9680 before: ``` 127.0.0.1:6379> set zset str OK 127.0.0.1:6379> zpopmin zset 0 (empty array) 127.0.0.1:6379> zpopmin zset -1 (empty array) ``` after: ``` 127.0.0.1:6379> set zset str OK 127.0.0.1:6379> zpopmin zset 0 (error) WRONGTYPE Operation against a key holding the wrong kind of value 127.0.0.1:6379> zpopmin zset -1 (error) ERR value is out of range, must be positive ```
* fix: lookupKey on SETNX and SETXX only once (#9640)perryitay2021-11-031-2/+2
| | | | | When using SETNX and SETXX we could end up doing key lookup twice. This presents a small inefficiency price. Also once we have statistics of write hit and miss they'll be wrong (recording the same key hit twice)
* Fix multiple COUNT in LMPOP/BLMPOP/ZMPOP/BZMPOP (#9701)Binbin2021-10-311-2/+4
| | | | | | | The previous code did not check whether COUNT is set. So we can use `lmpop 2 key1 key2 left count 1 count 2`. This situation can occur in LMPOP/BLMPOP/ZMPOP/BZMPOP commands. LMPOP/BLMPOP introduced in #9373, ZMPOP/BZMPOP introduced in #9484.
* Fix ziplist and listpack overflows and truncations (CVE-2021-32627, ↵Oran Agra2021-10-041-23/+39
| | | | | | | | | | | | | | | | CVE-2021-32628) (#9589) - fix possible heap corruption in ziplist and listpack resulting by trying to allocate more than the maximum size of 4GB. - prevent ziplist (hash and zset) from reaching size of above 1GB, will be converted to HT encoding, that's not a useful size. - prevent listpack (stream) from reaching size of above 1GB. - XADD will start a new listpack if the new record may cause the previous listpack to grow over 1GB. - XADD will respond with an error if a single stream record is over 1GB - List type (ziplist in quicklist) was truncating strings that were over 4GB, now it'll respond with an error. Co-authored-by: sundb <sundbcn@gmail.com>
* Cleanup typos, incorrect comments, and fixed small memory leak in redis-cli ↵Binbin2021-10-021-3/+3
| | | | | | | | | | (#9153) 1. Remove forward declarations from header files to functions that do not exist: hmsetCommand and rdbSaveTime. 2. Minor phrasing fixes in #9519 3. Add missing sdsfree(title) and fix typo in redis-benchmark. 4. Modify some error comments in some zset commands. 5. Fix copy-paste bug comment in syncWithMaster about `ip-address`.
* Use dictGetFairRandomKey() for HRANDFIELD,SRANDMEMBER,ZRANDMEMBER (#9538)sundb2021-09-241-1/+1
| | | | | | | | In the `HRANDFIELD`, `SRANDMEMBER` and `ZRANDMEMBER` commands, There are some strategies that could in some rare cases return an unfair random. these cases are where s small dict happens be be hashed unevenly. Specifically when `count*ZRANDMEMBER_SUB_STRATEGY_MUL > size`, using `dictGetRandomKey` to randomize from a dict will result in an unfair random result.
* Add ZMPOP/BZMPOP commands. (#9484)Binbin2021-09-231-62/+200
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is similar to the recent addition of LMPOP/BLMPOP (#9373), but zset. Syntax for the new ZMPOP command: `ZMPOP numkeys [<key> ...] MIN|MAX [COUNT count]` Syntax for the new BZMPOP command: `BZMPOP timeout numkeys [<key> ...] MIN|MAX [COUNT count]` Some background: - ZPOPMIN/ZPOPMAX take only one key, and can return multiple elements. - BZPOPMIN/BZPOPMAX take multiple keys, but return only one element from just one key. - ZMPOP/BZMPOP can take multiple keys, and can return multiple elements from just one key. Note that ZMPOP/BZMPOP can take multiple keys, it eventually operates on just on key. And it will propagate as ZPOPMIN or ZPOPMAX with the COUNT option. As new commands, if we can not pop any elements, the response like: - ZMPOP: Return a NIL in both RESP2 and RESP3, unlike ZPOPMIN/ZPOPMAX return emptyarray. - BZMPOP: Return a NIL in both RESP2 and RESP3 when timeout is reached, like BZPOPMIN/BZPOPMAX. For the normal response is nested arrays in RESP2 and RESP3: ``` ZMPOP/BZMPOP 1) keyname 2) 1) 1) member1 2) score1 2) 1) member2 2) score2 In RESP2: 1) "myzset" 2) 1) 1) "three" 2) "3" 2) 1) "two" 2) "2" In RESP3: 1) "myzset" 2) 1) 1) "three" 2) (double) 3 2) 1) "two" 2) (double) 2 ```
* Adds limit to SINTERCARD/ZINTERCARD. (#9425)Binbin2021-09-161-2/+37
| | | | | | | | | | | | | | | | Implements the [LIMIT limit] variant of SINTERCARD/ZINTERCARD. Now with the LIMIT, we can stop the searching when cardinality reaching the limit, and return the cardinality ASAP. Note that in SINTERCARD, the old synatx was: `SINTERCARD key [key ...]` In order to add a optional parameter, we must break the old synatx. So the new syntax of SINTERCARD will be consistent with ZINTERCARD. New syntax: `SINTERCARD numkeys key [key ...] [LIMIT limit]`. Note that this means that SINTERCARD has a different syntax than SINTER and SINTERSTORE (taking numkeys argument) As for ZINTERCARD, we can easily add a optional parameter to it. New syntax: `ZINTERCARD numkeys key [key ...] [LIMIT limit]`
* Replace all usage of ziplist with listpack for t_zset (#9366)sundb2021-09-091-221/+148
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Part two of implementing #8702 (zset), after #8887. ## Description of the feature Replaced all uses of ziplist with listpack in t_zset, and optimized some of the code to optimize performance. ## Rdb format changes New `RDB_TYPE_ZSET_LISTPACK` rdb type. ## Rdb loading improvements: 1) Pre-expansion of dict for validation of duplicate data for listpack and ziplist. 2) Simplifying the release of empty key objects when RDB loading. 3) Unify ziplist and listpack data verify methods for zset and hash, and move code to rdb.c. ## Interface changes 1) New `zset-max-listpack-entries` config is an alias for `zset-max-ziplist-entries` (same with `zset-max-listpack-value`). 2) OBJECT ENCODING will return listpack instead of ziplist. ## Listpack improvements: 1) Add `lpDeleteRange` and `lpDeleteRangeWithEntry` functions to delete a range of entries from listpack. 2) Improve the performance of `lpCompare`, converting from string to integer is faster than converting from integer to string. 3) Replace `snprintf` with `ll2string` to improve performance in converting numbers to strings in `lpGet()`. ## Zset improvements: 1) Improve the performance of `zzlFind` method, use `lpFind` instead of `lpCompare` in a loop. 2) Use `lpDeleteRangeWithEntry` instead of `lpDelete` twice to delete a element of zset. ## Tests 1) Add some unittests for `lpDeleteRange` and `lpDeleteRangeWithEntry` function. 2) Add zset RDB loading test. 3) Add benchmark test for `lpCompare` and `ziplsitCompare`. 4) Add empty listpack zset corrupt dump test.
* Add LMPOP/BLMPOP commands. (#9373)Binbin2021-09-091-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We want to add COUNT option for BLPOP. But we can't do it without breaking compatibility due to the command arguments syntax. So this commit introduce two new commands. Syntax for the new LMPOP command: `LMPOP numkeys [<key> ...] LEFT|RIGHT [COUNT count]` Syntax for the new BLMPOP command: `BLMPOP timeout numkeys [<key> ...] LEFT|RIGHT [COUNT count]` Some background: - LPOP takes one key, and can return multiple elements. - BLPOP takes multiple keys, but returns one element from just one key. - LMPOP can take multiple keys and return multiple elements from just one key. Note that LMPOP/BLMPOP can take multiple keys, it eventually operates on just one key. And it will propagate as LPOP or RPOP with the COUNT option. As a new command, it still return NIL if we can't pop any elements. For the normal response is nested arrays in RESP2 and RESP3, like: ``` LMPOP/BLMPOP 1) keyname 2) 1) element1 2) element2 ``` I.e. unlike BLPOP that returns a key name and one element so it uses a flat array, and LPOP that returns multiple elements with no key name, and again uses a flat array, this one has to return a nested array, and it does for for both RESP2 and RESP3 (like SCAN does) Some discuss can see: #766 #8824
* dict struct memory optimizations (#9228)yoav-steinberg2021-08-051-5/+5
| | | | | | | | | | | | | | | | | | Reduce dict struct memory overhead on 64bit dict size goes down from jemalloc's 96 byte bin to its 56 byte bin. summary of changes: - Remove `privdata` from callbacks and dict creation. (this affects many files, see "Interface change" below). - Meld `dictht` struct into the `dict` struct to eliminate struct padding. (this affects just dict.c and defrag.c) - Eliminate the `sizemask` field, can be calculated from size when needed. - Convert the `size` field into `size_exp` (exponent), utilizes one byte instead of 8. Interface change: pass dict pointer to dict type call back functions. This is instead of passing the removed privdata field. In the future if we'd like to have private data in the callbacks we can extract it from the dict type. We can extend dictType to include a custom dict struct allocator and use it to allocate more data at the end of the dict struct. This data can then be used to store private data later acccessed by the callbacks.
* Add SINTERCARD/ZINTERCARD Commands (#8946)Jonah H. Harris2021-08-031-12/+24
| | | | | | | | Add SINTERCARD and ZINTERCARD commands that are similar to ZINTER and SINTER but only return the cardinality with minimum processing and memory overheads. Co-authored-by: Oran Agra <oran@redislabs.com>
* fix zslGetRank bug in dead-code (#9246)Oran Agra2021-07-221-1/+1
| | | | | | | | | | This fixes an issue with zslGetRank which will happen only if the skiplist data stracture is added two entries with the same element name, this can't happen in redis zsets (we use dict), but in theory this is a bug in the underlaying skiplist code. Fixes #3081 and #4032 Co-authored-by: minjian.cai <cmjgithub@163.com>
* Fix rank overflow in zslInsert with more than 2B entries. (#9249)Binbin2021-07-181-1/+1
| | | | If there are more than 2B entries in a zset. The calculated span will overflow.
* hrandfield and zrandmember with count should return emptyarray when key does ↵Binbin2021-07-051-2/+2
| | | | | | | | | not exist. (#9178) due to a copy-paste bug, it used to reply with null response rather than empty array. this commit includes new tests that are looking at the RESP response directly in order to be able to tell the difference between them. Co-authored-by: Oran Agra <oran@redislabs.com>
* fix ZRANGESTORE - should return 0 when src points to an empty key (#9089)Leibale Eidelman2021-06-291-1/+6
| | | | | mistakenly it used to return an empty array rather than 0. Co-authored-by: Oran Agra <oran@redislabs.com>
* ZRANDMEMBER WITHSCORES with negative COUNT may return bad score (#9162)Binbin2021-06-291-1/+1
| | | | | Return a bad score when used with negative count (or count of 1), and non-ziplist encoded zset. Also add test to validate the return value and cover the issue.
* Remove extra semicolon (#9117)Binbin2021-06-211-1/+1
| | | | Remove extra semicolon.
* Change return value type for ZPOPMAX/MIN in RESP3 (#8981)Jason Elbaum2021-06-161-4/+15
| | | | | | | | | When using RESP3, ZPOPMAX/ZPOPMIN should return nested arrays for consistency with other commands (e.g. ZRANGE). We do that only when COUNT argument is present (similarly to how LPOP behaves). for reasoning see https://github.com/redis/redis/issues/8824#issuecomment-855427955 This is a breaking change only when RESP3 is used, and COUNT argument is present!
* Cleaning up the cluster interface by moving almost all related declar… (#9080)Uri Shachar2021-06-151-3/+0
| | | | | | | | | | | | | | * Cleaning up the cluster interface by moving almost all related declarations into cluster.h (no logic change -- just moving declarations/definitions around) This initial effort leaves two items out of scope - the configuration parsing into the server struct and the internals exposed by the clusterNode struct. * Remove unneeded declarations of dictSds* Ideally all the dictSds functionality would move from server.c into a dedicated module so we can avoid the duplication in redis-benchmark/cli * Move crc16 back into server.h, will be moved out once we create a seperate header file for hashing functions
* Fixed some typos, add a spell check ci and others minor fix (#8890)Binbin2021-06-101-5/+5
| | | | | | | | | | | | | | | | | | | | | This PR adds a spell checker CI action that will fail future PRs if they introduce typos and spelling mistakes. This spell checker is based on blacklist of common spelling mistakes, so it will not catch everything, but at least it is also unlikely to cause false positives. Besides that, the PR also fixes many spelling mistakes and types, not all are a result of the spell checker we use. Here's a summary of other changes: 1. Scanned the entire source code and fixes all sorts of typos and spelling mistakes (including missing or extra spaces). 2. Outdated function / variable / argument names in comments 3. Fix outdated keyspace masks error log when we check `config.notify-keyspace-events` in loadServerConfigFromString. 4. Trim the white space at the end of line in `module.c`. Check: https://github.com/redis/redis/pull/7751 5. Some outdated https link URLs. 6. Fix some outdated comment. Such as: - In README: about the rdb, we used to said create a `thread`, change to `process` - dbRandomKey function coment (about the dictGetRandomKey, change to dictGetFairRandomKey) - notifyKeyspaceEvent fucntion comment (add type arg) - Some others minor fix in comment (Most of them are incorrectly quoted by variable names) 7. Modified the error log so that users can easily distinguish between TCP and TLS in `changeBindAddr`
* Add missing zuiClearIterator in zrandmemberWithCountCommand. (#8979)Binbin2021-06-061-0/+3
| | | | | Also the bug that currently does not cause memory leaks. Because op->type = OBJ_ZSET, in zuiClearIterator, we will do nothing. Just a cleanup that zuiInitIterator and zuiClearIterator should appear in pairs.
* Fix error reply in case zset command is not the STORE variant (#8793)guybe72021-04-151-2/+2
|
* fix typo, stracture to structure (#8784)Bonsai2021-04-141-1/+1
|
* zsetAdd: Fix wrong reply in case of INCR and GT/LT (#8717)guybe72021-04-011-14/+19
| | | | | | | | | | | | | | If GT/LT fails the operation we need to reply with nill (like failure due to NX). Other changes: Add the missing $encoding suffix to many zset tests Note: there's a behavior change just in case of INCR + GT/LT that fails. The old code was replying with the wrong (rejected) score, and now it'll reply with nil. Note that that's anyway a corner case so this "behavior change" shouldn't have too much affect. Using GT/LT with INCR has a predictable result even before we run the command (INCR GT will only only / always fail if the increment is negative).
* reuse existing range comparators in the zset (#8714)wuYin2021-04-011-6/+3
| | | | | | There are 2 common range comparators for skiplist: zslValueGteMin and zslValueLteMax, but they're not being reused in zslDeleteRangeByScore This is a small change to make code cleaner.
* Cleanup ZADD_* flags (#8559)guybe72021-03-101-43/+40
| | | | | | | | | Have a clear separation between in and out flags Other changes: delete dead code in RM_ZsetIncrby: if zsetAdd returned error (happens only if the result of the operation is NAN or if score is NAN) we return immediately so there is no way that zsetAdd succeeded and returned NAN in the out-flags
* SRANDMEMBER RESP3 return should be Array, not Set (#8504)Wen Hui2021-02-221-0/+2
| | | | | | | | | | | | SRANDMEMBER with negative count (non unique) can return the same member multiple times, and the order of elements in the returned collection matters. For these reasons returning a RESP3 Set type is not valid for the negative count, but also not really valid for the positive (unique) variant either (the command returns an array of random picks, not a set) This PR also contains a minor optimization for SRANDMEMBER, HRANDFIELD, and ZRANDMEMBER, to avoid the temporary dict from being rehashed while it grows. Co-authored-by: Oran Agra <oran@redislabs.com>
* Optimize HRANDFIELD and ZRANDMEMBER case 4 when ziplist encoded (#8444)Oran Agra2021-02-071-14/+33
| | | | | | | | | | | | | | | | It is inefficient to repeatedly pick a single random element from a ziplist. For CASE4, which is when the user requested a low number of unique random picks from the collectoin, we used thta pattern. Now we use a different algorithm that picks unique elements from a ziplist, and guarentee no duplicate but doesn't provide random order (which is only needed in the non-unique random picks case) Unrelated changes: * change ziplist count and indexes variables to unsigned * solve compilation warnings about uninitialized vars in gcc 10.2 Co-authored-by: xinluton <xinluton@qq.com>
* RAND* commands: fix risk of OOM panic in hash and zset, use fair random in ↵sundb2021-02-051-15/+26
| | | | | | | | | | hash, and add tests for even distribution to all (#8429) Changes to HRANDFIELD and ZRANDMEMBER: * Fix risk of OOM panic when client query a very big negative count (avoid allocating huge temporary buffer). * Fix uneven random distribution in HRANDFIELD with negative count (wasn't using dictGetFairRandomKey). * Add tests to check an even random distribution (HRANDFIELD, SRANDMEMBER, ZRANDMEMBER). Co-authored-by: Oran Agra <oran@redislabs.com>
* Add HRANDFIELD and ZRANDMEMBER. improvements to SRANDMEMBER (#8297)Yang Bodong2021-01-291-4/+265
| | | | | | | | | | | | | | | | | | | | | | | New commands: `HRANDFIELD [<count> [WITHVALUES]]` `ZRANDMEMBER [<count> [WITHSCORES]]` Algorithms are similar to the one in SRANDMEMBER. Both return a simple bulk response when no arguments are given, and an array otherwise. In case values/scores are requested, RESP2 returns a long array, and RESP3 a nested array. note: in all 3 commands, the only option that also provides random order is the one with negative count. Changes to SRANDMEMBER * Optimization when count is 1, we can use the more efficient algorithm of non-unique random * optimization: work with sds strings rather than robj Other changes: * zzlGetScore: when zset needs to convert string to double, we use safer memcpy (in case the buffer is too small) * Solve a "bug" in SRANDMEMBER test: it intended to test a positive count (case 3 or case 4) and by accident used a negative count Co-authored-by: xinluton <xinluton@qq.com> Co-authored-by: Oran Agra <oran@redislabs.com>
* Add tests for RESP3 responce of ZINTER and ZRANGE (#8391)Oran Agra2021-01-261-0/+6
| | | | | | It was confusing as to why these don't return a map type. the reason is that order matters, so we need to make sure the client library knows to respect it. Added comments in the implementation and tests to cover it.
* Remove check leading to duplicate branches (#8398)Vladimir Maksimovski2021-01-251-6/+5
| | | Remove check leading to duplicate branches and unused withscores parameter
* Fix use of lookupKeyRead and lookupKeyWrite in zrangeGenericCommand, ↵sundb2021-01-131-3/+10
| | | | | | | | | | | | | zunionInterDiffGenericCommand (#8316) * Change zunionInterDiffGenericCommand to use lookupKeyRead if dstkey is null * Change zrangeGenericCommand to use lookupKey Write if dstkey isn't null ZRANGESTORE and UNION, ZINTER, ZDIFF are all new commands (6.2 RC1 and RC2). In redis 6.0 the ZRANGE was using lookupKeyRead, and ZUNIONSTORE / ZINTERSTORE were using lookupKeyWrite. So there bugs are introduced in 6.2 and will be resolved before it is released. the implications of this bug are also not big: The sole difference between LookupKeyRead and LookupKeyWrite is for command executed on a replica, which are not received from its master client. (for the master, and for the master client on the replica, these two functions behave the same)!
* Add ZRANGESTORE command, and improve ZSTORE command (#7844)Jonah H. Harris2021-01-071-233/+434
| | | | | | | | | | | | | | | | | | | Add ZRANGESTORE command, and improve ZSTORE command to deprecated Z[REV]RANGE[BYSCORE|BYLEX]. Syntax for the new ZRANGESTORE command: ZRANGESTORE [BYSCORE | BYLEX] [REV] [LIMIT offset count] New syntax for ZRANGE: ZRANGE [BYSCORE | BYLEX] [REV] [WITHSCORES] [LIMIT offset count] Old syntax for ZRANGE: ZRANGE [WITHSCORES] Other ZRANGE commands remain unchanged. The implementation uses common code for all of these, by utilizing a consumer interface that in one command response to the client, and in the other command stores a zset key. Co-authored-by: Oran Agra <oran@redislabs.com>
* Flow through the error handling path for most errors (#8226)Madelyn Olson2020-12-231-10/+10
| | | Properly throw errors for invalid replication stream and support https://github.com/redis/redis/pull/8217
* zset full dump sanitization bug (dup score instead of field) (#8167)Oran Agra2020-12-091-1/+1
|