summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* LFU: make counter log factor and decay time configurable.lfuantirez2016-07-205-14/+95
|
* LFU: Use the LRU pool for the LFU algorithm.antirez2016-07-181-36/+25
| | | | | Verified to have better real world performances with power-law access patterns because of the data accumulated across calls.
* LFU: Fix bugs in frequency decay code.antirez2016-07-181-2/+2
|
* LFU: Initial naive eviction cycle.antirez2016-07-183-4/+49
| | | | | | | It is possible to get better results by using the pool like in the LRU case. Also from tests during the morning I believe the current implementation has issues in the frequency decay function that should decrease the counter at periodic intervals.
* redis-cli LRU test mode: randomize value of key when setting.antirez2016-07-181-1/+4
| | | | | This way it is possible from an observer to tell when the key is replaced with a new one having the same name.
* redis-cli LRU test mode: remove newline from key names.antirez2016-07-181-1/+1
|
* LFU: Redis object level implementation.antirez2016-07-155-15/+142
| | | | | Implementation of LFU maxmemory policy for anything related to Redis objects. Still no actual eviction implemented.
* LFU simulator: remove dead code.antirez2016-07-141-5/+0
|
* LRU simulator: fix new entry creation decr time.antirez2016-07-141-1/+1
|
* LRU simulator: fix new entry creation.antirez2016-07-141-2/+2
|
* LFU: Simulation of the algorithm planned for Redis.antirez2016-07-141-0/+163
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We have 24 total bits of space in each object in order to implement an LFU (Least Frequently Used) eviction policy. We split the 24 bits into two fields: 8 bits 16 bits +--------+----------------+ | LOG_C | Last decr time | +--------+----------------+ LOG_C is a logarithmic counter that provides an indication of the access frequency. However this field must also be deceremented otherwise what used to be a frequently accessed key in the past, will remain ranked like that forever, while we want the algorithm to adapt to access pattern changes. So the remaining 16 bits are used in order to store the "decrement time", a reduced-precision unix time (we take 16 bits of the time converted in minutes since we don't care about wrapping around) where the LOG_C counter is halved if it has an high value, or just decremented if it has a low value. New keys don't start at zero, in order to have the ability to collect some accesses before being trashed away, so they start at COUNTER_INIT_VAL. The logaritmic increment performed on LOG_C takes care of COUNTER_INIT_VAL when incrementing the key, so that keys starting at COUNTER_INIT_VAL (or having a smaller value) have a very high chance of being incremented on access. The simulation starts with a power-law access pattern, and later converts into a flat access pattern in order to see how the algorithm adapts. Currenty the decrement operation period is 1 minute, however note that it is not guaranteed that each key will be scanned 1 time every minute, so the actual frequency can be lower. However under high load, we access 3/5 keys every newly inserted key (because of how Redis eviction works). This is a work in progress at this point to evaluate if this works well.
* LRU: Make cross-database choices for eviction.antirez2016-07-133-110/+161
| | | | | | | | | | | | | | The LRU eviction code used to make local choices: for each DB visited it selected the best key to evict. This was repeated for each DB. However this means that there could be DBs with very frequently accessed keys that are targeted by the LRU algorithm while there were other DBs with many better candidates to expire. This commit attempts to fix this problem for the LRU policy. However the TTL policy is still not fixed by this commit. The TTL policy will be fixed in a successive commit. This is an initial (partial because of TTL policy) fix for issue #2647.
* LRU: cache SDS strings in the eviction pool.antirez2016-07-121-13/+29
| | | | | | | To destroy and recreate the pool[].key element is slow, so we allocate in pool[].cached SDS strings that can account up to 255 chars keys and try to reuse them. This provides a solid 20% performance improvement in real world workload alike benchmarks.
* Move the struct evictionPoolEntry() into only file using it.antirez2016-07-122-12/+22
| | | | Local scope is always better when possible.
* Move prototype of evictionPoolAlloc() in server.h.antirez2016-07-122-2/+3
|
* LRU: use C99 variable len stack array in evictionPoolPopulate().antirez2016-07-121-11/+1
|
* redis-benchmark: new option to show server errors on stdout.antirez2016-07-121-1/+17
| | | | | Disabled by default, can be activated with -e. Maybe the reverse was more safe but departs from the past behavior.
* Remove useless memmove() from freeMemoryIfNeeded().antirez2016-07-111-7/+2
| | | | | | We start from the end of the pool to the initial item, zero-ing every entry we use or every ghost entry, there is nothing to memmove since to the right everything should be already set to NULL.
* LRU: Fix output fixes to new test-lru.rb.antirez2016-07-111-6/+6
|
* Merge branch 'unstable' of github.com:/antirez/redis into unstableantirez2016-07-111-1/+1
|\
| * redis_check_rdb(): the rio structure must be global.antirez2016-07-061-1/+1
| | | | | | | | | | | | | | | | The rio structure is referenced in the global 'riostate' structure in order for the logging functions to be always able to access the state of the "pseudo-loading" of the RDB, needed for the check. Courtesy of Valgrind.
* | LRU: test-lru.rb improved in different ways.antirez2016-07-112-98/+180
|/ | | | | | | | | | 1. Scan keys with pause to account for actual LRU precision. 2. Test cross-DB with 100 keys allocated in DB1. 3. Output results that don't fluctuate depending on number of keys. 4. Output results in percentage to make more sense. 5. Save file instead of outputting to STDOUT. 6. Support running multiple times with average of outputs. 7. Label each square (DIV) with its ID as HTML title.
* redis_check_rdb_main(): create shared objects only if needed.antirez2016-07-061-1/+5
| | | | | Otherwise Valgrind will complain a memory leak under certain tests where RDB checking is invoked from within Redis.
* Fix redis_check_rdb() return value.antirez2016-07-061-2/+3
|
* Remove dead code from geohash_helper.c.antirez2016-07-061-6/+0
| | | | | The function removed also had potential bugs related to signess of the expression, and is not used anyway.
* Fix signess issue in geohashEstimateStepsByRadius().antirez2016-07-061-1/+2
|
* Fix definition of M_PI in geohash_helper.c.antirez2016-07-062-1/+2
| | | | Without the right feature macros M_PI is not defined in math.h.
* geohash.c and geohash_helper.c are part of Redis.antirez2016-07-067-43/+12
| | | | | | | | | | They were under /deps since they originate from a different source tree, however at this point they are very modified and we took ownership of both the files making changes, fixing bugs, so there is no upgrade path from the original code tree. Given that, better to move the code under /src with proper dependencies and with a more simpler editing experience.
* Add expire.c and evict.c.antirez2016-07-062-0/+718
|
* Expire and LRU related code moved into different files.antirez2016-07-065-633/+4
|
* Makefile: don't build dependencies file for clean, distclean.antirez2016-07-061-0/+3
|
* Generate Makefile.dep at every build.antirez2016-07-063-194/+6
| | | | | Normally we used to update it from time to time. Too fragile... better to generate dependencies at every run and delete them on 'make clean'.
* Regression test for issue #3333.antirez2016-07-061-0/+6
|
* getLongLongFromObject: use string2ll() instead of strict_strtoll().antirez2016-07-061-15/+1
| | | | | | | | | | strict_strtoll() has a bug that reports the empty string as ok and parses it as zero. Apparently nobody ever replaced this old call with the faster/saner string2ll() which is used otherwise in the rest of the Redis core. This commit close #3333.
* redis-cli: check SELECT reply type just in state updated.antirez2016-07-051-1/+1
| | | | | | | | | | | | In issues #3361 / #3365 a problem was reported / fixed with redis-cli not updating correctly the current DB on error after SELECT. In theory this bug was fixed in 0042fb0e, but actually the commit only fixed the prompt updating, not the fact the state was set in a wrong way. This commit removes the check in the prompt update, now that hopefully it is the state that is correct, there is no longer need for this check.
* Merge pull request #3365 from sskorgal/unstableSalvatore Sanfilippo2016-07-051-1/+1
|\ | | | | Fix for redis_cli printing default DB when select command fails. #3361
| * Fix for redis_cli printing default DB when select command fails.sskorgal2016-07-011-1/+1
| |
* | Sentinel: fix cross-master Sentinel address update.antirez2016-07-041-2/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit both fixes the crash reported with issue #3364 and also properly closes the old links after the Sentinel address for the other masters gets updated. The two problems where: 1. The Sentinel that switched address may not monitor all the masters, it is possible that there is no match, and the 'match' variable is NULL. Now we check for no match and 'continue' to the next master. 2. By ispecting the code because of issue "1" I noticed that there was a problem in the code that disconnects the link of the Sentinel that needs the address update. Basically link->disconnected is non-zero even if just *a single link* (cc -- command link or pc -- pubsub link) are disconnected, so to check with if (link->disconnected) in order to close the links risks to leave one link connected. I was able to manually reproduce the crash at "1" and verify that the commit resolves the issue. Close #3364.
* | CONFIG GET is now no longer case sensitive.antirez2016-07-041-12/+12
| | | | | | | | Like CONFIG SET always was. Close #3369.
* | Fix test for new RDB checksum failure message.antirez2016-07-041-1/+1
| |
* | Make tcp-keepalive default to 300 in internal conf.antirez2016-07-041-1/+1
| | | | | | | | | | We already changed the default in the redis.conf template, but I forgot to change the internal config as well.
* | In Redis RDB check: more details in error reportings.rdb-checkantirez2016-07-011-13/+21
| |
* | In Redis RDB check: log decompression errors.antirez2016-07-012-2/+23
| |
* | In Redis RDB check: log object type on error.antirez2016-07-011-2/+27
| |
* | Added a trivial program to randomly corrupt RDB files in /utils.antirez2016-07-011-0/+44
| |
* | In Redis RDB check: minor output message changes.antirez2016-07-011-1/+4
| |
* | In Redis RDB check: better error reporting.antirez2016-07-014-14/+71
| |
* | In Redis RDB check: initial POC.antirez2016-06-302-660/+189
|/ | | | | | | | | | | | | | So far we used an external program (later executed within Redis) and parser in order to check RDB files for correctness. This forces, at each RDB format update, to have two copies of the same format implementation that are hard to keep in sync. Morover the former RDB checker only checked the very high-level format of the file, without actually trying to load things in memory. Certain corruptions can only be handled by really loading key-value pairs. This first commit attempts to unify the Redis RDB loadig code with the task of checking the RDB file for correctness. More work is needed but it looks like a sounding direction so far.
* Test: new randomized stress tester for #3343 alike bugs.antirez2016-06-281-2/+27
|
* Stress tester WIP.antirez2016-06-281-0/+3
|
View Lorry Controller Status