diff options
| author | Yossi Gottlieb <yossigo@gmail.com> | 2013-06-23 08:59:01 +0300 |
|---|---|---|
| committer | Yossi Gottlieb <yossigo@gmail.com> | 2013-06-23 08:59:01 +0300 |
| commit | a9a29ff16effa0d36a16c0859c3dadb4ab87889a (patch) | |
| tree | 89e376da1560d6fb16b20d1b570d9a53017458be | |
| parent | 9cc3257e94cdbea94ea259f834409a221f121da4 (diff) | |
| parent | 16ddbb7dfc435f1abb01ecd4df316827be341899 (diff) | |
| download | redis-a9a29ff16effa0d36a16c0859c3dadb4ab87889a.tar.gz | |
Merge upstream 2.6.14.2.6.14-1
61 files changed, 1446 insertions, 716 deletions
diff --git a/.gitignore b/.gitignore index 4eeffaad4..8349f9c47 100644 --- a/.gitignore +++ b/.gitignore @@ -1,8 +1,13 @@ .*.swp *.o -*.rdb *.log -redis-* +dump.rdb +redis-benchmark +redis-check-aof +redis-check-dump +redis-cli +redis-sentinel +redis-server doc-tools release misc/* diff --git a/00-RELEASENOTES b/00-RELEASENOTES index 3306784e5..b92835086 100644 --- a/00-RELEASENOTES +++ b/00-RELEASENOTES @@ -14,6 +14,70 @@ HIGH: There is a critical bug that may affect a subset of users. Upgrade! CRITICAL: There is a critical bug affecting MOST USERS. Upgrade ASAP. -------------------------------------------------------------------------------- +--[ Redis 2.6.14 ] Release date: 20 Jun 2013 + +UPGRADE URGENCY: HIGH because of the following two issues: + * Lua scripting + Replication + AOF in slaves problem (see Issue #1164). + * AOF + expires possible race condition (see Issue #1079). + +* [FIX] AOF bug: expire could be removed from key on AOF rewrite. +* [FIX] Allow writes from scripts called by AOF loading in read-only slaves. +* [FIX] Sentinel: parse new version of INFO replication output correctly. +* [NEW] Reset masterauth if an empty string is configured. + +--[ Redis 2.6.13 ] + +UPGRADE URGENCY: MODERATE, nothing very critical but upgrading is suggested + if you experienced: + + 1) Strange issues with Lua scripting. + 2) Not reconfigured reappearing master using Sentinel. + 3) Server continusly trying to save on save error. + + This version of Redis may also help with AOF and slow / busy + disks and latency issues. + +* [FIX] Throttle BGSAVE attempt on saving error. +* [FIX] redis-cli: raise error on bad command line switch. +* [FIX] Redis/Jemalloc Gitignore were too aggressive. +* [FIX] Test: fix RDB test checking file permissions. +* [FIX] Sentinel: always redirect on master->slave transition. +* [FIX] Lua updated to version 5.1.5. Fixes rare scripting issues. +* [NEW] AOF: improved latency figures with slow/busy disks. +* [NEW] Sentinel: turn old master into a slave when it comes back. +* [NEW] More explicit panic message on out of memory. +* [NEW] redis-cli: --latency-history mode implemented. + +--[ Redis 2.6.12 ] + +UPGRADE URGENCY: MODERATE, nothing very critical but a few non trivial bugs. + +* [BUGFIX] redis-cli --bigkeys: don't crash with empty DB. +* [BUGFIX] stop-writes-on-bgsave-error now works in redis.conf +* [BUGFIX] Don't crash at startup if RDB is there but can't be opened. +* [BUGFIX] Initial value for master_link_down_since_seconds is now huge. +* [BUGFIX] Allow SELECT while loading the DB. +* [BUGFIX] Don't replicate/AOF an empty MULTI/EXEC if the transaction + is empty or containing just read-only commands. +* [BUGFIX] EXPIRE should not be able to resurrect keys (see issue #1026). +* [IMPROVED] Extended SET back ported from Redis 2.8 / unstable + See http://redis.io/commands/set for more information. +* [IMPROVED] Test suite improved. + +--[ Redis 2.6.11 ] + +UPGRADE URGENCY: LOW, however updating is encouraged if you have many instances + per server and you want to lower the CPU / energy usage. + +* [BUGFIX] Replication: more strict error checking for master PING reply. +* [BUGFIX] redis-cli: use keepalive socket option for improved reliability. +* [BUGFIX] Allow AUTH while loading the DB in memory. +* [BUGFIX] Don't segfault on unbalanced quotes while parsing config file. +* [BUGFIX] Don't segfault if command gets propagated to AOF / replication + link as another command name that was renamed in redis.conf +* [IMPROVED] serverCron() frequency is now a runtime parameter (was REDIS_HZ). +* [IMPROVED] Use a lot less CPU when idle, even with many configured DBs. + --[ Redis 2.6.10 ] UPGRADE URGENCY: MODERATE, this release contains many non-critical fixes @@ -4,17 +4,64 @@ Redis Manifesto =============== -1 - A DSL for Abstract Data Types. Redis is a DSL (Domain Specific Language) that manipulates abstract data types and implemented as a TCP daemon. Commands manipulate a key space where keys are binary-safe strings and values are different kinds of abstract data types. Every data type represents an abstract version of a fundamental data structure. For instance Redis Lists are an abstract representation of linked lists. In Redis, the essence of a data type isn't just the kind of operations that the data types support, but also the space and time complexity of the data type and the operations performed upon it. +1 - A DSL for Abstract Data Types. Redis is a DSL (Domain Specific Language) + that manipulates abstract data types and implemented as a TCP daemon. + Commands manipulate a key space where keys are binary-safe strings and + values are different kinds of abstract data types. Every data type + represents an abstract version of a fundamental data structure. For instance + Redis Lists are an abstract representation of linked lists. In Redis, the + essence of a data type isn't just the kind of operations that the data types + support, but also the space and time complexity of the data type and the + operations performed upon it. -2 - Memory storage is #1. The Redis data set, composed of defined key-value pairs, is primarily stored in the computer's memory. The amount of memory in all kinds of computers, including entry-level servers, is increasing significantly each year. Memory is fast, and allows Redis to have very predictable performance. Datasets composed of 10k or 40 millions keys will perform similarly. Complex data types like Redis Sorted Sets are easy to implement and manipulate in memory with good performance, making Redis very simple. Redis will continue to explore alternative options (where data can be optionally stored on disk, say) but the main goal of the project remains the development of an in-memory database. +2 - Memory storage is #1. The Redis data set, composed of defined key-value + pairs, is primarily stored in the computer's memory. The amount of memory in + all kinds of computers, including entry-level servers, is increasing + significantly each year. Memory is fast, and allows Redis to have very + predictable performance. Datasets composed of 10k or 40 millions keys will + perform similarly. Complex data types like Redis Sorted Sets are easy to + implement and manipulate in memory with good performance, making Redis very + simple. Redis will continue to explore alternative options (where data can + be optionally stored on disk, say) but the main goal of the project remains + the development of an in-memory database. -3 - Fundamental data structures for a fundamental API. The Redis API is a direct consequence of fundamental data structures. APIs can often be arbitrary but not an API that resembles the nature of fundamental data structures. If we ever meet intelligent life forms from another part of the universe, they'll likely know, understand and recognize the same basic data structures we have in our computer science books. Redis will avoid intermediate layers in API, so that the complexity is obvious and more complex operations can be performed as the sum of the basic operations. +3 - Fundamental data structures for a fundamental API. The Redis API is a direct + consequence of fundamental data structures. APIs can often be arbitrary but + not an API that resembles the nature of fundamental data structures. If we + ever meet intelligent life forms from another part of the universe, they'll + likely know, understand and recognize the same basic data structures we have + in our computer science books. Redis will avoid intermediate layers in API, + so that the complexity is obvious and more complex operations can be + performed as the sum of the basic operations. -4 - Code is like a poem; it's not just something we write to reach some practical result. Sometimes people that are far from the Redis philosophy suggest using other code written by other authors (frequently in other languages) in order to implement something Redis currently lacks. But to us this is like if Shakespeare decided to end Enrico IV using the Paradiso from the Divina Commedia. Is using any external code a bad idea? Not at all. Like in "One Thousand and One Nights" smaller self contained stories are embedded in a bigger story, we'll be happy to use beautiful self contained libraries when needed. At the same time, when writing the Redis story we're trying to write smaller stories that will fit in to other code. +4 - Code is like a poem; it's not just something we write to reach some + practical result. Sometimes people that are far from the Redis philosophy + suggest using other code written by other authors (frequently in other + languages) in order to implement something Redis currently lacks. But to us + this is like if Shakespeare decided to end Enrico IV using the Paradiso from + the Divina Commedia. Is using any external code a bad idea? Not at all. Like + in "One Thousand and One Nights" smaller self contained stories are embedded + in a bigger story, we'll be happy to use beautiful self contained libraries + when needed. At the same time, when writing the Redis story we're trying to + write smaller stories that will fit in to other code. -5 - We're against complexity. We believe designing systems is a fight against complexity. We'll accept to fight the complexity when it's worthwhile but we'll try hard to recognize when a small feature is not worth 1000s of lines of code. Most of the time the best way to fight complexity is by not creating it at all. +5 - We're against complexity. We believe designing systems is a fight against + complexity. We'll accept to fight the complexity when it's worthwhile but + we'll try hard to recognize when a small feature is not worth 1000s of lines + of code. Most of the time the best way to fight complexity is by not + creating it at all. -6 - Two levels of API. The Redis API has two levels: 1) a subset of the API fits naturally into a distributed version of Redis and 2) a more complex API that supports multi-key operations. Both are useful if used judiciously but there's no way to make the more complex multi-keys API distributed in an opaque way without violating our other principles. We don't want to provide the illusion of something that will work magically when actually it can't in all cases. Instead we'll provide commands to quickly migrate keys from one instance to another to perform multi-key operations and expose the tradeoffs to the user. - -7 - We optimize for joy. We believe writing code is a lot of hard work, and the only way it can be worth is by enjoying it. When there is no longer joy in writing code, the best thing to do is stop. To prevent this, we'll avoid taking paths that will make Redis less of a joy to develop. +6 - Two levels of API. The Redis API has two levels: 1) a subset of the API fits + naturally into a distributed version of Redis and 2) a more complex API that + supports multi-key operations. Both are useful if used judiciously but + there's no way to make the more complex multi-keys API distributed in an + opaque way without violating our other principles. We don't want to provide + the illusion of something that will work magically when actually it can't in + all cases. Instead we'll provide commands to quickly migrate keys from one + instance to another to perform multi-key operations and expose the tradeoffs + to the user. +7 - We optimize for joy. We believe writing code is a lot of hard work, and the + only way it can be worth is by enjoying it. When there is no longer joy in + writing code, the best thing to do is stop. To prevent this, we'll avoid + taking paths that will make Redis less of a joy to develop. @@ -26,9 +26,20 @@ After building Redis is a good idea to test it, using: % make test -NOTE: if after building Redis with a 32 bit target you need to rebuild it - with a 64 bit target you need to perform a "make clean" in the root - directory of the Redis distribution. +Fixing problems building 32 bit binaries +--------- + +If after building Redis with a 32 bit target you need to rebuild it +with a 64 bit target, or the other way around, you need to perform a +"make distclean" in the root directory of the Redis distribution. + +In case of build errors when trying to build a 32 bit binary of Redis, try +the following steps: + +* Install the packages libc6-dev-i386 (also try g++-multilib). +* Try using the following command line instead of "make 32bit": + + make CFLAGS="-m32 -march=native" LDFLAGS="-m32" Allocator --------- diff --git a/deps/jemalloc/.gitignore b/deps/jemalloc/.gitignore index 6607a5fd1..0406413a0 100644 --- a/deps/jemalloc/.gitignore +++ b/deps/jemalloc/.gitignore @@ -2,24 +2,19 @@ /config.stamp /config.log /config.status -/configure /doc/html.xsl /doc/manpages.xsl /doc/jemalloc.xml -/doc/jemalloc.html -/doc/jemalloc.3 /lib/ /Makefile /include/jemalloc/internal/jemalloc_internal\.h /include/jemalloc/internal/size_classes\.h /include/jemalloc/jemalloc\.h /include/jemalloc/jemalloc_defs\.h -/test/jemalloc_test\.h /src/*.[od] /test/*.[od] /test/*.out /test/[a-zA-Z_]* !test/*.c !test/*.exp -/VERSION /bin/jemalloc.sh diff --git a/deps/lua/COPYRIGHT b/deps/lua/COPYRIGHT index 3a53e741e..a86026803 100644 --- a/deps/lua/COPYRIGHT +++ b/deps/lua/COPYRIGHT @@ -9,7 +9,7 @@ For details and rationale, see http://www.lua.org/license.html . =============================================================================== -Copyright (C) 1994-2008 Lua.org, PUC-Rio. +Copyright (C) 1994-2012 Lua.org, PUC-Rio. Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal diff --git a/deps/lua/Makefile b/deps/lua/Makefile index 6e78f66fa..209a13244 100644 --- a/deps/lua/Makefile +++ b/deps/lua/Makefile @@ -48,7 +48,7 @@ TO_MAN= lua.1 luac.1 # Lua version and release. V= 5.1 -R= 5.1.4 +R= 5.1.5 all: $(PLAT) diff --git a/deps/lua/doc/amazon.gif b/deps/lua/doc/amazon.gif Binary files differdeleted file mode 100644 index f2586d576..000000000 --- a/deps/lua/doc/amazon.gif +++ /dev/null diff --git a/deps/lua/doc/contents.html b/deps/lua/doc/contents.html index 8e58e18ca..3d83da98a 100644 --- a/deps/lua/doc/contents.html +++ b/deps/lua/doc/contents.html @@ -3,11 +3,10 @@ <HEAD> <TITLE>Lua 5.1 Reference Manual - contents</TITLE> <LINK REL="stylesheet" TYPE="text/css" HREF="lua.css"> -<META HTTP-EQUIV="content-type" CONTENT="text/html; charset=iso-8859-1"> +<META HTTP-EQUIV="content-type" CONTENT="text/html; charset=utf-8"> <STYLE TYPE="text/css"> ul { list-style-type: none ; - list-style-position: outside ; } </STYLE> </HEAD> @@ -20,7 +19,13 @@ ul { Lua 5.1 Reference Manual </H1> -This is an online version of +<P> +The reference manual is the official definition of the Lua language. +For a complete introduction to Lua programming, see the book +<A HREF="http://www.lua.org/docs.html#pil">Programming in Lua</A>. + +<P> +This manual is also available as a book: <BLOCKQUOTE> <A HREF="http://www.amazon.com/exec/obidos/ASIN/8590379833/lua-indexmanual-20"> <IMG SRC="cover.png" ALT="" TITLE="buy from Amazon" BORDER=1 ALIGN="left" HSPACE=12> @@ -29,128 +34,119 @@ This is an online version of <BR>by R. Ierusalimschy, L. H. de Figueiredo, W. Celes <BR>Lua.org, August 2006 <BR>ISBN 85-903798-3-3 -<BR><A HREF="http://www.amazon.com/exec/obidos/ASIN/8590379833/lua-indexmanual-20"> -<IMG SRC="amazon.gif" ALT="[Buy from Amazon]" BORDER=0></A> <BR CLEAR="all"> </BLOCKQUOTE> -<P> -Buy a copy of this book and +<P> +<A HREF="http://www.amazon.com/exec/obidos/ASIN/8590379833/lua-indexmanual-20">Buy a copy</A> +of this book and <A HREF="http://www.lua.org/donations.html">help to support</A> the Lua project. -<P> -The reference manual is the official definition of the Lua language. -For a complete introduction to Lua programming, see the book -<A HREF="http://www.lua.org/docs.html#books">Programming in Lua</A>. <P> - <A HREF="manual.html">start</A> · <A HREF="#contents">contents</A> · <A HREF="#index">index</A> · -<A HREF="http://www.lua.org/manual/5.1/pt/">português</A> -· -<A HREF="http://www.lua.org/manual/5.1/es/">español</A> +<A HREF="http://www.lua.org/manual/">other versions</A> <HR> <SMALL> -Copyright © 2006-2008 Lua.org, PUC-Rio. +Copyright © 2006–2012 Lua.org, PUC-Rio. Freely available under the terms of the -<a href="http://www.lua.org/license.html#5">Lua license</a>. +<A HREF="http://www.lua.org/license.html">Lua license</A>. </SMALL> -<P> <H2><A NAME="contents">Contents</A></H2> <UL style="padding: 0"> -<LI><A HREF="manual.html">1 - Introduction</A> +<LI><A HREF="manual.html">1 – Introduction</A> <P> -<LI><A HREF="manual.html#2">2 - The Language</A> +<LI><A HREF="manual.html#2">2 – The Language</A> <UL> -<LI><A HREF="manual.html#2.1">2.1 - Lexical Conventions</A> -<LI><A HREF="manual.html#2.2">2.2 - Values and Types</A> +<LI><A HREF="manual.html#2.1">2.1 – Lexical Conventions</A> +<LI><A HREF="manual.html#2.2">2.2 – Values and Types</A> <UL> -<LI><A HREF="manual.html#2.2.1">2.2.1 - Coercion</A> +<LI><A HREF="manual.html#2.2.1">2.2.1 – Coercion</A> </UL> -<LI><A HREF="manual.html#2.3">2.3 - Variables</A> -<LI><A HREF="manual.html#2.4">2.4 - Statements</A> +<LI><A HREF="manual.html#2.3">2.3 – Variables</A> +<LI><A HREF="manual.html#2.4">2.4 – Statements</A> <UL> -<LI><A HREF="manual.html#2.4.1">2.4.1 - Chunks</A> -<LI><A HREF="manual.html#2.4.2">2.4.2 - Blocks</A> -<LI><A HREF="manual.html#2.4.3">2.4.3 - Assignment</A> -<LI><A HREF="manual.html#2.4.4">2.4.4 - Control Structures</A> -<LI><A HREF="manual.html#2.4.5">2.4.5 - For Statement</A> -<LI><A HREF="manual.html#2.4.6">2.4.6 - Function Calls as Statements</A> -<LI><A HREF="manual.html#2.4.7">2.4.7 - Local Declarations</A> +<LI><A HREF="manual.html#2.4.1">2.4.1 – Chunks</A> +<LI><A HREF="manual.html#2.4.2">2.4.2 – Blocks</A> +<LI><A HREF="manual.html#2.4.3">2.4.3 – Assignment</A> +<LI><A HREF="manual.html#2.4.4">2.4.4 – Control Structures</A> +<LI><A HREF="manual.html#2.4.5">2.4.5 – For Statement</A> +<LI><A HREF="manual.html#2.4.6">2.4.6 – Function Calls as Statements</A> +<LI><A HREF="manual.html#2.4.7">2.4.7 – Local Declarations</A> </UL> -<LI><A HREF="manual.html#2.5">2.5 - Expressions</A> +<LI><A HREF="manual.html#2.5">2.5 – Expressions</A> <UL> -<LI><A HREF="manual.html#2.5.1">2.5.1 - Arithmetic Operators</A> -<LI><A HREF="manual.html#2.5.2">2.5.2 - Relational Operators</A> -<LI><A HREF="manual.html#2.5.3">2.5.3 - Logical Operators</A> -<LI><A HREF="manual.html#2.5.4">2.5.4 - Concatenation</A> -<LI><A HREF="manual.html#2.5.5">2.5.5 - The Length Operator</A> -<LI><A HREF="manual.html#2.5.6">2.5.6 - Precedence</A> -<LI><A HREF="manual.html#2.5.7">2.5.7 - Table Constructors</A> -<LI><A HREF="manual.html#2.5.8">2.5.8 - Function Calls</A> -<LI><A HREF="manual.html#2.5.9">2.5.9 - Function Definitions</A> +<LI><A HREF="manual.html#2.5.1">2.5.1 – Arithmetic Operators</A> +<LI><A HREF="manual.html#2.5.2">2.5.2 – Relational Operators</A> +<LI><A HREF="manual.html#2.5.3">2.5.3 – Logical Operators</A> +<LI><A HREF="manual.html#2.5.4">2.5.4 – Concatenation</A> +<LI><A HREF="manual.html#2.5.5">2.5.5 – The Length Operator</A> +<LI><A HREF="manual.html#2.5.6">2.5.6 – Precedence</A> +<LI><A HREF="manual.html#2.5.7">2.5.7 – Table Constructors</A> +<LI><A HREF="manual.html#2.5.8">2.5.8 – Function Calls</A> +<LI><A HREF="manual.html#2.5.9">2.5.9 – Function Definitions</A> </UL> -<LI><A HREF="manual.html#2.6">2.6 - Visibility Rules</A> -<LI><A HREF="manual.html#2.7">2.7 - Error Handling</A> -<LI><A HREF="manual.html#2.8">2.8 - Metatables</A> -<LI><A HREF="manual.html#2.9">2.9 - Environments</A> -<LI><A HREF="manual.html#2.10">2.10 - Garbage Collection</A> +<LI><A HREF="manual.html#2.6">2.6 – Visibility Rules</A> +<LI><A HREF="manual.html#2.7">2.7 – Error Handling</A> +<LI><A HREF="manual.html#2.8">2.8 – Metatables</A> +<LI><A HREF="manual.html#2.9">2.9 – Environments</A> +<LI><A HREF="manual.html#2.10">2.10 – Garbage Collection</A> <UL> -<LI><A HREF="manual.html#2.10.1">2.10.1 - Garbage-Collection Metamethods</A> -<LI><A HREF="manual.html#2.10.2">2.10.2 - Weak Tables</A> +<LI><A HREF="manual.html#2.10.1">2.10.1 – Garbage-Collection Metamethods</A> +<LI><A HREF="manual.html#2.10.2">2.10.2 – Weak Tables</A> </UL> -<LI><A HREF="manual.html#2.11">2.11 - Coroutines</A> +<LI><A HREF="manual.html#2.11">2.11 – Coroutines</A> </UL> <P> -<LI><A HREF="manual.html#3">3 - The Application Program Interface</A> +<LI><A HREF="manual.html#3">3 – The Application Program Interface</A> <UL> -<LI><A HREF="manual.html#3.1">3.1 - The Stack</A> -<LI><A HREF="manual.html#3.2">3.2 - Stack Size</A> -<LI><A HREF="manual.html#3.3">3.3 - Pseudo-Indices</A> -<LI><A HREF="manual.html#3.4">3.4 - C Closures</A> -<LI><A HREF="manual.html#3.5">3.5 - Registry</A> -<LI><A HREF="manual.html#3.6">3.6 - Error Handling in C</A> -<LI><A HREF="manual.html#3.7">3.7 - Functions and Types</A> -<LI><A HREF="manual.html#3.8">3.8 - The Debug Interface</A> +<LI><A HREF="manual.html#3.1">3.1 – The Stack</A> +<LI><A HREF="manual.html#3.2">3.2 – Stack Size</A> +<LI><A HREF="manual.html#3.3">3.3 – Pseudo-Indices</A> +<LI><A HREF="manual.html#3.4">3.4 – C Closures</A> +<LI><A HREF="manual.html#3.5">3.5 – Registry</A> +<LI><A HREF="manual.html#3.6">3.6 – Error Handling in C</A> +<LI><A HREF="manual.html#3.7">3.7 – Functions and Types</A> +<LI><A HREF="manual.html#3.8">3.8 – The Debug Interface</A> </UL> <P> -<LI><A HREF="manual.html#4">4 - The Auxiliary Library</A> +<LI><A HREF="manual.html#4">4 – The Auxiliary Library</A> <UL> -<LI><A HREF="manual.html#4.1">4.1 - Functions and Types</A> +<LI><A HREF="manual.html#4.1">4.1 – Functions and Types</A> </UL> <P> -<LI><A HREF="manual.html#5">5 - Standard Libraries</A> +<LI><A HREF="manual.html#5">5 – Standard Libraries</A> <UL> -<LI><A HREF="manual.html#5.1">5.1 - Basic Functions</A> -<LI><A HREF="manual.html#5.2">5.2 - Coroutine Manipulation</A> -<LI><A HREF="manual.html#5.3">5.3 - Modules</A> -<LI><A HREF="manual.html#5.4">5.4 - String Manipulation</A> +<LI><A HREF="manual.html#5.1">5.1 – Basic Functions</A> +<LI><A HREF="manual.html#5.2">5.2 – Coroutine Manipulation</A> +<LI><A HREF="manual.html#5.3">5.3 – Modules</A> +<LI><A HREF="manual.html#5.4">5.4 – String Manipulation</A> <UL> -<LI><A HREF="manual.html#5.4.1">5.4.1 - Patterns</A> +<LI><A HREF="manual.html#5.4.1">5.4.1 – Patterns</A> </UL> -<LI><A HREF="manual.html#5.5">5.5 - Table Manipulation</A> -<LI><A HREF="manual.html#5.6">5.6 - Mathematical Functions</A> -<LI><A HREF="manual.html#5.7">5.7 - Input and Output Facilities</A> -<LI><A HREF="manual.html#5.8">5.8 - Operating System Facilities</A> -<LI><A HREF="manual.html#5.9">5.9 - The Debug Library</A> +<LI><A HREF="manual.html#5.5">5.5 – Table Manipulation</A> +<LI><A HREF="manual.html#5.6">5.6 – Mathematical Functions</A> +<LI><A HREF="manual.html#5.7">5.7 – Input and Output Facilities</A> +<LI><A HREF="manual.html#5.8">5.8 – Operating System Facilities</A> +<LI><A HREF="manual.html#5.9">5.9 – The Debug Library</A> </UL> <P> -<LI><A HREF="manual.html#6">6 - Lua Stand-alone</A> +<LI><A HREF="manual.html#6">6 – Lua Stand-alone</A> <P> -<LI><A HREF="manual.html#7">7 - Incompatibilities with the Previous Version</A> +<LI><A HREF="manual.html#7">7 – Incompatibilities with the Previous Version</A> <UL> -<LI><A HREF="manual.html#7.1">7.1 - Changes in the Language</A> -<LI><A HREF="manual.html#7.2">7.2 - Changes in the Libraries</A> -<LI><A HREF="manual.html#7.3">7.3 - Changes in the API</A> +<LI><A HREF="manual.html#7.1">7.1 – Changes in the Language</A> +<LI><A HREF="manual.html#7.2">7.2 – Changes in the Libraries</A> +<LI><A HREF="manual.html#7.3">7.3 – Changes in the API</A> </UL> <P> -<LI><A HREF="manual.html#8">8 - The Complete Syntax of Lua</A> +<LI><A HREF="manual.html#8">8 – The Complete Syntax of Lua</A> </UL> <H2><A NAME="index">Index</A></H2> @@ -160,6 +156,8 @@ Freely available under the terms of the <H3><A NAME="functions">Lua functions</A></H3> <A HREF="manual.html#pdf-_G">_G</A><BR> <A HREF="manual.html#pdf-_VERSION">_VERSION</A><BR> +<P> + <A HREF="manual.html#pdf-assert">assert</A><BR> <A HREF="manual.html#pdf-collectgarbage">collectgarbage</A><BR> <A HREF="manual.html#pdf-dofile">dofile</A><BR> @@ -487,12 +485,12 @@ Freely available under the terms of the <P> <HR> -<SMALL> +<SMALL CLASS="footer"> Last update: -Sat Jan 19 13:24:29 BRST 2008 +Mon Feb 13 18:53:32 BRST 2012 </SMALL> <!-- -Last change: revised for Lua 5.1.3 +Last change: revised for Lua 5.1.5 --> </BODY> diff --git a/deps/lua/doc/lua.css b/deps/lua/doc/lua.css index 039cf1169..7fafbb1bb 100644 --- a/deps/lua/doc/lua.css +++ b/deps/lua/doc/lua.css @@ -1,17 +1,37 @@ body { color: #000000 ; background-color: #FFFFFF ; - font-family: sans-serif ; + font-family: Helvetica, Arial, sans-serif ; text-align: justify ; - margin-right: 20px ; - margin-left: 20px ; + margin-right: 30px ; + margin-left: 30px ; } h1, h2, h3, h4 { + font-family: Verdana, Geneva, sans-serif ; font-weight: normal ; font-style: italic ; } +h2 { + padding-top: 0.4em ; + padding-bottom: 0.4em ; + padding-left: 30px ; + padding-right: 30px ; + margin-left: -30px ; + background-color: #E0E0FF ; +} + +h3 { + padding-left: 0.5em ; + border-left: solid #E0E0FF 1em ; +} + +table h3 { + padding-left: 0px ; + border-left: none ; +} + a:link { color: #000080 ; background-color: inherit ; @@ -39,3 +59,25 @@ hr { background-color: #a0a0a0 ; } +:target { + background-color: #F8F8F8 ; + padding: 8px ; + border: solid #a0a0a0 2px ; +} + +.footer { + color: gray ; + font-size: small ; +} + +input[type=text] { + border: solid #a0a0a0 2px ; + border-radius: 2em ; + -moz-border-radius: 2em ; + background-image: url('images/search.png') ; + background-repeat: no-repeat; + background-position: 4px center ; + padding-left: 20px ; + height: 2em ; +} + diff --git a/deps/lua/doc/manual.css b/deps/lua/doc/manual.css index eed5afd9e..b49b36293 100644 --- a/deps/lua/doc/manual.css +++ b/deps/lua/doc/manual.css @@ -1,13 +1,24 @@ h3 code { font-family: inherit ; + font-size: inherit ; } -pre { - font-size: 105% ; +pre, code { + font-size: 12pt ; } span.apii { float: right ; font-family: inherit ; + font-style: normal ; + font-size: small ; + color: gray ; } +p+h1, ul+h1 { + padding-top: 0.4em ; + padding-bottom: 0.4em ; + padding-left: 30px ; + margin-left: -30px ; + background-color: #E0E0FF ; +} diff --git a/deps/lua/doc/manual.html b/deps/lua/doc/manual.html index f46f17c8e..4e41683d0 100644 --- a/deps/lua/doc/manual.html +++ b/deps/lua/doc/manual.html @@ -19,9 +19,9 @@ Lua 5.1 Reference Manual by Roberto Ierusalimschy, Luiz Henrique de Figueiredo, Waldemar Celes <p> <small> -Copyright © 2006-2008 Lua.org, PUC-Rio. +Copyright © 2006–2012 Lua.org, PUC-Rio. Freely available under the terms of the -<a href="http://www.lua.org/license.html#5">Lua license</a>. +<a href="http://www.lua.org/license.html">Lua license</a>. </small> <hr> <p> @@ -29,11 +29,13 @@ Freely available under the terms of the <a href="contents.html#contents">contents</A> · <a href="contents.html#index">index</A> +· +<A HREF="http://www.lua.org/manual/">other versions</A> <!-- ====================================================================== --> <p> -<!-- $Id: manual.of,v 1.48 2008/08/18 15:24:20 roberto Exp $ --> +<!-- $Id: manual.of,v 1.49.1.2 2012/01/13 20:23:26 roberto Exp $ --> @@ -5645,7 +5647,7 @@ Currently, Lua has the following standard libraries: <ul> -<li>basic library,</li> which includes the coroutine sub-library; +<li>basic library, which includes the coroutine sub-library;</li> <li>package library;</li> @@ -5709,7 +5711,7 @@ when absent, it defaults to "assertion failed!" <p> -<hr><h3><a name="pdf-collectgarbage"><code>collectgarbage (opt [, arg])</code></a></h3> +<hr><h3><a name="pdf-collectgarbage"><code>collectgarbage ([opt [, arg]])</code></a></h3> <p> @@ -5718,6 +5720,11 @@ It performs different functions according to its first argument, <code>opt</code <ul> +<li><b>"collect":</b> +performs a full garbage-collection cycle. +This is the default option. +</li> + <li><b>"stop":</b> stops the garbage collector. </li> @@ -5726,10 +5733,6 @@ stops the garbage collector. restarts the garbage collector. </li> -<li><b>"collect":</b> -performs a full garbage-collection cycle. -</li> - <li><b>"count":</b> returns the total memory in use by Lua (in Kbytes). </li> @@ -5760,7 +5763,7 @@ Returns the previous value for <em>step</em>. <p> -<hr><h3><a name="pdf-dofile"><code>dofile (filename)</code></a></h3> +<hr><h3><a name="pdf-dofile"><code>dofile ([filename])</code></a></h3> Opens the named file and executes its contents as a Lua chunk. When called without arguments, <code>dofile</code> executes the contents of the standard input (<code>stdin</code>). @@ -8421,7 +8424,7 @@ Otherwise, it returns the name of the upvalue. <p> -<hr><h3><a name="pdf-debug.traceback"><code>debug.traceback ([thread,] [message] [, level])</code></a></h3> +<hr><h3><a name="pdf-debug.traceback"><code>debug.traceback ([thread,] [message [, level]])</code></a></h3> <p> @@ -8789,12 +8792,12 @@ Here is the complete syntax of Lua in extended BNF. <HR> -<SMALL> +<SMALL CLASS="footer"> Last update: -Mon Aug 18 13:25:46 BRT 2008 +Mon Feb 13 18:54:19 BRST 2012 </SMALL> <!-- -Last change: revised for Lua 5.1.4 +Last change: revised for Lua 5.1.5 --> </body></html> diff --git a/deps/lua/doc/readme.html b/deps/lua/doc/readme.html index 38be6dbbf..3ed6a8189 100644 --- a/deps/lua/doc/readme.html +++ b/deps/lua/doc/readme.html @@ -12,7 +12,7 @@ Documentation </H1> -This is the documentation included in the source distribution of Lua 5.1.4. +This is the documentation included in the source distribution of Lua 5.1.5. <UL> <LI><A HREF="contents.html">Reference manual</A> @@ -33,7 +33,7 @@ especially the <HR> <SMALL> Last update: -Tue Aug 12 14:46:07 BRT 2008 +Fri Feb 3 09:44:42 BRST 2012 </SMALL> </BODY> diff --git a/deps/lua/etc/lua.pc b/deps/lua/etc/lua.pc index f52f55b01..07e2852b0 100644 --- a/deps/lua/etc/lua.pc +++ b/deps/lua/etc/lua.pc @@ -5,7 +5,7 @@ # grep '^V=' ../Makefile V= 5.1 # grep '^R=' ../Makefile -R= 5.1.4 +R= 5.1.5 # grep '^INSTALL_.*=' ../Makefile | sed 's/INSTALL_TOP/prefix/' prefix= /usr/local diff --git a/deps/lua/src/Makefile b/deps/lua/src/Makefile index 77d6a48b4..cbbb7d55f 100644 --- a/deps/lua/src/Makefile +++ b/deps/lua/src/Makefile @@ -48,7 +48,7 @@ o: $(ALL_O) a: $(ALL_A) $(LUA_A): $(CORE_O) $(LIB_O) - $(AR) $@ $? + $(AR) $@ $(CORE_O) $(LIB_O) # DLL needs all object files $(RANLIB) $@ $(LUA_T): $(LUA_O) $(LUA_A) diff --git a/deps/lua/src/lbaselib.c b/deps/lua/src/lbaselib.c index 2a4c079d3..2ab550bd4 100644 --- a/deps/lua/src/lbaselib.c +++ b/deps/lua/src/lbaselib.c @@ -631,7 +631,7 @@ static void base_open (lua_State *L) { luaL_register(L, "_G", base_funcs); lua_pushliteral(L, LUA_VERSION); lua_setglobal(L, "_VERSION"); /* set global _VERSION */ - /* `ipairs' and `pairs' need auxliliary functions as upvalues */ + /* `ipairs' and `pairs' need auxiliary functions as upvalues */ auxopen(L, "ipairs", luaB_ipairs, ipairsaux); auxopen(L, "pairs", luaB_pairs, luaB_next); /* `newproxy' needs a weaktable as upvalue */ diff --git a/deps/lua/src/lcode.c b/deps/lua/src/lcode.c index cff626b7f..679cb9cfd 100644 --- a/deps/lua/src/lcode.c +++ b/deps/lua/src/lcode.c @@ -1,5 +1,5 @@ /* -** $Id: lcode.c,v 2.25.1.3 2007/12/28 15:32:23 roberto Exp $ +** $Id: lcode.c,v 2.25.1.5 2011/01/31 14:53:16 roberto Exp $ ** Code generator for Lua ** See Copyright Notice in lua.h */ @@ -544,10 +544,6 @@ void luaK_goiftrue (FuncState *fs, expdesc *e) { pc = NO_JUMP; /* always true; do nothing */ break; } - case VFALSE: { - pc = luaK_jump(fs); /* always jump */ - break; - } case VJMP: { invertjump(fs, e); pc = e->u.s.info; @@ -572,10 +568,6 @@ static void luaK_goiffalse (FuncState *fs, expdesc *e) { pc = NO_JUMP; /* always false; do nothing */ break; } - case VTRUE: { - pc = luaK_jump(fs); /* always jump */ - break; - } case VJMP: { pc = e->u.s.info; break; diff --git a/deps/lua/src/ldblib.c b/deps/lua/src/ldblib.c index 67de1222a..2027eda59 100644 --- a/deps/lua/src/ldblib.c +++ b/deps/lua/src/ldblib.c @@ -1,5 +1,5 @@ /* -** $Id: ldblib.c,v 1.104.1.3 2008/01/21 13:11:21 roberto Exp $ +** $Id: ldblib.c,v 1.104.1.4 2009/08/04 18:50:18 roberto Exp $ ** Interface from Lua to its debug API ** See Copyright Notice in lua.h */ @@ -45,6 +45,7 @@ static int db_setmetatable (lua_State *L) { static int db_getfenv (lua_State *L) { + luaL_checkany(L, 1); lua_getfenv(L, 1); return 1; } diff --git a/deps/lua/src/ldo.c b/deps/lua/src/ldo.c index 8de05f728..d1bf786cb 100644 --- a/deps/lua/src/ldo.c +++ b/deps/lua/src/ldo.c @@ -1,5 +1,5 @@ /* -** $Id: ldo.c,v 2.38.1.3 2008/01/18 22:31:22 roberto Exp $ +** $Id: ldo.c,v 2.38.1.4 2012/01/18 02:27:10 roberto Exp $ ** Stack and Call structure of Lua ** See Copyright Notice in lua.h */ @@ -217,6 +217,7 @@ static StkId adjust_varargs (lua_State *L, Proto *p, int actual) { int nvar = actual - nfixargs; /* number of extra arguments */ lua_assert(p->is_vararg & VARARG_HASARG); luaC_checkGC(L); + luaD_checkstack(L, p->maxstacksize); htab = luaH_new(L, nvar, 1); /* create `arg' table */ for (i=0; i<nvar; i++) /* put extra arguments into `arg' table */ setobj2n(L, luaH_setnum(L, htab, i+1), L->top - nvar + i); diff --git a/deps/lua/src/lgc.c b/deps/lua/src/lgc.c index d9e0b7829..e909c79a9 100644 --- a/deps/lua/src/lgc.c +++ b/deps/lua/src/lgc.c @@ -1,5 +1,5 @@ /* -** $Id: lgc.c,v 2.38.1.1 2007/12/27 13:02:25 roberto Exp $ +** $Id: lgc.c,v 2.38.1.2 2011/03/18 18:05:38 roberto Exp $ ** Garbage Collector ** See Copyright Notice in lua.h */ @@ -627,7 +627,6 @@ void luaC_step (lua_State *L) { } } else { - lua_assert(g->totalbytes >= g->estimate); setthreshold(g); } } diff --git a/deps/lua/src/liolib.c b/deps/lua/src/liolib.c index e79ed1cb2..649f9a595 100644 --- a/deps/lua/src/liolib.c +++ b/deps/lua/src/liolib.c @@ -1,5 +1,5 @@ /* -** $Id: liolib.c,v 2.73.1.3 2008/01/18 17:47:43 roberto Exp $ +** $Id: liolib.c,v 2.73.1.4 2010/05/14 15:33:51 roberto Exp $ ** Standard I/O (and system) library ** See Copyright Notice in lua.h */ @@ -276,7 +276,10 @@ static int read_number (lua_State *L, FILE *f) { lua_pushnumber(L, d); return 1; } - else return 0; /* read fails */ + else { + lua_pushnil(L); /* "result" to be removed */ + return 0; /* read fails */ + } } diff --git a/deps/lua/src/llex.c b/deps/lua/src/llex.c index 6dc319358..88c6790c0 100644 --- a/deps/lua/src/llex.c +++ b/deps/lua/src/llex.c @@ -1,5 +1,5 @@ /* -** $Id: llex.c,v 2.20.1.1 2007/12/27 13:02:25 roberto Exp $ +** $Id: llex.c,v 2.20.1.2 2009/11/23 14:58:22 roberto Exp $ ** Lexical Analyzer ** See Copyright Notice in lua.h */ @@ -118,8 +118,10 @@ TString *luaX_newstring (LexState *ls, const char *str, size_t l) { lua_State *L = ls->L; TString *ts = luaS_newlstr(L, str, l); TValue *o = luaH_setstr(L, ls->fs->h, ts); /* entry for `str' */ - if (ttisnil(o)) + if (ttisnil(o)) { setbvalue(o, 1); /* make sure `str' will not be collected */ + luaC_checkGC(L); + } return ts; } diff --git a/deps/lua/src/loadlib.c b/deps/lua/src/loadlib.c index 0d401eba1..6158c5353 100644 --- a/deps/lua/src/loadlib.c +++ b/deps/lua/src/loadlib.c @@ -1,5 +1,5 @@ /* -** $Id: loadlib.c,v 1.52.1.3 2008/08/06 13:29:28 roberto Exp $ +** $Id: loadlib.c,v 1.52.1.4 2009/09/09 13:17:16 roberto Exp $ ** Dynamic library loader for Lua ** See Copyright Notice in lua.h ** @@ -639,7 +639,7 @@ LUALIB_API int luaopen_package (lua_State *L) { lua_pushvalue(L, -1); lua_replace(L, LUA_ENVIRONINDEX); /* create `loaders' table */ - lua_createtable(L, 0, sizeof(loaders)/sizeof(loaders[0]) - 1); + lua_createtable(L, sizeof(loaders)/sizeof(loaders[0]) - 1, 0); /* fill it with pre-defined loaders */ for (i=0; loaders[i] != NULL; i++) { lua_pushcfunction(L, loaders[i]); diff --git a/deps/lua/src/lparser.c b/deps/lua/src/lparser.c index 1e2a9a88b..dda7488dc 100644 --- a/deps/lua/src/lparser.c +++ b/deps/lua/src/lparser.c @@ -1,5 +1,5 @@ /* -** $Id: lparser.c,v 2.42.1.3 2007/12/28 15:32:23 roberto Exp $ +** $Id: lparser.c,v 2.42.1.4 2011/10/21 19:31:42 roberto Exp $ ** Lua Parser ** See Copyright Notice in lua.h */ @@ -374,9 +374,9 @@ static void close_func (LexState *ls) { lua_assert(luaG_checkcode(f)); lua_assert(fs->bl == NULL); ls->fs = fs->prev; - L->top -= 2; /* remove table and prototype from the stack */ /* last token read was anchored in defunct function; must reanchor it */ if (fs) anchor_token(ls); + L->top -= 2; /* remove table and prototype from the stack */ } diff --git a/deps/lua/src/lstrlib.c b/deps/lua/src/lstrlib.c index 1b4763d4e..7a03489be 100644 --- a/deps/lua/src/lstrlib.c +++ b/deps/lua/src/lstrlib.c @@ -1,5 +1,5 @@ /* -** $Id: lstrlib.c,v 1.132.1.4 2008/07/11 17:27:21 roberto Exp $ +** $Id: lstrlib.c,v 1.132.1.5 2010/05/14 15:34:19 roberto Exp $ ** Standard library for string operations and pattern-matching ** See Copyright Notice in lua.h */ @@ -754,6 +754,7 @@ static void addintlen (char *form) { static int str_format (lua_State *L) { + int top = lua_gettop(L); int arg = 1; size_t sfl; const char *strfrmt = luaL_checklstring(L, arg, &sfl); @@ -768,7 +769,8 @@ static int str_format (lua_State *L) { else { /* format item */ char form[MAX_FORMAT]; /* to store the format (`%...') */ char buff[MAX_ITEM]; /* to store the formatted item */ - arg++; + if (++arg > top) + luaL_argerror(L, arg, "no value"); strfrmt = scanformat(L, strfrmt, form); switch (*strfrmt++) { case 'c': { diff --git a/deps/lua/src/lua.h b/deps/lua/src/lua.h index e4bdfd3b9..a4b73e743 100644 --- a/deps/lua/src/lua.h +++ b/deps/lua/src/lua.h @@ -1,5 +1,5 @@ /* -** $Id: lua.h,v 1.218.1.5 2008/08/06 13:30:12 roberto Exp $ +** $Id: lua.h,v 1.218.1.7 2012/01/13 20:36:20 roberto Exp $ ** Lua - An Extensible Extension Language ** Lua.org, PUC-Rio, Brazil (http://www.lua.org) ** See Copyright Notice at the end of this file @@ -17,9 +17,9 @@ #define LUA_VERSION "Lua 5.1" -#define LUA_RELEASE "Lua 5.1.4" +#define LUA_RELEASE "Lua 5.1.5" #define LUA_VERSION_NUM 501 -#define LUA_COPYRIGHT "Copyright (C) 1994-2008 Lua.org, PUC-Rio" +#define LUA_COPYRIGHT "Copyright (C) 1994-2012 Lua.org, PUC-Rio" #define LUA_AUTHORS "R. Ierusalimschy, L. H. de Figueiredo & W. Celes" @@ -362,7 +362,7 @@ struct lua_Debug { /****************************************************************************** -* Copyright (C) 1994-2008 Lua.org, PUC-Rio. All rights reserved. +* Copyright (C) 1994-2012 Lua.org, PUC-Rio. All rights reserved. * * Permission is hereby granted, free of charge, to any person obtaining * a copy of this software and associated documentation files (the diff --git a/deps/lua/src/lvm.c b/deps/lua/src/lvm.c index ee3256ab9..e0a0cd852 100644 --- a/deps/lua/src/lvm.c +++ b/deps/lua/src/lvm.c @@ -1,5 +1,5 @@ /* -** $Id: lvm.c,v 2.63.1.3 2007/12/28 15:32:23 roberto Exp $ +** $Id: lvm.c,v 2.63.1.5 2011/08/17 20:43:11 roberto Exp $ ** Lua virtual machine ** See Copyright Notice in lua.h */ @@ -133,6 +133,7 @@ void luaV_gettable (lua_State *L, const TValue *t, TValue *key, StkId val) { void luaV_settable (lua_State *L, const TValue *t, TValue *key, StkId val) { int loop; + TValue temp; for (loop = 0; loop < MAXTAGLOOP; loop++) { const TValue *tm; if (ttistable(t)) { /* `t' is a table? */ @@ -141,6 +142,7 @@ void luaV_settable (lua_State *L, const TValue *t, TValue *key, StkId val) { if (!ttisnil(oldval) || /* result is no nil? */ (tm = fasttm(L, h->metatable, TM_NEWINDEX)) == NULL) { /* or no TM? */ setobj2t(L, oldval, val); + h->flags = 0; luaC_barriert(L, h, val); return; } @@ -152,7 +154,9 @@ void luaV_settable (lua_State *L, const TValue *t, TValue *key, StkId val) { callTM(L, tm, t, key, val); return; } - t = tm; /* else repeat with `tm' */ + /* else repeat with `tm' */ + setobj(L, &temp, tm); /* avoid pointing inside table (may rehash) */ + t = &temp; } luaG_runerror(L, "loop in settable"); } diff --git a/redis.conf b/redis.conf index 24e0372f4..dfeca7cf7 100755 --- a/redis.conf +++ b/redis.conf @@ -182,7 +182,7 @@ dir ./ # still reply to client requests, possibly with out of date data, or the # data set may just be empty if this is the first synchronization. # -# 2) if slave-serve-stale data is set to 'no' the slave will reply with +# 2) if slave-serve-stale-data is set to 'no' the slave will reply with # an error "SYNC with master in progress" to all the kind of commands # but to INFO and SLAVEOF. # @@ -283,6 +283,9 @@ slave-priority 100 # an empty string: # # rename-command CONFIG "" +# +# Please note that changing the name of commands that are logged into the +# AOF file or transmitted to slaves may cause problems. ################################### LIMITS #################################### @@ -577,6 +580,32 @@ client-output-buffer-limit normal 0 0 0 client-output-buffer-limit slave 256mb 64mb 60 client-output-buffer-limit pubsub 32mb 8mb 60 +# Redis calls an internal function to perform many background tasks, like +# closing connections of clients in timeot, purging expired keys that are +# never requested, and so forth. +# +# Not all tasks are perforemd with the same frequency, but Redis checks for +# tasks to perform accordingly to the specified "hz" value. +# +# By default "hz" is set to 10. Raising the value will use more CPU when +# Redis is idle, but at the same time will make Redis more responsive when +# there are many keys expiring at the same time, and timeouts may be +# handled with more precision. +# +# The range is between 1 and 500, however a value over 100 is usually not +# a good idea. Most users should use the default of 10 and raise this up to +# 100 only in environments where very low latency is required. +hz 10 + +# When a child rewrites the AOF file, if the following option is enabled +# the file will be fsync-ed every 32 MB of data generated. This is useful +# in order to commit the file to the disk more incrementally and avoid +# big latency spikes. +aof-rewrite-incremental-fsync yes + +# Same as aof-rewrite-incremental-fsync, but for RDB files. +rdb-incremental-fsync yes + ################################## INCLUDES ################################### # Include one or more other config files here. This is useful if you @@ -863,7 +863,9 @@ int rewriteAppendOnlyFile(char *filename) { return REDIS_ERR; } - rioInitWithFileAndFsyncInterval(&aof,fp, 1024*1024*16); + rioInitWithFile(&aof,fp); + if (server.aof_rewrite_incremental_fsync) + rioSetAutoSync(&aof,REDIS_AOF_AUTOSYNC_BYTES); for (j = 0; j < server.dbnum; j++) { char selectcmd[] = "*2\r\n$6\r\nSELECT\r\n"; redisDb *db = server.db+j; @@ -891,6 +893,9 @@ int rewriteAppendOnlyFile(char *filename) { expiretime = getExpire(db,&key); + /* If this key is already expired skip it */ + if (expiretime != -1 && expiretime < now) continue; + /* Save the key and associated value */ if (o->type == REDIS_STRING) { /* Emit a SET command */ @@ -913,8 +918,6 @@ int rewriteAppendOnlyFile(char *filename) { /* Save the expire time */ if (expiretime != -1) { char cmd[]="*3\r\n$9\r\nPEXPIREAT\r\n"; - /* If this key is already expired skip it */ - if (expiretime < now) continue; if (rioWrite(&aof,cmd,sizeof(cmd)-1) == 0) goto werr; if (rioWriteBulkObject(&aof,&key) == 0) goto werr; if (rioWriteBulkLongLong(&aof,expiretime) == 0) goto werr; diff --git a/src/bitops.c b/src/bitops.c index 75f3317a9..926d1eacd 100644 --- a/src/bitops.c +++ b/src/bitops.c @@ -58,8 +58,8 @@ static int getBitOffsetFromArgument(redisClient *c, robj *o, size_t *offset) { /* Count number of bits set in the binary array pointed by 's' and long * 'count' bytes. The implementation of this function is required to * work with a input string length up to 512 MB. */ -long popcount(void *s, long count) { - long bits = 0; +size_t popcount(void *s, long count) { + size_t bits = 0; unsigned char *p; uint32_t *p4 = s; static const unsigned char bitsinbyte[256] = {0,1,1,2,1,2,2,3,1,2,2,3,2,3,3,4,1,2,2,3,2,3,3,4,2,3,3,4,3,4,4,5,1,2,2,3,2,3,3,4,2,3,3,4,3,4,4,5,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,1,2,2,3,2,3,3,4,2,3,3,4,3,4,4,5,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,3,4,4,5,4,5,5,6,4,5,5,6,5,6,6,7,1,2,2,3,2,3,3,4,2,3,3,4,3,4,4,5,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,3,4,4,5,4,5,5,6,4,5,5,6,5,6,6,7,2,3,3,4,3,4,4,5,3,4,4,5,4,5,5,6,3,4,4,5,4,5,5,6,4,5,5,6,5,6,6,7,3,4,4,5,4,5,5,6,4,5,5,6,5,6,6,7,4,5,5,6,5,6,6,7,5,6,6,7,6,7,7,8}; diff --git a/src/config.c b/src/config.c index dbea5513d..711162cfd 100755 --- a/src/config.c +++ b/src/config.c @@ -68,11 +68,21 @@ void loadServerConfigFromString(char *config) { linenum = i+1; lines[i] = sdstrim(lines[i]," \t\r\n"); - /* Skip comments and blank lines*/ + /* Skip comments and blank lines */ if (lines[i][0] == '#' || lines[i][0] == '\0') continue; /* Split into arguments */ argv = sdssplitargs(lines[i],&argc); + if (argv == NULL) { + err = "Unbalanced quotes in configuration line"; + goto loaderr; + } + + /* Skip this line if the resulting command vector is empty. */ + if (argc == 0) { + sdsfreesplitres(argv,argc); + return; + } sdstolower(argv[0]); /* Execute config directives */ @@ -281,6 +291,10 @@ void loadServerConfigFromString(char *config) { if ((server.conditional_sync = yesnotoi(argv[1])) == -1) { err = "argument must be 'yes' or 'no'"; goto loaderr; } + } else if (!strcasecmp(argv[0],"hz") && argc == 2) { + server.hz = atoi(argv[1]); + if (server.hz < REDIS_MIN_HZ) server.hz = REDIS_MIN_HZ; + if (server.hz > REDIS_MAX_HZ) server.hz = REDIS_MAX_HZ; } else if (!strcasecmp(argv[0],"appendonly") && argc == 2) { int yes; @@ -319,6 +333,18 @@ void loadServerConfigFromString(char *config) { argc == 2) { server.aof_rewrite_min_size = memtoll(argv[1],NULL); + } else if (!strcasecmp(argv[0],"aof-rewrite-incremental-fsync") && + argc == 2) + { + if ((server.aof_rewrite_incremental_fsync = yesnotoi(argv[1])) == -1) { + err = "argument must be 'yes' or 'no'"; goto loaderr; + } + } else if (!strcasecmp(argv[0],"rdb-incremental-fsync") && + argc == 2) + { + if ((server.rdb_incremental_fsync = yesnotoi(argv[1])) == -1) { + err = "argument must be 'yes' or 'no'"; goto loaderr; + } } else if (!strcasecmp(argv[0],"requirepass") && argc == 2) { if (strlen(argv[1]) > REDIS_AUTHPASS_MAX_LEN) { err = "Password is longer than REDIS_AUTHPASS_MAX_LEN"; @@ -495,7 +521,7 @@ void configSetCommand(redisClient *c) { server.requirepass = ((char*)o->ptr)[0] ? zstrdup(o->ptr) : NULL; } else if (!strcasecmp(c->argv[2]->ptr,"masterauth")) { zfree(server.masterauth); - server.masterauth = zstrdup(o->ptr); + server.masterauth = ((char*)o->ptr)[0] ? zstrdup(o->ptr) : NULL; } else if (!strcasecmp(c->argv[2]->ptr,"maxmemory")) { if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt; @@ -506,6 +532,12 @@ void configSetCommand(redisClient *c) { } freeMemoryIfNeeded(server.maxmemory); } + } else if (!strcasecmp(c->argv[2]->ptr,"hz")) { + if (getLongLongFromObject(o,&ll) == REDIS_ERR || + ll < 0) goto badfmt; + server.hz = (int) ll; + if (server.hz < REDIS_MIN_HZ) server.hz = REDIS_MIN_HZ; + if (server.hz > REDIS_MAX_HZ) server.hz = REDIS_MAX_HZ; } else if (!strcasecmp(c->argv[2]->ptr,"maxmemory-policy")) { if (!strcasecmp(o->ptr,"volatile-lru")) { server.maxmemory_policy = REDIS_MAXMEMORY_VOLATILE_LRU; @@ -568,6 +600,16 @@ void configSetCommand(redisClient *c) { } else if (!strcasecmp(c->argv[2]->ptr,"auto-aof-rewrite-min-size")) { if (getLongLongFromObject(o,&ll) == REDIS_ERR || ll < 0) goto badfmt; server.aof_rewrite_min_size = ll; + } else if (!strcasecmp(c->argv[2]->ptr,"aof-rewrite-incremental-fsync")) { + int yn = yesnotoi(o->ptr); + + if (yn == -1) goto badfmt; + server.aof_rewrite_incremental_fsync = yn; + } else if (!strcasecmp(c->argv[2]->ptr,"rdb-incremental-fsync")) { + int yn = yesnotoi(o->ptr); + + if (yn == -1) goto badfmt; + server.rdb_incremental_fsync = yn; } else if (!strcasecmp(c->argv[2]->ptr,"save")) { int vlen, j; sds *v = sdssplitlen(o->ptr,sdslen(o->ptr)," ",1,&vlen); @@ -792,7 +834,7 @@ void configGetCommand(redisClient *c) { config_get_string_field("dbfilename",server.rdb_filename); config_get_string_field("syncdbfilename",server.rdb_syncfilename); config_get_string_field("requirepass",server.requirepass); - config_get_string_field("masterauth",server.requirepass); + config_get_string_field("masterauth",server.masterauth); config_get_string_field("bind",server.bindaddr); config_get_string_field("unixsocket",server.unixsocket); config_get_string_field("logfile",server.logfile); @@ -833,6 +875,7 @@ void configGetCommand(redisClient *c) { config_get_numerical_field("maxclients",server.maxclients); config_get_numerical_field("watchdog-period",server.watchdog_period); config_get_numerical_field("slave-priority",server.slave_priority); + config_get_numerical_field("hz",server.hz); /* Bool (yes/no) values */ config_get_bool_field("no-appendfsync-on-rewrite", @@ -849,6 +892,11 @@ void configGetCommand(redisClient *c) { config_get_bool_field("activerehashing", server.activerehashing); config_get_bool_field("repl-disable-tcp-nodelay", server.repl_disable_tcp_nodelay); + config_get_bool_field("aof-rewrite-incremental-fsync", + server.aof_rewrite_incremental_fsync); + config_get_bool_field("rdb-incremental-fsync", + server.rdb_incremental_fsync); + /* Everything we can't handle with macros follows. */ @@ -551,7 +551,6 @@ int expireIfNeeded(redisDb *db, robj *key) { * unit is either UNIT_SECONDS or UNIT_MILLISECONDS, and is only used for * the argv[2] parameter. The basetime is always specified in milliseconds. */ void expireGenericCommand(redisClient *c, long long basetime, int unit) { - dictEntry *de; robj *key = c->argv[1], *param = c->argv[2]; long long when; /* unix time in milliseconds when the key will expire. */ @@ -561,11 +560,12 @@ void expireGenericCommand(redisClient *c, long long basetime, int unit) { if (unit == UNIT_SECONDS) when *= 1000; when += basetime; - de = dictFind(c->db->dict,key->ptr); - if (de == NULL) { + /* No key, return zero. */ + if (lookupKeyRead(c->db,key) == NULL) { addReply(c,shared.czero); return; } + /* EXPIRE with negative TTL, or EXPIREAT with a timestamp into the past * should never be executed as a DEL when load the AOF or in the context * of a slave instance. diff --git a/src/debug.c b/src/debug.c index a74358383..e1d42cc65 100644 --- a/src/debug.c +++ b/src/debug.c @@ -330,9 +330,14 @@ void debugCommand(redisClient *c) { usleep(utime); addReply(c,shared.ok); + } else if (!strcasecmp(c->argv[1]->ptr,"set-active-expire") && + c->argc == 3) + { + server.active_expire_enabled = atoi(c->argv[2]->ptr); + addReply(c,shared.ok); } else { - addReplyError(c, - "Syntax error, try DEBUG [SEGFAULT|OBJECT <key>|SWAPIN <key>|SWAPOUT <key>|RELOAD]"); + addReplyErrorFormat(c, "Unknown DEBUG subcommand or wrong number of arguments for '%s'", + (char*)c->argv[1]->ptr); } } @@ -382,9 +387,12 @@ void redisLogObjectDebugInfo(robj *o) { redisLog(REDIS_WARNING,"Object encoding: %d", o->encoding); redisLog(REDIS_WARNING,"Object refcount: %d", o->refcount); if (o->type == REDIS_STRING && o->encoding == REDIS_ENCODING_RAW) { - redisLog(REDIS_WARNING,"Object raw string len: %d", sdslen(o->ptr)); - if (sdslen(o->ptr) < 4096) - redisLog(REDIS_WARNING,"Object raw string content: \"%s\"", (char*)o->ptr); + redisLog(REDIS_WARNING,"Object raw string len: %zu", sdslen(o->ptr)); + if (sdslen(o->ptr) < 4096) { + sds repr = sdscatrepr(sdsempty(),o->ptr,sdslen(o->ptr)); + redisLog(REDIS_WARNING,"Object raw string content: %s", repr); + sdsfree(repr); + } } else if (o->type == REDIS_LIST) { redisLog(REDIS_WARNING,"List length: %d", (int) listTypeLength(o)); } else if (o->type == REDIS_SET) { @@ -794,7 +802,7 @@ void enableWatchdog(int period) { /* If the configured period is smaller than twice the timer period, it is * too short for the software watchdog to work reliably. Fix it now * if needed. */ - min_period = (1000/REDIS_HZ)*2; + min_period = (1000/server.hz)*2; if (period < min_period) period = min_period; watchdogScheduleSignal(period); /* Adjust the current timer. */ server.watchdog_period = period; diff --git a/src/help.h b/src/help.h index 1378b8b85..60a5726bf 100644 --- a/src/help.h +++ b/src/help.h @@ -69,6 +69,11 @@ struct commandHelp { "Pop a value from a list, push it to another list and return it; or block until one is available", 2, "2.2.0" }, + { "CLIENT GETNAME", + "-", + "Get the current connection name", + 9, + "2.6.9" }, { "CLIENT KILL", "ip:port", "Kill the connection of a client", @@ -79,6 +84,11 @@ struct commandHelp { "Get the list of client connections", 9, "2.4.0" }, + { "CLIENT SETNAME", + "connection-name", + "Set the current connection name", + 9, + "2.6.9" }, { "CONFIG GET", "parameter", "Get the value of a configuration parameter", @@ -280,7 +290,7 @@ struct commandHelp { 1, "2.6.0" }, { "INFO", - "-", + "[section]", "Get information and statistics about the server", 9, "1.0.0" }, @@ -525,7 +535,7 @@ struct commandHelp { 8, "1.0.0" }, { "SET", - "key value", + "key value [EX seconds] [PX milliseconds] [NX|XX]", "Set the string value of a key", 1, "1.0.0" }, @@ -665,7 +675,7 @@ struct commandHelp { 7, "2.2.0" }, { "ZADD", - "key score member [score] [member]", + "key score member [score member ...]", "Add one or more members to a sorted set, or update its score if it already exists", 4, "1.2.0" }, diff --git a/src/mkreleasehdr.sh b/src/mkreleasehdr.sh index dbf948c8a..386bd24d1 100755 --- a/src/mkreleasehdr.sh +++ b/src/mkreleasehdr.sh @@ -1,6 +1,6 @@ #!/bin/sh GIT_SHA1=`(git show-ref --head --hash=8 2> /dev/null || echo 00000000) | head -n1` -GIT_DIRTY=`git diff 2> /dev/null | wc -l` +GIT_DIRTY=`git diff --no-ext-diff 2> /dev/null | wc -l` test -f release.h || touch release.h (cat release.h | grep SHA1 | grep $GIT_SHA1) && \ (cat release.h | grep DIRTY | grep $GIT_DIRTY) && exit 0 # Already up-to-date diff --git a/src/multi.c b/src/multi.c index 2b7e6b552..cd07971e2 100755 --- a/src/multi.c +++ b/src/multi.c @@ -103,13 +103,11 @@ void discardCommand(redisClient *c) { /* Send a MULTI command to all the slaves and AOF file. Check the execCommand * implementation for more information. */ -void execCommandReplicateMulti(redisClient *c) { +void execCommandPropagateMulti(redisClient *c) { robj *multistring = createStringObject("MULTI",5); - if (server.aof_state != REDIS_AOF_OFF) - feedAppendOnlyFile(server.multiCommand,c->db->id,&multistring,1); - if (listLength(server.slaves)) - replicationFeedSlaves(server.slaves,c->db->id,&multistring,1); + propagate(server.multiCommand,c->db->id,&multistring,1, + REDIS_PROPAGATE_AOF|REDIS_PROPAGATE_REPL); decrRefCount(multistring); } @@ -118,6 +116,7 @@ void execCommand(redisClient *c) { robj **orig_argv; int orig_argc; struct redisCommand *orig_cmd; + int must_propagate = 0; /* Need to propagate MULTI/EXEC to AOF / slaves? */ if (!(c->flags & REDIS_MULTI)) { addReplyError(c,"EXEC without MULTI"); @@ -133,19 +132,10 @@ void execCommand(redisClient *c) { if (c->flags & (REDIS_DIRTY_CAS|REDIS_DIRTY_EXEC)) { addReply(c, c->flags & REDIS_DIRTY_EXEC ? shared.execaborterr : shared.nullmultibulk); - freeClientMultiState(c); - initClientMultiState(c); - c->flags &= ~(REDIS_MULTI|REDIS_DIRTY_CAS|REDIS_DIRTY_EXEC); - unwatchAllKeys(c); + discardTransaction(c); goto handle_monitor; } - /* Replicate a MULTI request now that we are sure the block is executed. - * This way we'll deliver the MULTI/..../EXEC block as a whole and - * both the AOF and the replication link will have the same consistency - * and atomicity guarantees. */ - execCommandReplicateMulti(c); - /* Exec all the queued commands */ unwatchAllKeys(c); /* Unwatch ASAP otherwise we'll waste CPU cycles */ orig_argv = c->argv; @@ -156,6 +146,16 @@ void execCommand(redisClient *c) { c->argc = c->mstate.commands[j].argc; c->argv = c->mstate.commands[j].argv; c->cmd = c->mstate.commands[j].cmd; + + /* Propagate a MULTI request once we encounter the first write op. + * This way we'll deliver the MULTI/..../EXEC block as a whole and + * both the AOF and the replication link will have the same consistency + * and atomicity guarantees. */ + if (!must_propagate && !(c->cmd->flags & REDIS_CMD_READONLY)) { + execCommandPropagateMulti(c); + must_propagate = 1; + } + call(c,REDIS_CALL_FULL); /* Commands may alter argc/argv, restore mstate. */ @@ -166,13 +166,10 @@ void execCommand(redisClient *c) { c->argv = orig_argv; c->argc = orig_argc; c->cmd = orig_cmd; - freeClientMultiState(c); - initClientMultiState(c); - c->flags &= ~(REDIS_MULTI|REDIS_DIRTY_CAS|REDIS_DIRTY_EXEC); - /* Make sure the EXEC command is always replicated / AOF, since we - * always send the MULTI command (we can't know beforehand if the - * next operations will contain at least a modification to the DB). */ - server.dirty++; + discardTransaction(c); + /* Make sure the EXEC command will be propagated as well if MULTI + * was already propagated. */ + if (must_propagate) server.dirty++; handle_monitor: /* Send EXEC to clients waiting data from MONITOR. We do it here diff --git a/src/networking.c b/src/networking.c index 3f14b17c8..569b64945 100644 --- a/src/networking.c +++ b/src/networking.c @@ -1259,7 +1259,7 @@ void rewriteClientCommandVector(redisClient *c, int argc, ...) { /* Replace argv and argc with our new versions. */ c->argv = argv; c->argc = argc; - c->cmd = lookupCommand(c->argv[0]->ptr); + c->cmd = lookupCommandOrOriginal(c->argv[0]->ptr); redisAssertWithInfo(c,NULL,c->cmd != NULL); va_end(ap); } @@ -1277,7 +1277,7 @@ void rewriteClientCommandArgument(redisClient *c, int i, robj *newval) { /* If this is the command name make sure to fix c->cmd. */ if (i == 0) { - c->cmd = lookupCommand(c->argv[0]->ptr); + c->cmd = lookupCommandOrOriginal(c->argv[0]->ptr); redisAssertWithInfo(c,NULL,c->cmd != NULL); } } diff --git a/src/pubsub.c b/src/pubsub.c index ece32445d..ed2b0766e 100644 --- a/src/pubsub.c +++ b/src/pubsub.c @@ -306,7 +306,6 @@ void punsubscribeCommand(redisClient *c) { slowlogAddComplexityParam('M', listLength(server.pubsub_patterns)); if (c->argc == 1) { pubsubUnsubscribeAllPatterns(c,1); - return; } else { int j; @@ -648,7 +648,9 @@ int rdbSave(char *filename, int dbnum) { return REDIS_ERR; } - rioInitWithFileAndFsyncInterval(&rdb,fp, 1024*1024*16); + rioInitWithFile(&rdb,fp); + if (server.rdb_incremental_fsync) + rioSetAutoSync(&rdb,REDIS_RDB_AUTOSYNC_BYTES); if (server.rdb_checksum) rdb.update_cksum = rioGenericUpdateChecksum; snprintf(magic,sizeof(magic),"REDIS%04d",REDIS_RDB_VERSION); @@ -763,6 +765,7 @@ int rdbSaveBackground(char *filename, int bgsavetype, int dbnum) { if (bgsavetype == REDIS_BGSAVE_NORMAL) server.stat_rdb_saves++; server.dirty_before_bgsave = server.dirty; + server.lastbgsave_try = time(NULL); start = ustime(); if (server.rdb_bgsavefilename) zfree(server.rdb_bgsavefilename); @@ -1144,11 +1147,8 @@ int rdbLoad(char *filename) { FILE *fp; rio rdb; - fp = fopen(filename,"r"); - if (!fp) { - errno = ENOENT; - return REDIS_ERR; - } + if ((fp = fopen(filename,"r")) == NULL) return REDIS_ERR; + rioInitWithFile(&rdb,fp); rdb.update_cksum = rdbLoadProgressCallback; rdb.max_processing_chunk = server.loading_process_events_interval_bytes; diff --git a/src/redis-cli.c b/src/redis-cli.c index c182ac17d..f12059ca4 100644 --- a/src/redis-cli.c +++ b/src/redis-cli.c @@ -42,6 +42,7 @@ #include <sys/time.h> #include <assert.h> #include <fcntl.h> +#include <limits.h> #include "hiredis.h" #include "sds.h" @@ -56,6 +57,7 @@ #define OUTPUT_STANDARD 0 #define OUTPUT_RAW 1 #define OUTPUT_CSV 2 +#define REDIS_CLI_KEEPALIVE_INTERVAL 15 /* seconds */ static redisContext *context; static struct config { @@ -70,11 +72,13 @@ static struct config { int monitor_mode; int pubsub_mode; int latency_mode; + int latency_history; int cluster_mode; int cluster_reissue_command; int slave_mode; int pipe_mode; int getrdb_mode; + int stat_mode; char *rdb_filename; int bigkeys; int stdinarg; /* get last arg from stdin. (-x option) */ @@ -332,6 +336,12 @@ static int cliConnect(int force) { return REDIS_ERR; } + /* Set aggressive KEEP_ALIVE socket option in the Redis context socket + * in order to prevent timeouts caused by the execution of long + * commands. At the same time this improves the detection of real + * errors. */ + anetKeepAlive(NULL, context->fd, REDIS_CLI_KEEPALIVE_INTERVAL); + /* Do AUTH and select the right DB. */ if (cliAuth() != REDIS_OK) return REDIS_ERR; @@ -623,6 +633,36 @@ static int cliSendCommand(int argc, char **argv, int repeat) { return REDIS_OK; } +/* Send the INFO command, reconnecting the link if needed. */ +static redisReply *reconnectingInfo(void) { + redisContext *c = context; + redisReply *reply = NULL; + int tries = 0; + + assert(!c->err); + while(reply == NULL) { + while (c->err & (REDIS_ERR_IO | REDIS_ERR_EOF)) { + printf("Reconnecting (%d)...\r", ++tries); + fflush(stdout); + + redisFree(c); + c = redisConnect(config.hostip,config.hostport); + usleep(1000000); + } + + reply = redisCommand(c,"INFO"); + if (c->err && !(c->err & (REDIS_ERR_IO | REDIS_ERR_EOF))) { + fprintf(stderr, "Error: %s\n", c->errstr); + exit(1); + } else if (tries > 0) { + printf("\n"); + } + } + + context = c; + return reply; +} + /*------------------------------------------------------------------------------ * User interface *--------------------------------------------------------------------------- */ @@ -661,8 +701,13 @@ static int parseOptions(int argc, char **argv) { config.output = OUTPUT_CSV; } else if (!strcmp(argv[i],"--latency")) { config.latency_mode = 1; + } else if (!strcmp(argv[i],"--latency-history")) { + config.latency_mode = 1; + config.latency_history = 1; } else if (!strcmp(argv[i],"--slave")) { config.slave_mode = 1; + } else if (!strcmp(argv[i],"--stat")) { + config.stat_mode = 1; } else if (!strcmp(argv[i],"--rdb") && !lastarg) { config.getrdb_mode = 1; config.rdb_filename = argv[++i]; @@ -683,7 +728,15 @@ static int parseOptions(int argc, char **argv) { sdsfree(version); exit(0); } else { - break; + if (argv[i][0] == '-') { + fprintf(stderr, + "Unrecognized option or bad number of args for: '%s'\n", + argv[i]); + exit(1); + } else { + /* Likely the command name, stop here. */ + break; + } } } return i; @@ -712,26 +765,29 @@ static void usage() { "redis-cli %s\n" "\n" "Usage: redis-cli [OPTIONS] [cmd [arg [arg ...]]]\n" -" -h <hostname> Server hostname (default: 127.0.0.1)\n" -" -p <port> Server port (default: 6379)\n" -" -s <socket> Server socket (overrides hostname and port)\n" -" -a <password> Password to use when connecting to the server\n" -" -r <repeat> Execute specified command N times\n" -" -i <interval> When -r is used, waits <interval> seconds per command.\n" -" It is possible to specify sub-second times like -i 0.1\n" -" -n <db> Database number\n" -" -x Read last argument from STDIN\n" -" -d <delimiter> Multi-bulk delimiter in for raw formatting (default: \\n)\n" -" -c Enable cluster mode (follow -ASK and -MOVED redirections)\n" -" --raw Use raw formatting for replies (default when STDOUT is not a tty)\n" -" --latency Enter a special mode continuously sampling latency\n" -" --slave Simulate a slave showing commands received from the master\n" -" --rdb <filename> Transfer an RDB dump from remote server to local file.\n" -" --pipe Transfer raw Redis protocol from stdin to server\n" -" --bigkeys Sample Redis keys looking for big keys\n" -" --eval <file> Send an EVAL command using the Lua script at <file>\n" -" --help Output this help and exit\n" -" --version Output version and exit\n" +" -h <hostname> Server hostname (default: 127.0.0.1)\n" +" -p <port> Server port (default: 6379)\n" +" -s <socket> Server socket (overrides hostname and port)\n" +" -a <password> Password to use when connecting to the server\n" +" -r <repeat> Execute specified command N times\n" +" -i <interval> When -r is used, waits <interval> seconds per command.\n" +" It is possible to specify sub-second times like -i 0.1\n" +" -n <db> Database number\n" +" -x Read last argument from STDIN\n" +" -d <delimiter> Multi-bulk delimiter in for raw formatting (default: \\n)\n" +" -c Enable cluster mode (follow -ASK and -MOVED redirections)\n" +" --raw Use raw formatting for replies (default when STDOUT is\n" +" not a tty)\n" +" --latency Enter a special mode continuously sampling latency\n" +" --latency-history Like --latency but tracking latency changes over time.\n" +" Default time interval is 15 sec. Change it using -i.\n" +" --slave Simulate a slave showing commands received from the master\n" +" --rdb <filename> Transfer an RDB dump from remote server to local file.\n" +" --pipe Transfer raw Redis protocol from stdin to server\n" +" --bigkeys Sample Redis keys looking for big keys\n" +" --eval <file> Send an EVAL command using the Lua script at <file>\n" +" --help Output this help and exit\n" +" --version Output version and exit\n" "\n" "Examples:\n" " cat /etc/passwd | redis-cli -x set mypasswd\n" @@ -843,8 +899,7 @@ static void repl() { } } /* Free the argument vector */ - while(argc--) sdsfree(argv[argc]); - zfree(argv); + sdsfreesplitres(argv,argc); } /* linenoise() returns malloc-ed lines like readline() */ free(line); @@ -903,10 +958,16 @@ static int evalMode(int argc, char **argv) { return cliSendCommand(argc+3-got_comma, argv2, config.repeat); } +#define LATENCY_SAMPLE_RATE 10 /* milliseconds. */ +#define LATENCY_HISTORY_DEFAULT_INTERVAL 15000 /* milliseconds. */ static void latencyMode(void) { redisReply *reply; long long start, latency, min = 0, max = 0, tot = 0, count = 0; + long long history_interval = + config.interval ? config.interval/1000 : + LATENCY_HISTORY_DEFAULT_INTERVAL; double avg; + long long history_start = mstime(); if (!context) exit(1); while(1) { @@ -931,7 +992,13 @@ static void latencyMode(void) { printf("\x1b[0G\x1b[2Kmin: %lld, max: %lld, avg: %.2f (%lld samples)", min, max, avg, count); fflush(stdout); - usleep(10000); + if (config.latency_history && mstime()-history_start > history_interval) + { + printf(" -- %.2f seconds range\n", (float)(mstime()-history_start)/1000); + history_start = mstime(); + min = max = tot = count = 0; + } + usleep(LATENCY_SAMPLE_RATE * 1000); } } @@ -1199,7 +1266,11 @@ static void findBigKeys(void) { fprintf(stderr, "RANDOMKEY error: %s\n", reply1->str); exit(1); + } else if (reply1->type == REDIS_REPLY_NIL) { + fprintf(stderr, "It looks like the database is empty!\n"); + exit(1); } + /* Get the key type */ reply2 = redisCommand(context,"TYPE %s",reply1->str); assert(reply2 && reply2->type == REDIS_REPLY_STATUS); @@ -1255,6 +1326,145 @@ static void findBigKeys(void) { } } +/* Return the specified INFO field from the INFO command output "info". + * A new buffer is allocated for the result, that needs to be free'd. + * If the field is not found NULL is returned. */ +static char *getInfoField(char *info, char *field) { + char *p = strstr(info,field); + char *n1, *n2; + char *result; + + if (!p) return NULL; + p += strlen(field)+1; + n1 = strchr(p,'\r'); + n2 = strchr(p,','); + if (n2 && n2 < n1) n1 = n2; + result = malloc(sizeof(char)*(n1-p)+1); + memcpy(result,p,(n1-p)); + result[n1-p] = '\0'; + return result; +} + +/* Like the above function but automatically convert the result into + * a long. On error (missing field) LONG_MIN is returned. */ +static long getLongInfoField(char *info, char *field) { + char *value = getInfoField(info,field); + long l; + + if (!value) return LONG_MIN; + l = strtol(value,NULL,10); + free(value); + return l; +} + +/* Convert number of bytes into a human readable string of the form: + * 100B, 2G, 100M, 4K, and so forth. */ +void bytesToHuman(char *s, long long n) { + double d; + + if (n < 0) { + *s = '-'; + s++; + n = -n; + } + if (n < 1024) { + /* Bytes */ + sprintf(s,"%lluB",n); + return; + } else if (n < (1024*1024)) { + d = (double)n/(1024); + sprintf(s,"%.2fK",d); + } else if (n < (1024LL*1024*1024)) { + d = (double)n/(1024*1024); + sprintf(s,"%.2fM",d); + } else if (n < (1024LL*1024*1024*1024)) { + d = (double)n/(1024LL*1024*1024); + sprintf(s,"%.2fG",d); + } +} + +static void statMode() { + redisReply *reply; + long aux, requests = 0; + int i = 0; + + while(1) { + char buf[64]; + int j; + + reply = reconnectingInfo(); + if (reply->type == REDIS_REPLY_ERROR) { + printf("ERROR: %s\n", reply->str); + exit(1); + } + + if ((i++ % 20) == 0) { + printf( +"------- data ------ --------------------- load -------------------- - child -\n" +"keys mem clients blocked requests connections \n"); + } + + /* Keys */ + aux = 0; + for (j = 0; j < 20; j++) { + long k; + + sprintf(buf,"db%d:keys",j); + k = getLongInfoField(reply->str,buf); + if (k == LONG_MIN) continue; + aux += k; + } + sprintf(buf,"%ld",aux); + printf("%-11s",buf); + + /* Used memory */ + aux = getLongInfoField(reply->str,"used_memory"); + bytesToHuman(buf,aux); + printf("%-8s",buf); + + /* Clients */ + aux = getLongInfoField(reply->str,"connected_clients"); + sprintf(buf,"%ld",aux); + printf(" %-8s",buf); + + /* Blocked (BLPOPPING) Clients */ + aux = getLongInfoField(reply->str,"blocked_clients"); + sprintf(buf,"%ld",aux); + printf("%-8s",buf); + + /* Requets */ + aux = getLongInfoField(reply->str,"total_commands_processed"); + sprintf(buf,"%ld (+%ld)",aux,requests == 0 ? 0 : aux-requests); + printf("%-19s",buf); + requests = aux; + + /* Connections */ + aux = getLongInfoField(reply->str,"total_connections_received"); + sprintf(buf,"%ld",aux); + printf(" %-12s",buf); + + /* Children */ + aux = getLongInfoField(reply->str,"bgsave_in_progress"); + aux |= getLongInfoField(reply->str,"aof_rewrite_in_progress") << 1; + switch(aux) { + case 0: break; + case 1: + printf("SAVE"); + break; + case 2: + printf("AOF"); + break; + case 3: + printf("SAVE+AOF"); + break; + } + + printf("\n"); + freeReplyObject(reply); + usleep(config.interval); + } +} + int main(int argc, char **argv) { int firstarg; @@ -1269,6 +1479,7 @@ int main(int argc, char **argv) { config.monitor_mode = 0; config.pubsub_mode = 0; config.latency_mode = 0; + config.latency_history = 0; config.cluster_mode = 0; config.slave_mode = 0; config.getrdb_mode = 0; @@ -1319,6 +1530,13 @@ int main(int argc, char **argv) { findBigKeys(); } + /* Stat mode */ + if (config.stat_mode) { + if (cliConnect(0) == REDIS_ERR) exit(1); + if (config.interval == 0) config.interval = 1000000; + statMode(); + } + /* Start interactive mode when no command is provided */ if (argc == 0 && !config.eval) { /* Note that in repl mode we don't abort on connection error. diff --git a/src/redis.c b/src/redis.c index a7d468104..3e9b178ac 100755 --- a/src/redis.c +++ b/src/redis.c @@ -114,7 +114,7 @@ struct redisCommand *commandTable; */ struct redisCommand redisCommandTable[] = { {"get",getCommand,2,"r",0,NULL,1,1,1,0,0}, - {"set",setCommand,3,"wm",0,noPreloadGetKeys,1,1,1,0,0}, + {"set",setCommand,-3,"wm",0,noPreloadGetKeys,1,1,1,0,0}, {"setnx",setnxCommand,3,"wm",0,noPreloadGetKeys,1,1,1,0,0}, {"setex",setexCommand,4,"wm",0,noPreloadGetKeys,1,1,1,0,0}, {"psetex",psetexCommand,4,"wm",0,noPreloadGetKeys,1,1,1,0,0}, @@ -197,7 +197,7 @@ struct redisCommand redisCommandTable[] = { {"mset",msetCommand,-3,"wm",0,NULL,1,-1,2,0,0}, {"msetnx",msetnxCommand,-3,"wm",0,NULL,1,-1,2,0,0}, {"randomkey",randomkeyCommand,1,"rR",0,NULL,0,0,0,0,0}, - {"select",selectCommand,2,"r",0,NULL,0,0,0,0,0}, + {"select",selectCommand,2,"rl",0,NULL,0,0,0,0,0}, {"move",moveCommand,3,"w",0,NULL,1,1,1,0,0}, {"rename",renameCommand,3,"w",0,renameGetKeys,1,2,1,0,0}, {"renamenx",renamenxCommand,3,"w",0,renameGetKeys,1,2,1,0,0}, @@ -207,7 +207,7 @@ struct redisCommand redisCommandTable[] = { {"pexpireat",pexpireatCommand,3,"w",0,NULL,1,1,1,0,0}, {"keys",keysCommand,2,"rS",0,NULL,0,0,0,0,0}, {"dbsize",dbsizeCommand,1,"r",0,NULL,0,0,0,0,0}, - {"auth",authCommand,2,"rs",0,NULL,0,0,0,0,0}, + {"auth",authCommand,2,"rsl",0,NULL,0,0,0,0,0}, {"ping",pingCommand,1,"r",0,NULL,0,0,0,0,0}, {"echo",echoCommand,2,"r",0,NULL,0,0,0,0,0}, {"save",saveCommand,1,"ars",0,NULL,0,0,0,0,0}, @@ -238,7 +238,7 @@ struct redisCommand redisCommandTable[] = { {"unsubscribe",unsubscribeCommand,-1,"rpslt",0,NULL,0,0,0,0,0}, {"psubscribe",psubscribeCommand,-2,"rpslt",0,NULL,0,0,0,0,0}, {"punsubscribe",punsubscribeCommand,-1,"rpslt",0,NULL,0,0,0,0,0}, - {"publish",publishCommand,3,"pflt",0,NULL,0,0,0,0,0}, + {"publish",publishCommand,3,"pfltr",0,NULL,0,0,0,0,0}, {"watch",watchCommand,-2,"rs",0,noPreloadGetKeys,1,-1,1,0,0}, {"unwatch",unwatchCommand,1,"rs",0,NULL,0,0,0,0,0}, {"restore",restoreCommand,4,"awm",0,NULL,1,1,1,0,0}, @@ -574,36 +574,32 @@ int htNeedsResize(dict *dict) { /* If the percentage of used slots in the HT reaches REDIS_HT_MINFILL * we resize the hash table to save memory */ -void tryResizeHashTables(void) { - int j; - - for (j = 0; j < server.dbnum; j++) { - if (htNeedsResize(server.db[j].dict)) - dictResize(server.db[j].dict); - if (htNeedsResize(server.db[j].expires)) - dictResize(server.db[j].expires); - } +void tryResizeHashTables(int dbid) { + if (htNeedsResize(server.db[dbid].dict)) + dictResize(server.db[dbid].dict); + if (htNeedsResize(server.db[dbid].expires)) + dictResize(server.db[dbid].expires); } /* Our hash table implementation performs rehashing incrementally while * we write/read from the hash table. Still if the server is idle, the hash * table will use two tables for a long time. So we try to use 1 millisecond - * of CPU time at every serverCron() loop in order to rehash some key. */ -void incrementallyRehash(void) { - int j; - - for (j = 0; j < server.dbnum; j++) { - /* Keys dictionary */ - if (dictIsRehashing(server.db[j].dict)) { - dictRehashMilliseconds(server.db[j].dict,1); - break; /* already used our millisecond for this loop... */ - } - /* Expires */ - if (dictIsRehashing(server.db[j].expires)) { - dictRehashMilliseconds(server.db[j].expires,1); - break; /* already used our millisecond for this loop... */ - } + * of CPU time at every call of this function to perform some rehahsing. + * + * The function returns 1 if some rehashing was performed, otherwise 0 + * is returned. */ +int incrementallyRehash(int dbid) { + /* Keys dictionary */ + if (dictIsRehashing(server.db[dbid].dict)) { + dictRehashMilliseconds(server.db[dbid].dict,1); + return 1; /* already used our millisecond for this loop... */ + } + /* Expires */ + if (dictIsRehashing(server.db[dbid].expires)) { + dictRehashMilliseconds(server.db[dbid].expires,1); + return 1; /* already used our millisecond for this loop... */ } + return 0; } /* This function is called once a background process of some kind terminates, @@ -624,28 +620,57 @@ void updateDictResizePolicy(void) { /* Try to expire a few timed out keys. The algorithm used is adaptive and * will use few CPU cycles if there are few expiring keys, otherwise * it will get more aggressive to avoid that too much memory is used by - * keys that can be removed from the keyspace. */ + * keys that can be removed from the keyspace. + * + * No more than REDIS_DBCRON_DBS_PER_CALL databases are tested at every + * iteration. */ void activeExpireCycle(void) { - int j, iteration = 0; + /* This function has some global state in order to continue the work + * incrementally across calls. */ + static unsigned int current_db = 0; /* Last DB tested. */ + static int timelimit_exit = 0; /* Time limit hit in previous call? */ + + unsigned int j, iteration = 0; + unsigned int dbs_per_call = REDIS_DBCRON_DBS_PER_CALL; long long start = ustime(), timelimit; + /* We usually should test REDIS_DBCRON_DBS_PER_CALL per iteration, with + * two exceptions: + * + * 1) Don't test more DBs than we have. + * 2) If last time we hit the time limit, we want to scan all DBs + * in this iteration, as there is work to do in some DB and we don't want + * expired keys to use memory for too much time. */ + if (dbs_per_call > server.dbnum || timelimit_exit) + dbs_per_call = server.dbnum; + /* We can use at max REDIS_EXPIRELOOKUPS_TIME_PERC percentage of CPU time * per iteration. Since this function gets called with a frequency of - * REDIS_HZ times per second, the following is the max amount of + * server.hz times per second, the following is the max amount of * microseconds we can spend in this function. */ - timelimit = 1000000*REDIS_EXPIRELOOKUPS_TIME_PERC/REDIS_HZ/100; + timelimit = 1000000*REDIS_EXPIRELOOKUPS_TIME_PERC/server.hz/100; + timelimit_exit = 0; if (timelimit <= 0) timelimit = 1; - for (j = 0; j < server.dbnum; j++) { + for (j = 0; j < dbs_per_call; j++) { int expired; - redisDb *db = server.db+j; + redisDb *db = server.db+(current_db % server.dbnum); + + /* Increment the DB now so we are sure if we run out of time + * in the current DB we'll restart from the next. This allows to + * distribute the time evenly across DBs. */ + current_db++; /* Continue to expire if at the end of the cycle more than 25% * of the keys were expired. */ do { - unsigned long num = dictSize(db->expires); - unsigned long slots = dictSlots(db->expires); - long long now = mstime(); + unsigned long num, slots; + long long now; + + /* If there is nothing to expire try next DB ASAP. */ + if ((num = dictSize(db->expires)) == 0) break; + slots = dictSlots(db->expires); + now = mstime(); /* When there are less than 1% filled slots getting random * keys is expensive, so stop here waiting for better times... @@ -679,8 +704,12 @@ void activeExpireCycle(void) { * expire. So after a given amount of milliseconds return to the * caller waiting for the other active expire cycle. */ iteration++; - if ((iteration & 0xf) == 0 && /* check once every 16 cycles. */ - (ustime()-start) > timelimit) return; + if ((iteration & 0xf) == 0 && /* check once every 16 iterations. */ + (ustime()-start) > timelimit) + { + timelimit_exit = 1; + return; + } } while (expired > REDIS_EXPIRELOOKUPS_PER_CRON/4); } } @@ -766,13 +795,13 @@ int clientsCronResizeQueryBuffer(redisClient *c) { } void clientsCron(void) { - /* Make sure to process at least 1/(REDIS_HZ*10) of clients per call. - * Since this function is called REDIS_HZ times per second we are sure that + /* Make sure to process at least 1/(server.hz*10) of clients per call. + * Since this function is called server.hz times per second we are sure that * in the worst case we process all the clients in 10 seconds. * In normal conditions (a reasonable number of clients) we process * all the clients in a shorter time. */ int numclients = listLength(server.clients); - int iterations = numclients/(REDIS_HZ*10); + int iterations = numclients/(server.hz*10); if (iterations < 50) iterations = (numclients < 50) ? numclients : 50; @@ -872,7 +901,52 @@ void checkOSMemory(void) { } -/* This is our timer interrupt, called REDIS_HZ times per second. +/* This function handles 'background' operations we are required to do + * incrementally in Redis databases, such as active key expiring, resizing, + * rehashing. */ +void databasesCron(void) { + /* Expire keys by random sampling. Not required for slaves + * as master will synthesize DELs for us. */ + if (server.active_expire_enabled && server.masterhost == NULL) + activeExpireCycle(); + + /* Perform hash tables rehashing if needed, but only if there are no + * other processes saving the DB on disk. Otherwise rehashing is bad + * as will cause a lot of copy-on-write of memory pages. */ + if (server.rdb_child_pid == -1 && server.aof_child_pid == -1) { + /* We use global counters so if we stop the computation at a given + * DB we'll be able to start from the successive in the next + * cron loop iteration. */ + static unsigned int resize_db = 0; + static unsigned int rehash_db = 0; + unsigned int dbs_per_call = REDIS_DBCRON_DBS_PER_CALL; + unsigned int j; + + /* Don't test more DBs than we have. */ + if (dbs_per_call > server.dbnum) dbs_per_call = server.dbnum; + + /* Resize */ + for (j = 0; j < dbs_per_call; j++) { + tryResizeHashTables(resize_db % server.dbnum); + resize_db++; + } + + /* Rehash */ + if (server.activerehashing) { + for (j = 0; j < dbs_per_call; j++) { + int work_done = incrementallyRehash(rehash_db % server.dbnum); + rehash_db++; + if (work_done) { + /* If the function did some work, stop here, we'll do + * more at the next cron loop. */ + break; + } + } + } + } +} + +/* This is our timer interrupt, called server.hz times per second. * Here is where we do a number of things that need to be done asynchronously. * For instance: * @@ -886,7 +960,7 @@ void checkOSMemory(void) { * - Replication reconnection. * - Many more... * - * Everything directly called here will be called REDIS_HZ times per second, + * Everything directly called here will be called server.hz times per second, * so in order to throttle execution of things we want to do less frequently * a macro is used: run_with_period(milliseconds) { .... } */ @@ -959,17 +1033,6 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) { } } - /* We don't want to resize the hash tables while a background saving - * is in progress: the saving child is created using fork() that is - * implemented with a copy-on-write semantic in most modern systems, so - * if we resize the HT while there is the saving child at work actually - * a lot of memory movements in the parent will cause a lot of pages - * copied. */ - if (server.rdb_child_pid == -1 && server.aof_child_pid == -1) { - tryResizeHashTables(); - if (server.activerehashing) incrementallyRehash(); - } - /* Show information about connected clients */ if (!server.sentinel_mode) { run_with_period(5000) { @@ -984,6 +1047,9 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) { /* We need to do a few operations on clients asynchronously. */ clientsCron(); + /* Handle background operations on Redis databases. */ + databasesCron(); + /* Start a scheduled AOF rewrite if this was requested by the user while * a BGSAVE was in progress. */ if (server.rdb_child_pid == -1 && server.aof_child_pid == -1 && @@ -1020,8 +1086,16 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) { for (j = 0; j < server.saveparamslen; j++) { struct saveparam *sp = server.saveparams+j; + /* Save if we reached the given amount of changes, + * the given amount of seconds, and if the latest bgsave was + * successful or if, in case of an error, at least + * REDIS_BGSAVE_RETRY_DELAY seconds already elapsed. */ if (server.dirty >= sp->changes && - server.unixtime-server.lastsave > sp->seconds) { + server.unixtime-server.lastsave > sp->seconds && + (server.unixtime-server.lastbgsave_try > + REDIS_BGSAVE_RETRY_DELAY || + server.lastbgsave_status == REDIS_OK)) + { redisLog(REDIS_NOTICE,"%d changes in %d seconds. Saving...", sp->changes, sp->seconds); rdbSaveBackground(server.rdb_filename, REDIS_BGSAVE_NORMAL, -1); @@ -1050,11 +1124,6 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) { * cron function is called. */ if (server.aof_flush_postponed_start) flushAppendOnlyFile(0); - /* Expire a few keys per cycle, only if this is a master. - * On slaves we wait for DEL operations synthesized by the master - * in order to guarantee a strict consistency. */ - if (server.masterhost == NULL) activeExpireCycle(); - /* Close clients that need to be closed asynchronous */ freeClientsInAsyncFreeQueue(); @@ -1068,7 +1137,7 @@ int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData) { } server.cronloops++; - return 1000/REDIS_HZ; + return 1000/server.hz; } /* This function gets called every time Redis is entering the @@ -1176,6 +1245,7 @@ void createSharedObjects(void) { void initServerConfig() { getRandomHexChars(server.runid,REDIS_RUN_ID_SIZE); + server.hz = REDIS_DEFAULT_HZ; server.runid[REDIS_RUN_ID_SIZE] = '\0'; server.arch_bits = (sizeof(long) == 8) ? 64 : 32; server.port = REDIS_SERVERPORT; @@ -1188,6 +1258,7 @@ void initServerConfig() { server.verbosity = REDIS_NOTICE; server.maxidletime = REDIS_MAXIDLETIME; server.tcpkeepalive = 0; + server.active_expire_enabled = 1; server.client_max_querybuf_len = REDIS_MAX_QUERYBUF_LEN; server.saveparams = NULL; server.loading = 0; @@ -1213,12 +1284,15 @@ void initServerConfig() { server.aof_fd = -1; server.aof_selected_db = -1; /* Make sure the first time will not match */ server.aof_flush_postponed_start = 0; + server.aof_rewrite_incremental_fsync = 1; + server.rdb_incremental_fsync = 1; server.pidfile = zstrdup("/var/run/redis.pid"); server.rdb_filename = zstrdup("dump.rdb"); server.aof_filename = zstrdup("appendonly.aof"); server.requirepass = NULL; server.rdb_compression = 1; server.rdb_checksum = 1; + server.stop_writes_on_bgsave_err = 1; server.activerehashing = 1; server.maxclients = REDIS_MAX_CLIENTS; server.bpop_blocked_clients = 0; @@ -1256,7 +1330,7 @@ void initServerConfig() { server.repl_syncio_timeout = REDIS_REPL_SYNCIO_TIMEOUT; server.repl_serve_stale_data = 1; server.repl_slave_ro = 1; - server.repl_down_since = time(NULL); + server.repl_down_since = 0; /* Never connected, repl is down since EVER. */ server.repl_disable_tcp_nodelay = 0; server.slave_priority = REDIS_DEFAULT_SLAVE_PRIORITY; @@ -1281,6 +1355,7 @@ void initServerConfig() { * initial configuration, since command names may be changed via * redis.conf using the rename-command directive. */ server.commands = dictCreate(&commandTableDictType,NULL); + server.orig_commands = dictCreate(&commandTableDictType,NULL); populateCommandTable(); server.delCommand = lookupCommandByCString("del"); server.multiCommand = lookupCommandByCString("multi"); @@ -1406,7 +1481,8 @@ void initServer() { server.aof_child_pid = -1; aofRewriteBufferReset(); server.aof_buf = sdsempty(); - server.lastsave = time(NULL); + server.lastsave = time(NULL); /* At startup we consider the DB saved. */ + server.lastbgsave_try = 0; /* At startup we never tried to BGSAVE. */ server.rdb_save_time_last = -1; server.rdb_save_time_start = -1; server.dirty = 0; @@ -1431,8 +1507,10 @@ void initServer() { server.ops_sec_last_sample_ops = 0; server.unixtime = time(NULL); server.lastbgsave_status = REDIS_OK; - server.stop_writes_on_bgsave_err = 1; - aeCreateTimeEvent(server.el, 1, serverCron, NULL, NULL); + if(aeCreateTimeEvent(server.el, 1, serverCron, NULL, NULL) == AE_ERR) { + redisPanic("create time event failed"); + exit(1); + } if (server.ipfd > 0 && aeCreateFileEvent(server.el,server.ipfd,AE_READABLE, acceptTcpHandler,NULL) == AE_ERR) redisPanic("Unrecoverable error creating server.ipfd file event."); if (server.sofd > 0 && aeCreateFileEvent(server.el,server.sofd,AE_READABLE, @@ -1472,7 +1550,7 @@ void populateCommandTable(void) { for (j = 0; j < numcommands; j++) { struct redisCommand *c = redisCommandTable+j; char *f = c->sflags; - int retval; + int retval1, retval2; while(*f != '\0') { switch(*f) { @@ -1493,8 +1571,11 @@ void populateCommandTable(void) { f++; } - retval = dictAdd(server.commands, sdsnew(c->name), c); - assert(retval == DICT_OK); + retval1 = dictAdd(server.commands, sdsnew(c->name), c); + /* Populate an additional dictionary that will be unaffected + * by rename-command statements in redis.conf. */ + retval2 = dictAdd(server.orig_commands, sdsnew(c->name), c); + redisAssert(retval1 == DICT_OK && retval2 == DICT_OK); } } @@ -1562,6 +1643,20 @@ struct redisCommand *lookupCommandByCString(char *s) { return cmd; } +/* Lookup the command in the current table, if not found also check in + * the original table containing the original command names unaffected by + * redis.conf rename-command statement. + * + * This is used by functions rewriting the argument vector such as + * rewriteClientCommandVector() in order to set client->cmd pointer + * correctly even if the command was renamed. */ +struct redisCommand *lookupCommandOrOriginal(sds name) { + struct redisCommand *cmd = dictFetchValue(server.commands, name); + + if (!cmd) cmd = dictFetchValue(server.orig_commands,name); + return cmd; +} + /* Propagate the specified command (in the context of the specified database id) * to AOF and Slaves. * @@ -2020,6 +2115,7 @@ sds genRedisInfoString(char *section) { "tcp_port:%d\r\n" "uptime_in_seconds:%ld\r\n" "uptime_in_days:%ld\r\n" + "hz:%d\r\n" "lru_clock:%ld\r\n", REDIS_VERSION, redisGitSHA1(), @@ -2038,6 +2134,7 @@ sds genRedisInfoString(char *section) { server.port, uptime, uptime/(3600*24), + server.hz, (unsigned long) server.lruclock); } @@ -2698,7 +2795,7 @@ void loadDataFromDisk(void) { redisLog(REDIS_NOTICE,"DB loaded from disk: %.3f seconds", (float)(ustime()-start)/1000000); } else if (errno != ENOENT) { - redisLog(REDIS_WARNING,"Fatal error loading the DB. Exiting."); + redisLog(REDIS_WARNING,"Fatal error loading the DB: %s. Exiting.",strerror(errno)); exit(1); } } @@ -2707,7 +2804,7 @@ void loadDataFromDisk(void) { void redisOutOfMemoryHandler(size_t allocation_size) { redisLog(REDIS_WARNING,"Out Of Memory allocating %zu bytes!", allocation_size); - redisPanic("OOM"); + redisPanic("Redis aborting for OUT OF MEMORY"); } int main(int argc, char **argv) { diff --git a/src/redis.h b/src/redis.h index a969778d8..0c590f634 100755 --- a/src/redis.h +++ b/src/redis.h @@ -67,13 +67,16 @@ #define REDIS_ERR -1 /* Static server configuration */ -#define REDIS_HZ 100 /* Time interrupt calls/sec. */ +#define REDIS_DEFAULT_HZ 10 /* Time interrupt calls/sec. */ +#define REDIS_MIN_HZ 1 +#define REDIS_MAX_HZ 500 #define REDIS_SERVERPORT 6379 /* TCP port */ #define REDIS_MAXIDLETIME 0 /* default client timeout: infinite */ #define REDIS_DEFAULT_DBNUM 16 #define REDIS_CONFIGLINE_MAX 1024 #define REDIS_EXPIRELOOKUPS_PER_CRON 10 /* lookup 10 expires per loop */ #define REDIS_EXPIRELOOKUPS_TIME_PERC 25 /* CPU max % for keys collection */ +#define REDIS_DBCRON_DBS_PER_CALL 16 #define REDIS_MAX_WRITE_PER_EVENT (1024*64) #define REDIS_SHARED_SELECT_CMDS 10 #define REDIS_SHARED_INTEGERS 10000 @@ -92,6 +95,7 @@ #define REDIS_REPL_PING_SLAVE_PERIOD 10 #define REDIS_RUN_ID_SIZE 40 #define REDIS_OPS_SEC_SAMPLES 16 +#define REDIS_BGSAVE_RETRY_DELAY 5 /* Wait a few secs before trying again. */ /* Protocol and I/O related defines */ #define REDIS_MAX_QUERYBUF_LEN (1024*1024*1024) /* 1GB max query buffer. */ @@ -99,6 +103,8 @@ #define REDIS_REPLY_CHUNK_BYTES (16*1024) /* 16k output buffer */ #define REDIS_INLINE_MAX_SIZE (1024*64) /* Max size of inline reads */ #define REDIS_MBULK_BIG_ARG (1024*32) +#define REDIS_AOF_AUTOSYNC_BYTES (1024*1024*16) /* fdatasync every 16MB */ +#define REDIS_RDB_AUTOSYNC_BYTES (1024*1024*16) /* fdatasync every 16MB */ /* Hash table parameters */ #define REDIS_HT_MINFILL 10 /* Minimal hash table fill 10% */ @@ -302,8 +308,8 @@ /* Using the following macro you can run code inside serverCron() with the * specified period, specified in milliseconds. - * The actual resolution depends on REDIS_HZ. */ -#define run_with_period(_ms_) if (!(server.cronloops%((_ms_)/(1000/REDIS_HZ)))) + * The actual resolution depends on server.hz. */ +#define run_with_period(_ms_) if ((_ms_ <= 1000/server.hz) || !(server.cronloops%((_ms_)/(1000/server.hz)))) /* We can print the stacktrace, so our assert is defined this way: */ #define redisAssertWithInfo(_c,_o,_e) ((_e)?(void)0 : (_redisAssertWithInfo(_c,_o,#_e,__FILE__,__LINE__),_exit(1))) @@ -510,8 +516,10 @@ struct complexity_param { struct redisServer { /* General */ + int hz; /* serverCron() calls frequency in hertz */ redisDb *db; - dict *commands; /* Command table hash table */ + dict *commands; /* Command table */ + dict *orig_commands; /* Command table before command renaming. */ aeEventLoop *el; unsigned lruclock:22; /* Clock incrementing every minute, for LRU */ unsigned lruclock_padding:10; @@ -575,6 +583,7 @@ struct redisServer { int verbosity; /* Loglevel in redis.conf */ int maxidletime; /* Client timeout in seconds */ int tcpkeepalive; /* Set SO_KEEPALIVE if non-zero. */ + int active_expire_enabled; /* Can be disabled for testing purposes. */ size_t client_max_querybuf_len; /* Limit for client query buffer length */ int dbnum; /* Total number of configured DBs */ int daemonize; /* True if running as a daemon */ @@ -602,6 +611,7 @@ struct redisServer { time_t aof_rewrite_time_start; /* Current AOF rewrite start time. */ int aof_lastbgrewrite_status; /* REDIS_OK or REDIS_ERR */ unsigned long aof_delayed_fsync; /* delayed AOF fsync() counter */ + int aof_rewrite_incremental_fsync;/* fsync incrementally while rewriting? */ /* RDB persistence */ long long dirty; /* Changes to DB from the last save */ long long dirty_before_bgsave; /* Used to restore dirty on failed BGSAVE */ @@ -616,10 +626,12 @@ struct redisServer { int rdb_compression; /* Use compression in RDB? */ int rdb_checksum; /* Use RDB checksum? */ time_t lastsave; /* Unix time of last successful save */ + time_t lastbgsave_try; /* Unix time of last attempted bgsave */ time_t rdb_save_time_last; /* Time used by last RDB save run. */ time_t rdb_save_time_start; /* Current RDB save start time. */ int lastbgsave_status; /* REDIS_OK or REDIS_ERR */ int stop_writes_on_bgsave_err; /* Don't allow writes if can't BGSAVE */ + int rdb_incremental_fsync; /* fsync incrementally while rewriting? */ /* Propagation of commands in AOF / replication */ redisOpArray also_propagate; /* Additional command to propagate. */ int draining; /* Currently draining to slaves? */ @@ -979,6 +991,7 @@ int processCommand(redisClient *c); void setupSignalHandlers(void); struct redisCommand *lookupCommand(sds name); struct redisCommand *lookupCommandByCString(char *s); +struct redisCommand *lookupCommandOrOriginal(sds name); void call(redisClient *c, int flags); void propagate(struct redisCommand *cmd, int dbid, robj **argv, int argc, int flags); void alsoPropagate(struct redisCommand *cmd, int dbid, robj **argv, int argc, int target); diff --git a/src/replication.c b/src/replication.c index d8283590d..4d11cf6b3 100755 --- a/src/replication.c +++ b/src/replication.c @@ -697,11 +697,16 @@ void syncWithMaster(aeEventLoop *el, int fd, void *privdata, int mask) { goto error; } - /* We don't care about the reply, it can be +PONG or an error since - * the server requires AUTH. As long as it replies correctly, it's - * fine from our point of view. */ - if (buf[0] != '-' && buf[0] != '+') { - redisLog(REDIS_WARNING,"Unexpected reply to PING from master."); + /* We accept only two replies as valid, a positive +PONG reply + * (we just check for "+") or an authentication error. + * Note that older versions of Redis replied with "operation not + * permitted" instead of using a proper error code, so we test + * both. */ + if (buf[0] != '+' && + strncmp(buf,"-NOAUTH",7) != 0 && + strncmp(buf,"-ERR operation not permitted",28) != 0) + { + redisLog(REDIS_WARNING,"Error reply to PING from master: '%s'",buf); goto error; } else { redisLog(REDIS_NOTICE, @@ -924,7 +929,7 @@ void replicationCron(void) { * So slaves can implement an explicit timeout to masters, and will * be able to detect a link disconnection even if the TCP connection * will not actually go down. */ - if (!(server.cronloops % (server.repl_ping_slave_period * REDIS_HZ))) { + if (!(server.cronloops % (server.repl_ping_slave_period * server.hz))) { listIter li; listNode *ln; @@ -51,6 +51,8 @@ #include <unistd.h> #include "rio.h" #include "util.h" +#include "config.h" +#include "redis.h" uint64_t crc64(uint64_t crc, const unsigned char *s, uint64_t l); @@ -79,10 +81,10 @@ static off_t rioBufferTell(rio *r) { static size_t rioFileWrite(rio *r, const void *buf, size_t len) { size_t bytes_written = 0; while (len) { - size_t bytes_to_write = (r->io.file.fsync_interval && r->io.file.fsync_interval < len) ? r->io.file.fsync_interval : len; + size_t bytes_to_write = (r->io.file.autosync && r->io.file.autosync < len) ? r->io.file.autosync : len; if (fwrite((char*)buf + bytes_written,bytes_to_write,1,r->io.file.fp) != 1) return 0; - if (r->io.file.fsync_interval && (r->processed_bytes + bytes_written)/r->io.file.fsync_interval < (r->processed_bytes + bytes_written + bytes_to_write)/r->io.file.fsync_interval) + if (r->io.file.autosync && (r->processed_bytes + bytes_written)/r->io.file.autosync < (r->processed_bytes + bytes_written + bytes_to_write)/r->io.file.autosync) fsync(fileno(r->io.file.fp)); bytes_written += bytes_to_write; len -= bytes_to_write; @@ -125,12 +127,7 @@ static const rio rioFileIO = { void rioInitWithFile(rio *r, FILE *fp) { *r = rioFileIO; r->io.file.fp = fp; -} - -void rioInitWithFileAndFsyncInterval(rio *r, FILE *fp, size_t fsyncInterval) { - *r = rioFileIO; - r->io.file.fp = fp; - r->io.file.fsync_interval = fsyncInterval; + r->io.file.autosync = 0; } void rioInitWithBuffer(rio *r, sds s) { @@ -145,6 +142,19 @@ void rioGenericUpdateChecksum(rio *r, const void *buf, size_t len) { r->cksum = crc64(r->cksum,buf,len); } +/* Set the file-based rio object to auto-fsync every 'bytes' file written. + * By default this is set to zero that means no automatic file sync is + * performed. + * + * This feature is useful in a few contexts since when we rely on OS write + * buffers sometimes the OS buffers way too much, resulting in too many + * disk I/O concentrated in very little time. When we fsync in an explicit + * way instead the I/O pressure is more distributed across time. */ +void rioSetAutoSync(rio *r, off_t bytes) { + redisAssert(r->read == rioFileIO.read); + r->io.file.autosync = bytes; +} + /* ------------------------------ Higher level interface --------------------------- * The following higher level functions use lower level rio.c functions to help * generating the Redis protocol for the Append Only File. */ @@ -43,10 +43,11 @@ struct _rio { size_t (*read)(struct _rio *, void *buf, size_t len); size_t (*write)(struct _rio *, const void *buf, size_t len); off_t (*tell)(struct _rio *); - /* The update_cksum method if not NULL is used to compute the checksum of all the - * data that was read or written so far. The method should be designed so that - * can be called with the current checksum, and the buf and len fields pointing - * to the new block of data to add to the checksum computation. */ + /* The update_cksum method if not NULL is used to compute the checksum of + * all the data that was read or written so far. The method should be + * designed so that can be called with the current checksum, and the buf + * and len fields pointing to the new block of data to add to the checksum + * computation. */ void (*update_cksum)(struct _rio *, const void *buf, size_t len); /* The current checksum */ @@ -66,7 +67,7 @@ struct _rio { } buffer; struct { FILE *fp; - size_t fsync_interval; + off_t autosync; /* fsync after 'autosync' bytes written. */ } file; } io; }; @@ -108,7 +109,6 @@ static inline off_t rioTell(rio *r) { } void rioInitWithFile(rio *r, FILE *fp); -void rioInitWithFileAndFsyncInterval(rio *r, FILE *fp, size_t fsyncInterval); void rioInitWithBuffer(rio *r, sds s); size_t rioWriteBulkCount(rio *r, char prefix, int count); @@ -117,5 +117,6 @@ size_t rioWriteBulkLongLong(rio *r, long long l); size_t rioWriteBulkDouble(rio *r, double d); void rioGenericUpdateChecksum(rio *r, const void *buf, size_t len); +void rioSetAutoSync(rio *r, off_t bytes); #endif diff --git a/src/scripting.c b/src/scripting.c index ee3d0405b..ff8c83cdc 100755 --- a/src/scripting.c +++ b/src/scripting.c @@ -258,6 +258,7 @@ int luaRedisGenericCommand(lua_State *lua, int raise_error) { "Write commands not allowed after non deterministic commands"); goto cleanup; } else if (server.masterhost && server.repl_slave_ro && + !server.loading && !(server.lua_caller->flags & REDIS_MASTER)) { luaPushError(lua, shared.roslaveerr->ptr); @@ -475,11 +475,18 @@ int hex_digit_to_int(char c) { * foo bar "newline are supported\n" and "\xff\x00otherstuff" * * The number of arguments is stored into *argc, and an array - * of sds is returned. The caller should sdsfree() all the returned - * strings and finally zfree() the array itself. + * of sds is returned. + * + * The caller should free the resulting array of sds strings with + * sdsfreesplitres(). * * Note that sdscatrepr() is able to convert back a string into * a quoted string in the same format sdssplitargs() is able to parse. + * + * The function returns the allocated tokens on success, even when the + * input string is empty, or NULL if the input contains unbalanced + * quotes or closed quotes followed by non space characters + * as in: "foo"bar or "foo' */ sds *sdssplitargs(const char *line, int *argc) { const char *p = line; @@ -576,6 +583,8 @@ sds *sdssplitargs(const char *line, int *argc) { (*argc)++; current = NULL; } else { + /* Even on empty input string return something not NULL. */ + if (vector == NULL) vector = zmalloc(sizeof(void*)); return vector; } } @@ -585,16 +594,10 @@ err: sdsfree(vector[*argc]); zfree(vector); if (current) sdsfree(current); + *argc = 0; return NULL; } -void sdssplitargs_free(sds *argv, int argc) { - int j; - - for (j = 0 ;j < argc; j++) sdsfree(argv[j]); - zfree(argv); -} - /* Modify the string substituting all the occurrences of the set of * characters specified in the 'from' string to the corresponding character * in the 'to' array. @@ -88,7 +88,6 @@ void sdstoupper(sds s); sds sdsfromlonglong(long long value); sds sdscatrepr(sds s, const char *p, size_t len); sds *sdssplitargs(const char *line, int *argc); -void sdssplitargs_free(sds *argv, int argc); sds sdsmapchars(sds s, const char *from, const char *to, size_t setlen); /* Low level functions exposed to the user API */ diff --git a/src/sentinel.c b/src/sentinel.c index fc857344c..ed0978694 100644 --- a/src/sentinel.c +++ b/src/sentinel.c @@ -74,6 +74,8 @@ typedef struct sentinelAddr { #define SRI_RECONF_DONE (1<<13) /* Slave synchronized with new master. */ #define SRI_FORCE_FAILOVER (1<<14) /* Force failover with master up. */ #define SRI_SCRIPT_KILL_SENT (1<<15) /* SCRIPT KILL already sent on -BUSY */ +#define SRI_DEMOTE (1<<16) /* If the instance claims to be a master, demote + it into a slave sending SLAVEOF. */ #define SENTINEL_INFO_PERIOD 10000 #define SENTINEL_PING_PERIOD 1000 @@ -403,7 +405,7 @@ void initSentinel(void) { /* Initialize various data structures. */ sentinel.masters = dictCreate(&instancesDictType,NULL); sentinel.tilt = 0; - sentinel.tilt_start_time = mstime(); + sentinel.tilt_start_time = 0; sentinel.previous_time = mstime(); sentinel.running_scripts = 0; sentinel.scripts_queue = listCreate(); @@ -1130,7 +1132,6 @@ int sentinelResetMastersByPattern(char *pattern, int flags) { * TODO: make this reset so that original sentinels are re-added with * same ip / port / runid. */ - int sentinelResetMasterAndChangeAddress(sentinelRedisInstance *master, char *ip, int port) { sentinelAddr *oldaddr, *newaddr; @@ -1139,12 +1140,26 @@ int sentinelResetMasterAndChangeAddress(sentinelRedisInstance *master, char *ip, sentinelResetMaster(master,SENTINEL_NO_FLAGS); oldaddr = master->addr; master->addr = newaddr; + master->o_down_since_time = 0; + master->s_down_since_time = 0; + /* Release the old address at the end so we are safe even if the function * gets the master->addr->ip and master->addr->port as arguments. */ releaseSentinelAddr(oldaddr); return REDIS_OK; } +/* Return non-zero if there was no SDOWN or ODOWN error associated to this + * instance in the latest 'ms' milliseconds. */ +int sentinelRedisInstanceNoDownFor(sentinelRedisInstance *ri, mstime_t ms) { + mstime_t most_recent; + + most_recent = ri->s_down_since_time; + if (ri->o_down_since_time > most_recent) + most_recent = ri->o_down_since_time; + return most_recent == 0 || (mstime() - most_recent) > ms; +} + /* ============================ Config handling ============================= */ char *sentinelHandleConfiguration(char **argv, int argc) { sentinelRedisInstance *ri; @@ -1382,26 +1397,39 @@ void sentinelRefreshInstanceInfo(sentinelRedisInstance *ri, const char *info) { } } - /* slave0:<ip>,<port>,<state> */ + /* old versions: slave0:<ip>,<port>,<state> + * new versions: slave0:ip=127.0.0.1,port=9999,... */ if ((ri->flags & SRI_MASTER) && sdslen(l) >= 7 && !memcmp(l,"slave",5) && isdigit(l[5])) { char *ip, *port, *end; - ip = strchr(l,':'); if (!ip) continue; - ip++; /* Now ip points to start of ip address. */ - port = strchr(ip,','); if (!port) continue; - *port = '\0'; /* nul term for easy access. */ - port++; /* Now port points to start of port number. */ - end = strchr(port,','); if (!end) continue; - *end = '\0'; /* nul term for easy access. */ + if (strstr(l,"ip=") == NULL) { + /* Old format. */ + ip = strchr(l,':'); if (!ip) continue; + ip++; /* Now ip points to start of ip address. */ + port = strchr(ip,','); if (!port) continue; + *port = '\0'; /* nul term for easy access. */ + port++; /* Now port points to start of port number. */ + end = strchr(port,','); if (!end) continue; + *end = '\0'; /* nul term for easy access. */ + } else { + /* New format. */ + ip = strstr(l,"ip="); if (!ip) continue; + ip += 3; /* Now ip points to start of ip address. */ + port = strstr(l,"port="); if (!port) continue; + port += 5; /* Now port points to start of port number. */ + /* Nul term both fields for easy access. */ + end = strchr(ip,','); if (end) *end = '\0'; + end = strchr(port,','); if (end) *end = '\0'; + } /* Check if we already have this slave into our table, * otherwise add it. */ if (sentinelRedisInstanceLookupSlave(ri,ip,atoi(port)) == NULL) { if ((slave = createSentinelRedisInstance(NULL,SRI_SLAVE,ip, - atoi(port), ri->quorum,ri)) != NULL) + atoi(port), ri->quorum, ri)) != NULL) { sentinelEvent(REDIS_NOTICE,"+slave",slave,"%@"); } @@ -1446,44 +1474,80 @@ void sentinelRefreshInstanceInfo(sentinelRedisInstance *ri, const char *info) { ri->info_refresh = mstime(); sdsfreesplitres(lines,numlines); - /* ---------------------------- Acting half ----------------------------- */ - if (sentinel.tilt) return; + /* ---------------------------- Acting half ----------------------------- + * Some things will not happen if sentinel.tilt is true, but some will + * still be processed. */ - /* Act if a master turned into a slave. */ - if ((ri->flags & SRI_MASTER) && role == SRI_SLAVE) { - if ((first_runid || runid_changed) && ri->slave_master_host) { - /* If it is the first time we receive INFO from it, but it's - * a slave while it was configured as a master, we want to monitor - * its master instead. */ - sentinelEvent(REDIS_WARNING,"+redirect-to-master",ri, - "%s %s %d %s %d", - ri->name, ri->addr->ip, ri->addr->port, - ri->slave_master_host, ri->slave_master_port); - sentinelResetMasterAndChangeAddress(ri,ri->slave_master_host, - ri->slave_master_port); - return; - } + /* When what we believe is our master, turned into a slave, the wiser + * thing we can do is to follow the events and redirect to the new + * master, always. */ + if ((ri->flags & SRI_MASTER) && role == SRI_SLAVE && ri->slave_master_host) + { + sentinelEvent(REDIS_WARNING,"+redirect-to-master",ri, + "%s %s %d %s %d", + ri->name, ri->addr->ip, ri->addr->port, + ri->slave_master_host, ri->slave_master_port); + sentinelResetMasterAndChangeAddress(ri,ri->slave_master_host, + ri->slave_master_port); + return; /* Don't process anything after this event. */ } - /* Act if a slave turned into a master. */ + /* Handle slave -> master role switch. */ if ((ri->flags & SRI_SLAVE) && role == SRI_MASTER) { - if (!(ri->master->flags & SRI_FAILOVER_IN_PROGRESS) && - (runid_changed || first_runid)) + if (!sentinel.tilt && ri->flags & SRI_DEMOTE) { + /* If this sentinel was partitioned from the slave's master, + * or tilted recently, wait some time before to act, + * so that DOWN and roles INFO will be refreshed. */ + mstime_t wait_time = SENTINEL_INFO_PERIOD*2 + + ri->master->down_after_period*2; + + if (!sentinelRedisInstanceNoDownFor(ri->master,wait_time) || + (mstime()-sentinel.tilt_start_time) < wait_time) + return; + + /* Old master returned back? Turn it into a slave ASAP if + * we can reach what we believe is the new master now, and + * have a recent role information for it. + * + * Note: we'll clear the DEMOTE flag only when we have the + * acknowledge that it's a slave again. */ + if (ri->master->flags & SRI_MASTER && + (ri->master->flags & (SRI_S_DOWN|SRI_O_DOWN)) == 0 && + (mstime() - ri->master->info_refresh) < SENTINEL_INFO_PERIOD*2) + { + int retval; + retval = redisAsyncCommand(ri->cc, + sentinelDiscardReplyCallback, NULL, "SLAVEOF %s %d", + ri->master->addr->ip, + ri->master->addr->port); + if (retval == REDIS_OK) + sentinelEvent(REDIS_NOTICE,"+demote-old-slave",ri,"%@"); + } else { + /* Otherwise if there are not the conditions to demote, we + * no longer trust the DEMOTE flag and remove it. */ + ri->flags &= ~SRI_DEMOTE; + sentinelEvent(REDIS_NOTICE,"-demote-flag-cleared",ri,"%@"); + } + } else if (!(ri->master->flags & SRI_FAILOVER_IN_PROGRESS) && + (runid_changed || first_runid)) { /* If a slave turned into master but: * * 1) Failover not in progress. - * 2) RunID hs changed, or its the first time we see an INFO output. + * 2) RunID has changed or its the first time we see an INFO output. * * We assume this is a reboot with a wrong configuration. - * Log the event and remove the slave. */ + * Log the event and remove the slave. Note that this is processed + * in tilt mode as well, otherwise we lose the information that the + * runid changed (reboot?) and when the tilt mode ends a fake + * failover will be detected. */ int retval; sentinelEvent(REDIS_WARNING,"-slave-restart-as-master",ri,"%@ #removing it from the attached slaves"); retval = dictDelete(ri->master->slaves,ri->name); redisAssert(retval == REDIS_OK); return; - } else if (ri->flags & SRI_PROMOTED) { + } else if (!sentinel.tilt && ri->flags & SRI_PROMOTED) { /* If this is a promoted slave we can change state to the * failover state machine. */ if ((ri->master->flags & SRI_FAILOVER_IN_PROGRESS) && @@ -1499,11 +1563,12 @@ void sentinelRefreshInstanceInfo(sentinelRedisInstance *ri, const char *info) { sentinelCallClientReconfScript(ri->master,SENTINEL_LEADER, "start",ri->master->addr,ri->addr); } - } else if (!(ri->master->flags & SRI_FAILOVER_IN_PROGRESS) || - ((ri->master->flags & SRI_FAILOVER_IN_PROGRESS) && - (ri->master->flags & SRI_I_AM_THE_LEADER) && - ri->master->failover_state == - SENTINEL_FAILOVER_STATE_WAIT_START)) + } else if (!sentinel.tilt && ( + !(ri->master->flags & SRI_FAILOVER_IN_PROGRESS) || + ((ri->master->flags & SRI_FAILOVER_IN_PROGRESS) && + (ri->master->flags & SRI_I_AM_THE_LEADER) && + ri->master->failover_state == + SENTINEL_FAILOVER_STATE_WAIT_START))) { /* No failover in progress? Then it is the start of a failover * and we are an observer. @@ -1523,6 +1588,7 @@ void sentinelRefreshInstanceInfo(sentinelRedisInstance *ri, const char *info) { ri->master->failover_state_change_time = mstime(); ri->master->promoted_slave = ri; ri->flags |= SRI_PROMOTED; + ri->flags &= ~SRI_DEMOTE; sentinelCallClientReconfScript(ri->master,SENTINEL_OBSERVER, "start", ri->master->addr,ri->addr); /* We are an observer, so we can only assume that the leader @@ -1534,6 +1600,10 @@ void sentinelRefreshInstanceInfo(sentinelRedisInstance *ri, const char *info) { } } + /* None of the following conditions are processed when in tilt mode, so + * return asap. */ + if (sentinel.tilt) return; + /* Detect if the slave that is in the process of being reconfigured * changed state. */ if ((ri->flags & SRI_SLAVE) && role == SRI_SLAVE && @@ -1564,6 +1634,13 @@ void sentinelRefreshInstanceInfo(sentinelRedisInstance *ri, const char *info) { ri->failover_state_change_time = mstime(); } } + + /* Detect if the old master was demoted as slave and generate the + * +slave event. */ + if (role == SRI_SLAVE && ri->flags & SRI_DEMOTE) { + sentinelEvent(REDIS_NOTICE,"+slave",ri,"%@"); + ri->flags &= ~SRI_DEMOTE; + } } void sentinelInfoReplyCallback(redisAsyncContext *c, void *reply, void *privdata) { @@ -1835,6 +1912,7 @@ void addReplySentinelRedisInstance(redisClient *c, sentinelRedisInstance *ri) { if (ri->flags & SRI_RECONF_SENT) flags = sdscat(flags,"reconf_sent,"); if (ri->flags & SRI_RECONF_INPROG) flags = sdscat(flags,"reconf_inprog,"); if (ri->flags & SRI_RECONF_DONE) flags = sdscat(flags,"reconf_done,"); + if (ri->flags & SRI_DEMOTE) flags = sdscat(flags,"demote,"); if (sdslen(flags) != 0) flags = sdsrange(flags,0,-2); /* remove last "," */ addReplyBulkCString(c,flags); @@ -2592,6 +2670,7 @@ sentinelRedisInstance *sentinelSelectSlave(sentinelRedisInstance *master) { mstime_t info_validity_time = mstime()-SENTINEL_INFO_VALIDITY_TIME; if (slave->flags & (SRI_S_DOWN|SRI_O_DOWN|SRI_DISCONNECTED)) continue; + if (slave->flags & SRI_DEMOTE) continue; /* Old master not yet ready. */ if (slave->last_avail_time < info_validity_time) continue; if (slave->slave_priority == 0) continue; @@ -2837,12 +2916,28 @@ void sentinelFailoverReconfNextSlave(sentinelRedisInstance *master) { void sentinelFailoverSwitchToPromotedSlave(sentinelRedisInstance *master) { sentinelRedisInstance *ref = master->promoted_slave ? master->promoted_slave : master; + sds old_master_ip; + int old_master_port; sentinelEvent(REDIS_WARNING,"+switch-master",master,"%s %s %d %s %d", master->name, master->addr->ip, master->addr->port, ref->addr->ip, ref->addr->port); + old_master_ip = sdsdup(master->addr->ip); + old_master_port = master->addr->port; sentinelResetMasterAndChangeAddress(master,ref->addr->ip,ref->addr->port); + /* If this is a real switch, that is, we have master->promoted_slave not + * NULL, then we want to add the old master as a slave of the new master, + * but flagging it with SRI_DEMOTE so that we know we'll need to send + * SLAVEOF once the old master is reachable again. */ + if (master != ref) { + /* Add the new slave, but don't generate a Sentinel event as it will + * happen later when finally the instance will claim to be a slave + * in the INFO output. */ + createSentinelRedisInstance(NULL,SRI_SLAVE|SRI_DEMOTE, + old_master_ip, old_master_port, master->quorum, master); + } + sdsfree(old_master_ip); } void sentinelFailoverStateMachine(sentinelRedisInstance *ri) { diff --git a/src/t_string.c b/src/t_string.c index 0a7f22583..389ff86a8 100644 --- a/src/t_string.c +++ b/src/t_string.c @@ -42,7 +42,27 @@ static int checkStringLength(redisClient *c, long long size) { return REDIS_OK; } -void setGenericCommand(redisClient *c, int nx, robj *key, robj *val, robj *expire, int unit) { +/* The setGenericCommand() function implements the SET operation with different + * options and variants. This function is called in order to implement the + * following commands: SET, SETEX, PSETEX, SETNX. + * + * 'flags' changes the behavior of the command (NX or XX, see belove). + * + * 'expire' represents an expire to set in form of a Redis object as passed + * by the user. It is interpreted according to the specified 'unit'. + * + * 'ok_reply' and 'abort_reply' is what the function will reply to the client + * if the operation is performed, or when it is not because of NX or + * XX flags. + * + * If ok_reply is NULL "+OK" is used. + * If abort_reply is NULL, "$-1" is used. */ + +#define REDIS_SET_NO_FLAGS 0 +#define REDIS_SET_NX (1<<0) /* Set if key not exists. */ +#define REDIS_SET_XX (1<<1) /* Set if key exists. */ + +void setGenericCommand(redisClient *c, int flags, robj *key, robj *val, robj *expire, int unit, robj *ok_reply, robj *abort_reply) { long long milliseconds = 0; /* initialized to avoid any harmness warning */ if (expire) { @@ -55,34 +75,68 @@ void setGenericCommand(redisClient *c, int nx, robj *key, robj *val, robj *expir if (unit == UNIT_SECONDS) milliseconds *= 1000; } - if (nx && lookupKeyWrite(c->db,key) != NULL) { - addReply(c,shared.czero); + if ((flags & REDIS_SET_NX && lookupKeyWrite(c->db,key) != NULL) || + (flags & REDIS_SET_XX && lookupKeyWrite(c->db,key) == NULL)) + { + addReply(c, abort_reply ? abort_reply : shared.nullbulk); return; } setKey(c->db,key,val); server.dirty++; if (expire) setExpire(c->db,key,mstime()+milliseconds); - addReply(c, nx ? shared.cone : shared.ok); + addReply(c, ok_reply ? ok_reply : shared.ok); } +/* SET key value [NX] [XX] [EX <seconds>] [PX <milliseconds>] */ void setCommand(redisClient *c) { + int j; + robj *expire = NULL; + int unit = UNIT_SECONDS; + int flags = REDIS_SET_NO_FLAGS; + + for (j = 3; j < c->argc; j++) { + char *a = c->argv[j]->ptr; + robj *next = (j == c->argc-1) ? NULL : c->argv[j+1]; + + if ((a[0] == 'n' || a[0] == 'N') && + (a[1] == 'x' || a[1] == 'X') && a[2] == '\0') { + flags |= REDIS_SET_NX; + } else if ((a[0] == 'x' || a[0] == 'X') && + (a[1] == 'x' || a[1] == 'X') && a[2] == '\0') { + flags |= REDIS_SET_XX; + } else if ((a[0] == 'e' || a[0] == 'E') && + (a[1] == 'x' || a[1] == 'X') && a[2] == '\0' && next) { + unit = UNIT_SECONDS; + expire = next; + j++; + } else if ((a[0] == 'p' || a[0] == 'P') && + (a[1] == 'x' || a[1] == 'X') && a[2] == '\0' && next) { + unit = UNIT_MILLISECONDS; + expire = next; + j++; + } else { + addReply(c,shared.syntaxerr); + return; + } + } + c->argv[2] = tryObjectEncoding(c->argv[2]); - setGenericCommand(c,0,c->argv[1],c->argv[2],NULL,0); + setGenericCommand(c,flags,c->argv[1],c->argv[2],expire,unit,NULL,NULL); } void setnxCommand(redisClient *c) { c->argv[2] = tryObjectEncoding(c->argv[2]); - setGenericCommand(c,1,c->argv[1],c->argv[2],NULL,0); + setGenericCommand(c,REDIS_SET_NX,c->argv[1],c->argv[2],NULL,0,shared.cone,shared.czero); } void setexCommand(redisClient *c) { c->argv[3] = tryObjectEncoding(c->argv[3]); - setGenericCommand(c,0,c->argv[1],c->argv[3],c->argv[2],UNIT_SECONDS); + setGenericCommand(c,REDIS_SET_NO_FLAGS,c->argv[1],c->argv[3],c->argv[2],UNIT_SECONDS,NULL,NULL); } void psetexCommand(redisClient *c) { c->argv[3] = tryObjectEncoding(c->argv[3]); - setGenericCommand(c,0,c->argv[1],c->argv[3],c->argv[2],UNIT_MILLISECONDS); + setGenericCommand(c,REDIS_SET_NO_FLAGS,c->argv[1],c->argv[3],c->argv[2],UNIT_MILLISECONDS,NULL,NULL); } int getGenericCommand(redisClient *c) { diff --git a/src/version.h b/src/version.h index 765dc9100..2872b9fe7 100644 --- a/src/version.h +++ b/src/version.h @@ -1 +1 @@ -#define REDIS_VERSION "2.6.10" +#define REDIS_VERSION "2.6.14" diff --git a/tests/assets/default.conf b/tests/assets/default.conf index c76eca3eb..35a6525de 100644 --- a/tests/assets/default.conf +++ b/tests/assets/default.conf @@ -1,344 +1,23 @@ -# Redis configuration file example +# Redis configuration for testing. -# Note on units: when memory size is needed, it is possible to specifiy -# it in the usual form of 1k 5GB 4M and so forth: -# -# 1k => 1000 bytes -# 1kb => 1024 bytes -# 1m => 1000000 bytes -# 1mb => 1024*1024 bytes -# 1g => 1000000000 bytes -# 1gb => 1024*1024*1024 bytes -# -# units are case insensitive so 1GB 1Gb 1gB are all the same. - -# By default Redis does not run as a daemon. Use 'yes' if you need it. -# Note that Redis will write a pid file in /var/run/redis.pid when daemonized. daemonize no - -# When running daemonized, Redis writes a pid file in /var/run/redis.pid by -# default. You can specify a custom pid file location here. pidfile /var/run/redis.pid - -# Accept connections on the specified port, default is 6379. port 6379 - -# If you want you can bind a single interface, if the bind option is not -# specified all the interfaces will listen for incoming connections. -# -# bind 127.0.0.1 - -# Specify the path for the unix socket that will be used to listen for -# incoming connections. There is no default, so Redis will not listen -# on a unix socket when not specified. -# -# unixsocket /tmp/redis.sock - -# Close the connection after a client is idle for N seconds (0 to disable) timeout 0 - -# Set server verbosity to 'debug' -# it can be one of: -# debug (a lot of information, useful for development/testing) -# verbose (many rarely useful info, but not a mess like the debug level) -# notice (moderately verbose, what you want in production probably) -# warning (only very important / critical messages are logged) loglevel verbose - -# Specify the log file name. Also 'stdout' can be used to force -# Redis to log on the standard output. Note that if you use standard -# output for logging but daemonize, logs will be sent to /dev/null logfile stdout - -# To enable logging to the system logger, just set 'syslog-enabled' to yes, -# and optionally update the other syslog parameters to suit your needs. -# syslog-enabled no - -# Specify the syslog identity. -# syslog-ident redis - -# Specify the syslog facility. Must be USER or between LOCAL0-LOCAL7. -# syslog-facility local0 - -# Set the number of databases. The default database is DB 0, you can select -# a different one on a per-connection basis using SELECT <dbid> where -# dbid is a number between 0 and 'databases'-1 databases 16 -################################ SNAPSHOTTING ################################# -# -# Save the DB on disk: -# -# save <seconds> <changes> -# -# Will save the DB if both the given number of seconds and the given -# number of write operations against the DB occurred. -# -# In the example below the behaviour will be to save: -# after 900 sec (15 min) if at least 1 key changed -# after 300 sec (5 min) if at least 10 keys changed -# after 60 sec if at least 10000 keys changed -# -# Note: you can disable saving at all commenting all the "save" lines. - save 900 1 save 300 10 save 60 10000 -# Compress string objects using LZF when dump .rdb databases? -# For default that's set to 'yes' as it's almost always a win. -# If you want to save some CPU in the saving child set it to 'no' but -# the dataset will likely be bigger if you have compressible values or keys. rdbcompression yes - -# The filename where to dump the DB dbfilename dump.rdb - -# The working directory. -# -# The DB will be written inside this directory, with the filename specified -# above using the 'dbfilename' configuration directive. -# -# Also the Append Only File will be created inside this directory. -# -# Note that you must specify a directory here, not a file name. dir ./ -################################# REPLICATION ################################# - -# Master-Slave replication. Use slaveof to make a Redis instance a copy of -# another Redis server. Note that the configuration is local to the slave -# so for example it is possible to configure the slave to save the DB with a -# different interval, or to listen to another port, and so on. -# -# slaveof <masterip> <masterport> - -# If the master is password protected (using the "requirepass" configuration -# directive below) it is possible to tell the slave to authenticate before -# starting the replication synchronization process, otherwise the master will -# refuse the slave request. -# -# masterauth <master-password> - -# When a slave lost the connection with the master, or when the replication -# is still in progress, the slave can act in two different ways: -# -# 1) if slave-serve-stale-data is set to 'yes' (the default) the slave will -# still reply to client requests, possibly with out of data data, or the -# data set may just be empty if this is the first synchronization. -# -# 2) if slave-serve-stale data is set to 'no' the slave will reply with -# an error "SYNC with master in progress" to all the kind of commands -# but to INFO and SLAVEOF. -# slave-serve-stale-data yes - -################################## SECURITY ################################### - -# Require clients to issue AUTH <PASSWORD> before processing any other -# commands. This might be useful in environments in which you do not trust -# others with access to the host running redis-server. -# -# This should stay commented out for backward compatibility and because most -# people do not need auth (e.g. they run their own servers). -# -# Warning: since Redis is pretty fast an outside user can try up to -# 150k passwords per second against a good box. This means that you should -# use a very strong password otherwise it will be very easy to break. -# -# requirepass foobared - -# Command renaming. -# -# It is possilbe to change the name of dangerous commands in a shared -# environment. For instance the CONFIG command may be renamed into something -# of hard to guess so that it will be still available for internal-use -# tools but not available for general clients. -# -# Example: -# -# rename-command CONFIG b840fc02d524045429941cc15f59e41cb7be6c52 -# -# It is also possilbe to completely kill a command renaming it into -# an empty string: -# -# rename-command CONFIG "" - -################################### LIMITS #################################### - -# Set the max number of connected clients at the same time. By default there -# is no limit, and it's up to the number of file descriptors the Redis process -# is able to open. The special value '0' means no limits. -# Once the limit is reached Redis will close all the new connections sending -# an error 'max number of clients reached'. -# -# maxclients 128 - -# Don't use more memory than the specified amount of bytes. -# When the memory limit is reached Redis will try to remove keys with an -# EXPIRE set. It will try to start freeing keys that are going to expire -# in little time and preserve keys with a longer time to live. -# Redis will also try to remove objects from free lists if possible. -# -# If all this fails, Redis will start to reply with errors to commands -# that will use more memory, like SET, LPUSH, and so on, and will continue -# to reply to most read-only commands like GET. -# -# WARNING: maxmemory can be a good idea mainly if you want to use Redis as a -# 'state' server or cache, not as a real DB. When Redis is used as a real -# database the memory usage will grow over the weeks, it will be obvious if -# it is going to use too much memory in the long run, and you'll have the time -# to upgrade. With maxmemory after the limit is reached you'll start to get -# errors for write operations, and this may even lead to DB inconsistency. -# -# maxmemory <bytes> - -# MAXMEMORY POLICY: how Redis will select what to remove when maxmemory -# is reached? You can select among five behavior: -# -# volatile-lru -> remove the key with an expire set using an LRU algorithm -# allkeys-lru -> remove any key accordingly to the LRU algorithm -# volatile-random -> remove a random key with an expire set -# allkeys->random -> remove a random key, any key -# volatile-ttl -> remove the key with the nearest expire time (minor TTL) -# noeviction -> don't expire at all, just return an error on write operations -# -# Note: with all the kind of policies, Redis will return an error on write -# operations, when there are not suitable keys for eviction. -# -# At the date of writing this commands are: set setnx setex append -# incr decr rpush lpush rpushx lpushx linsert lset rpoplpush sadd -# sinter sinterstore sunion sunionstore sdiff sdiffstore zadd zincrby -# zunionstore zinterstore hset hsetnx hmset hincrby incrby decrby -# getset mset msetnx exec sort -# -# The default is: -# -# maxmemory-policy volatile-lru - -# LRU and minimal TTL algorithms are not precise algorithms but approximated -# algorithms (in order to save memory), so you can select as well the sample -# size to check. For instance for default Redis will check three keys and -# pick the one that was used less recently, you can change the sample size -# using the following configuration directive. -# -# maxmemory-samples 3 - -############################## APPEND ONLY MODE ############################### - -# By default Redis asynchronously dumps the dataset on disk. If you can live -# with the idea that the latest records will be lost if something like a crash -# happens this is the preferred way to run Redis. If instead you care a lot -# about your data and don't want to that a single record can get lost you should -# enable the append only mode: when this mode is enabled Redis will append -# every write operation received in the file appendonly.aof. This file will -# be read on startup in order to rebuild the full dataset in memory. -# -# Note that you can have both the async dumps and the append only file if you -# like (you have to comment the "save" statements above to disable the dumps). -# Still if append only mode is enabled Redis will load the data from the -# log file at startup ignoring the dump.rdb file. -# -# IMPORTANT: Check the BGREWRITEAOF to check how to rewrite the append -# log file in background when it gets too big. - appendonly no - -# The name of the append only file (default: "appendonly.aof") -# appendfilename appendonly.aof - -# The fsync() call tells the Operating System to actually write data on disk -# instead to wait for more data in the output buffer. Some OS will really flush -# data on disk, some other OS will just try to do it ASAP. -# -# Redis supports three different modes: -# -# no: don't fsync, just let the OS flush the data when it wants. Faster. -# always: fsync after every write to the append only log . Slow, Safest. -# everysec: fsync only if one second passed since the last fsync. Compromise. -# -# The default is "everysec" that's usually the right compromise between -# speed and data safety. It's up to you to understand if you can relax this to -# "no" that will will let the operating system flush the output buffer when -# it wants, for better performances (but if you can live with the idea of -# some data loss consider the default persistence mode that's snapshotting), -# or on the contrary, use "always" that's very slow but a bit safer than -# everysec. -# -# If unsure, use "everysec". - -# appendfsync always appendfsync everysec -# appendfsync no - -# When the AOF fsync policy is set to always or everysec, and a background -# saving process (a background save or AOF log background rewriting) is -# performing a lot of I/O against the disk, in some Linux configurations -# Redis may block too long on the fsync() call. Note that there is no fix for -# this currently, as even performing fsync in a different thread will block -# our synchronous write(2) call. -# -# In order to mitigate this problem it's possible to use the following option -# that will prevent fsync() from being called in the main process while a -# BGSAVE or BGREWRITEAOF is in progress. -# -# This means that while another child is saving the durability of Redis is -# the same as "appendfsync none", that in pratical terms means that it is -# possible to lost up to 30 seconds of log in the worst scenario (with the -# default Linux settings). -# -# If you have latency problems turn this to "yes". Otherwise leave it as -# "no" that is the safest pick from the point of view of durability. no-appendfsync-on-rewrite no - -############################### ADVANCED CONFIG ############################### - -# Hashes are encoded in a special way (much more memory efficient) when they -# have at max a given numer of elements, and the biggest element does not -# exceed a given threshold. You can configure this limits with the following -# configuration directives. -hash-max-ziplist-entries 64 -hash-max-ziplist-value 512 - -# Similarly to hashes, small lists are also encoded in a special way in order -# to save a lot of space. The special representation is only used when -# you are under the following limits: -list-max-ziplist-entries 512 -list-max-ziplist-value 64 - -# Sets have a special encoding in just one case: when a set is composed -# of just strings that happens to be integers in radix 10 in the range -# of 64 bit signed integers. -# The following configuration setting sets the limit in the size of the -# set in order to use this special memory saving encoding. -set-max-intset-entries 512 - -# Active rehashing uses 1 millisecond every 100 milliseconds of CPU time in -# order to help rehashing the main Redis hash table (the one mapping top-level -# keys to values). The hash table implementation redis uses (see dict.c) -# performs a lazy rehashing: the more operation you run into an hash table -# that is rhashing, the more rehashing "steps" are performed, so if the -# server is idle the rehashing is never complete and some more memory is used -# by the hash table. -# -# The default is to use this millisecond 10 times every second in order to -# active rehashing the main dictionaries, freeing memory when possible. -# -# If unsure: -# use "activerehashing no" if you have hard latency requirements and it is -# not a good thing in your environment that Redis can reply form time to time -# to queries with 2 milliseconds delay. -# -# use "activerehashing yes" if you don't have such hard requirements but -# want to free memory asap when possible. activerehashing yes - -################################## INCLUDES ################################### - -# Include one or more other config files here. This is useful if you -# have a standard template that goes to all redis server but also need -# to customize a few per-server settings. Include files can include -# other files, so use this wisely. -# -# include /path/to/local.conf -# include /path/to/other.conf diff --git a/tests/integration/rdb.tcl b/tests/integration/rdb.tcl index 85c5db9f5..71876a6ed 100644 --- a/tests/integration/rdb.tcl +++ b/tests/integration/rdb.tcl @@ -23,3 +23,76 @@ start_server [list overrides [list "dir" $server_path "dbfilename" "encodings.rd } } +set server_path [tmpdir "server.rdb-startup-test"] + +start_server [list overrides [list "dir" $server_path]] { + test {Server started empty with non-existing RDB file} { + r debug digest + } {0000000000000000000000000000000000000000} + # Save an RDB file, needed for the next test. + r save +} + +start_server [list overrides [list "dir" $server_path]] { + test {Server started empty with empty RDB file} { + r debug digest + } {0000000000000000000000000000000000000000} +} + +# Helper function to start a server and kill it, just to check the error +# logged. +set defaults {} +proc start_server_and_kill_it {overrides code} { + upvar defaults defaults srv srv server_path server_path + set config [concat $defaults $overrides] + set srv [start_server [list overrides $config]] + uplevel 1 $code + kill_server $srv +} + +# Make the RDB file unreadable +file attributes [file join $server_path dump.rdb] -permissions 0222 + +# Detect root account (it is able to read the file even with 002 perm) +set isroot 0 +catch { + open [file join $server_path dump.rdb] + set isroot 1 +} + +# Now make sure the server aborted with an error +if {!$isroot} { + start_server_and_kill_it [list "dir" $server_path] { + test {Server should not start if RDB file can't be open} { + wait_for_condition 50 100 { + [string match {*Fatal error loading*} \ + [exec tail -n1 < [dict get $srv stdout]]] + } else { + fail "Server started even if RDB was unreadable!" + } + } + } +} + +# Fix permissions of the RDB file. +file attributes [file join $server_path dump.rdb] -permissions 0666 + +# Corrupt its CRC64 checksum. +set filesize [file size [file join $server_path dump.rdb]] +set fd [open [file join $server_path dump.rdb] r+] +fconfigure $fd -translation binary +seek $fd -8 end +puts -nonewline $fd "foobar00"; # Corrupt the checksum +close $fd + +# Now make sure the server aborted with an error +start_server_and_kill_it [list "dir" $server_path] { + test {Server should not start if RDB is corrupted} { + wait_for_condition 50 100 { + [string match {*RDB checksum*} \ + [exec tail -n1 < [dict get $srv stdout]]] + } else { + fail "Server started even if RDB was corrupted!" + } + } +} diff --git a/tests/integration/replication-4.tcl b/tests/integration/replication-4.tcl index 0a0dcdc2b..29ab48fc2 100644 --- a/tests/integration/replication-4.tcl +++ b/tests/integration/replication-4.tcl @@ -10,21 +10,22 @@ proc stop_bg_complex_data {handle} { start_server {tags {"repl"}} { start_server {} { - set master [srv 0 client] - set master_host [srv 0 host] - set master_port [srv 0 port] + set master [srv -1 client] + set master_host [srv -1 host] + set master_port [srv -1 port] + set slave [srv 0 client] + set load_handle0 [start_bg_complex_data $master_host $master_port 9 100000] set load_handle1 [start_bg_complex_data $master_host $master_port 11 100000] set load_handle2 [start_bg_complex_data $master_host $master_port 12 100000] test {First server should have role slave after SLAVEOF} { - r -1 slaveof [srv 0 host] [srv 0 port] + $slave slaveof $master_host $master_port after 1000 - s -1 role + s 0 role } {slave} test {Test replication with parallel clients writing in differnet DBs} { - lappend slave [srv 0 client] after 5000 stop_bg_complex_data $load_handle0 stop_bg_complex_data $load_handle1 @@ -37,7 +38,7 @@ start_server {tags {"repl"}} { } assert {[$master dbsize] > 0} - if {[r debug digest] ne [r -1 debug digest]} { + if {[$master debug digest] ne [$slave debug digest]} { set csv1 [csvdump r] set csv2 [csvdump {r -1}] set fd [open /tmp/repldump1.txt w] diff --git a/tests/test_helper.tcl b/tests/test_helper.tcl index 483774219..95e0c83dc 100644 --- a/tests/test_helper.tcl +++ b/tests/test_helper.tcl @@ -408,6 +408,65 @@ for {set j 0} {$j < [llength $argv]} {incr j} { } } +proc attach_to_replication_stream {} { + set s [socket [srv 0 "host"] [srv 0 "port"]] + fconfigure $s -translation binary + puts -nonewline $s "SYNC\r\n" + flush $s + + # Get the count + set count [gets $s] + set prefix [string range $count 0 0] + if {$prefix ne {$}} { + error "attach_to_replication_stream error. Received '$count' as count." + } + set count [string range $count 1 end] + + # Consume the bulk payload + while {$count} { + set buf [read $s $count] + set count [expr {$count-[string length $buf]}] + } + return $s +} + +proc read_from_replication_stream {s} { + fconfigure $s -blocking 0 + set attempt 0 + while {[gets $s count] == -1} { + if {[incr attempt] == 10} return "" + after 100 + } + fconfigure $s -blocking 1 + + # Workaround for Redis 2.6, not always using the new protocol in the + # replication channel (this was fixed in >= 2.8). + if {[string tolower [lindex [split $count] 0]] eq {select}} { + return $count + } + + # Return a list of arguments for the command. + set count [string range $count 1 end] + set res {} + for {set j 0} {$j < $count} {incr j} { + read $s 1 + set arg [::redis::redis_bulk_read $s] + if {$j == 0} {set arg [string tolower $arg]} + lappend res $arg + } + return $res +} + +proc assert_replication_stream {s patterns} { + for {set j 0} {$j < [llength $patterns]} {incr j} { + assert_match [lindex $patterns $j] [read_from_replication_stream $s] + } +} + +proc close_replication_stream {s} { + close $s +} + # With the parallel test running multiple Redis instances at the same time # we need a fast enough computer, otherwise a lot of tests may generate # false positives. diff --git a/tests/unit/basic.tcl b/tests/unit/basic.tcl index 4210f48bc..b128f9438 100644 --- a/tests/unit/basic.tcl +++ b/tests/unit/basic.tcl @@ -712,4 +712,46 @@ start_server {tags {"basic"}} { assert_equal [string range $bin $_start $_end] [r getrange bin $start $end] } } + + test {Extended SET can detect syntax errors} { + set e {} + catch {r set foo bar non-existing-option} e + set e + } {*syntax*} + + test {Extended SET NX option} { + r del foo + set v1 [r set foo 1 nx] + set v2 [r set foo 2 nx] + list $v1 $v2 [r get foo] + } {OK {} 1} + + test {Extended SET XX option} { + r del foo + set v1 [r set foo 1 xx] + r set foo bar + set v2 [r set foo 2 xx] + list $v1 $v2 [r get foo] + } {{} OK 2} + + test {Extended SET EX option} { + r del foo + r set foo bar ex 10 + set ttl [r ttl foo] + assert {$ttl <= 10 && $ttl > 5} + } + + test {Extended SET PX option} { + r del foo + r set foo bar px 10000 + set ttl [r ttl foo] + assert {$ttl <= 10 && $ttl > 5} + } + + test {Extended SET using multiple options at once} { + r set foo val + assert {[r set foo bar xx px 10000] eq {OK}} + set ttl [r ttl foo] + assert {$ttl <= 10 && $ttl > 5} + } } diff --git a/tests/unit/expire.tcl b/tests/unit/expire.tcl index 56a59f768..2b9fc8f4a 100644 --- a/tests/unit/expire.tcl +++ b/tests/unit/expire.tcl @@ -142,6 +142,34 @@ start_server {tags {"expire"}} { list $size1 $size2 } {3 0} + test {Redis should lazy expire keys} { + r flushdb + r debug set-active-expire 0 + r psetex key1 500 a + r psetex key2 500 a + r psetex key3 500 a + set size1 [r dbsize] + # Redis expires random keys ten times every second so we are + # fairly sure that all the three keys should be evicted after + # one second. + after 1000 + set size2 [r dbsize] + r mget key1 key2 key3 + set size3 [r dbsize] + r debug set-active-expire 1 + list $size1 $size2 $size3 + } {3 3 0} + + test {EXPIRE should not resurrect keys (issue #1026)} { + r debug set-active-expire 0 + r set foo bar + r pexpire foo 500 + after 1000 + r expire foo 10 + r debug set-active-expire 1 + r exists foo + } {0} + test {5 keys in, 5 keys out} { r flushdb r set a c diff --git a/tests/unit/introspection.tcl b/tests/unit/introspection.tcl index a6c2130f5..54742bb02 100644 --- a/tests/unit/introspection.tcl +++ b/tests/unit/introspection.tcl @@ -50,6 +50,10 @@ start_server {tags {"introspection"}} { assert_match {*foobar*} [r client list] $rd close # Now the client should no longer be listed - string match {*foobar*} [r client list] - } {0} + wait_for_condition 50 100 { + [string match {*foobar*} [r client list]] == 0 + } else { + fail "Client still listed in CLIENT LIST after SETNAME." + } + } } diff --git a/tests/unit/multi.tcl b/tests/unit/multi.tcl index f7a1f6468..6655bf62c 100644 --- a/tests/unit/multi.tcl +++ b/tests/unit/multi.tcl @@ -205,7 +205,10 @@ start_server {tags {"multi"}} { r select 5 r multi r ping - r exec + set res [r exec] + # Restore original DB + r select 9 + set res } {PONG} test {WATCH will consider touched keys target of EXPIRE} { @@ -249,4 +252,58 @@ start_server {tags {"multi"}} { r incr x r exec } {11} + + test {MULTI / EXEC is propagated correctly (single write command)} { + set repl [attach_to_replication_stream] + r multi + r set foo bar + r exec + assert_replication_stream $repl { + {select *} + {multi} + {set foo bar} + {exec} + } + close_replication_stream $repl + } + + test {MULTI / EXEC is propagated correctly (empty transaction)} { + set repl [attach_to_replication_stream] + r multi + r exec + r set foo bar + assert_replication_stream $repl { + {select *} + {set foo bar} + } + close_replication_stream $repl + } + + test {MULTI / EXEC is propagated correctly (read-only commands)} { + r set foo value1 + set repl [attach_to_replication_stream] + r multi + r get foo + r exec + r set foo value2 + assert_replication_stream $repl { + {select *} + {set foo value2} + } + close_replication_stream $repl + } + + test {MULTI / EXEC is propagated correctly (write command, no effect)} { + r del bar foo bar + set repl [attach_to_replication_stream] + r multi + r del foo + r exec + assert_replication_stream $repl { + {select *} + {multi} + {exec} + } + close_replication_stream $repl + } } diff --git a/tests/unit/obuf-limits.tcl b/tests/unit/obuf-limits.tcl index e965e892d..5d625cf45 100644 --- a/tests/unit/obuf-limits.tcl +++ b/tests/unit/obuf-limits.tcl @@ -15,7 +15,7 @@ start_server {tags {"obuf-limits"}} { if {![regexp {omem=([0-9]+)} $c - omem]} break if {$omem > 200000} break } - assert {$omem >= 99000 && $omem < 200000} + assert {$omem >= 90000 && $omem < 200000} $rd1 close } diff --git a/tests/unit/scripting.tcl b/tests/unit/scripting.tcl index 64cf00d55..bbb6b5fd1 100644 --- a/tests/unit/scripting.tcl +++ b/tests/unit/scripting.tcl @@ -281,6 +281,23 @@ start_server {tags {"scripting"}} { assert_equal $rand1 $rand2 assert {$rand2 ne $rand3} } + + test {EVAL processes writes from AOF in read-only slaves} { + r flushall + r config set appendonly yes + r eval {redis.call("set","foo","100")} 0 + r eval {redis.call("incr","foo")} 0 + r eval {redis.call("incr","foo")} 0 + wait_for_condition 50 100 { + [s aof_rewrite_in_progress] == 0 + } else { + fail "AOF rewrite can't complete after CONFIG SET appendonly yes." + } + r config set slave-read-only yes + r slaveof 127.0.0.1 0 + r debug loadaof + r get foo + } {102} } # Start a new server since the last test in this stanza will kill the |
