summaryrefslogtreecommitdiff
path: root/src
diff options
context:
space:
mode:
authorOran Agra <oran@redislabs.com>2021-12-02 21:41:58 +0200
committerGitHub <noreply@github.com>2021-12-02 21:41:58 +0200
commit64f6159646337b4a3b56a400522ad4d028d55dac (patch)
tree46fa2b01fc19d6e0a9d1de473b7f9c414105d43c /src
parente57a4db5d753e427317075071b0992fd965168b3 (diff)
parentcbd463175f8b52d594fd4e6b953fa58a5db053c3 (diff)
downloadredis-64f6159646337b4a3b56a400522ad4d028d55dac.tar.gz
Merge Redis Functions PR (#9780)
# Redis Function This PR added the Redis Functions capabilities that were suggested on #8693. The PR also introduce a big refactoring to the current Lua implementation (i.e `scripting.c`). The main purpose of the refactoring is to have better code sharing between the Lua implementation that exists today on Redis (`scripting.c`) and the new Lua engine that is introduced on this PR. The refactoring includes code movements and file name changes as well as some logic changes that need to be carefully reviewed. To make the review easier, the PR was split into multiple commits. Each commit is deeply described later on but the main concept is that some commits are just moving code around without making any logical changes, those commits are less likely to cause any issues or regressions and can be reviewed fast. Other commits, which perform code and logic changes, need to be reviewed carefully, but those commits were created after the code movements so it's pretty easy to see what was changed. To sum up, it is highly recommended to review this PR commit by commit as it will be easier to see the changes, it is also recommended to read each commit description (written below) to understand what was changed on the commit and whether or not it's just a huge code movement or a logic changes. ## Terminology Currently, the terminology in Redis is not clearly defined. Scripts refer to Lua scripts and eval also refers only to Lua. Introducing Redis Function requires redefining those terms to be able to clearly understand what is been discussed on each context. * eval - legacy Lua script implementation. * Function - new scripting implementation (currently implemented in Lua but in the future, it might be other languages like javascript). * Engine - the component that is responsible for executing functions. * Script - Function or legacy Lua (executed with `eval` or `evalsha`) ## Refactoring New Structure Today, the entire scripting logic is located on `scripting.c`. This logic can be split into 3 main groups: 1. Script management - responsible for storing the scripts that were sent to Redis and retrieving them when they need to be run (base on the script sha on the current implementation). 2. Script invocation - invoke the script given on `eval` or `evalsha` command (this part includes finding the relevant script, preparing the arguments, ..) 3. Interact back with Redis (command invocation) Those 3 groups are tightly coupled on `scripting.c`. Redis Functions also need to use those groups logics, for example, to interact back with Redis or to execute Lua code. The refactoring attempts to split those 3 groups and define APIs so that we can reuse the code both on legacy Lua scripts and Redis Functions. In order to do so we define the following units: 1. script.c: responsible for interaction with Redis from within a script. 2. script_lua.c: responsible to execute Lua code, uses `script.c` to interact with Redis from within the Lua code. 3. function_lua.c: contains the Lua engine implementation, uses `script_lua.c` to execute the Lua code. 4. functions.c: Contains Redis Functions implementation (`FUNCTION` command,), uses `functions_lua.c` if the function it wants to invoke needs the Lua engine. 4. eval.c: the original `scripting.c` contains the Lua legacy implementation and was refactored to use `script_lua.c` to invoke the Lua code. ## Commits breakdown Notice: Some small commits are omitted from this list as they are small and insignificant (for example build fixes) ### First commit - code movements This commit rename `scripting.c` -> `eval.c` and introduce the new `script_lua.c` unit. The commit moves relevant code from `eval.c` (`scripting.c`) to `script_lua.c`, the purpose of moving the code is so that later we will be able to re-use the code on the Lua engine (`function_lua.c`). The commit only moves the code without modifying even a single line, so there is a very low risk of breaking anything and it also makes it much easier to see the changes on the following commits. Because the commit does not change the code (only moves it), it does not compile. But we do not care about it as the only purpose here is to make the review processes simpler. ### Second commit - move legacy Lua variables into `eval.c` Today, all Lua-related variables are located on the server struct. The commit attempt to identify those variable and take them out from the server struct, leaving only script related variables (variables that later need to be used also by engines) The following variable where renamed and left on the server struct: * lua_caller -> script_caller * lua_time_limit -> script_time_limit * lua_timedout -> script_timedout * lua_oom -> script_oom * lua_disable_deny_script -> script_disable_deny_script * in_eval -> in_script The following variables where moved to lctx under eval.c * lua * lua_client * lua_cur_script * lua_scripts * lua_scripts_mem * lua_replicate_commands * lua_write_dirty * lua_random_dirty * lua_multi_emitted * lua_repl * lua_kill * lua_time_start * lua_time_snapshot This commit is in a low risk of introducing any issues and it is just moving variables around and not changing any logic. ### Third commit - introducing script unit This commit introduces the `script.c` unit. Its purpose (as described above) is to provide an API for scripts to interact with Redis. Interaction includes mostly executing commands, but also other functionalities. The interaction is done using a `ScriptRunCtx` object that needs to be created by the user and initialized using `scriptPrepareForRun`. A detailed list of functionalities expose by the unit: 1. Calling commands (including all the validation checks such as acl, cluster, read only run, ...) 2. Set Resp 3. Set Replication method (AOF/REPLICATION/NONE) 4. Call Redis back on long-running scripts to allow Redis to reply to clients and perform script kill The commit introduces the new unit and uses it on eval commands to interact with Redis. ### Fourth commit - Moved functionality of invoke Lua code to `script_lua.c` This commit moves the logic of invoking the Lua code into `script_lua.c` so later it can be used also by Lua engine (`function_lua.c`). The code is located on `callFunction` function and assumes the Lua function already located on the top of the Lua stack. This commit also change `eval.c` to use the new functionality to invoke Lua code. ### Fith commit - Added Redis Functions unit (`functions.c`) and Lua engine (`function_lua.c`) Added Redis Functions unit under `functions.c`, included: 1. FUNCTION command: * FUNCTION CREATE * FUNCTION CALL * FUNCTION DELETE * FUNCTION KILL * FUNCTION INFO * FUNCTION STATS 2. Register engines In addition, this commit introduces the first engine that uses the Redis Functions capabilities, the Lua engine (`function_lua.c`) ## API Changes ### `lua-time-limit` configuration was renamed to `script-time-limit` (keep `lua-time-limit` as alias for backward compatibility). ### Error log changes When integrating with Redis from within a Lua script, the `Lua` term was removed from all the error messages and instead we write only `script`. For example: `Wrong number of args calling Redis command From Lua script` -> `Wrong number of args calling Redis command From script` ### `info memory` changes: Before stating all the changes made to memory stats we will try to explain the reason behind them and what we want to see on those metrics: * memory metrics should show both totals (for all scripting frameworks), as well as a breakdown per framework / vm. * The totals metrics should have "human" metrics while the breakdown shouldn't. * We did try to maintain backward compatibility in some way, that said we did make some repurpose to existing metrics where it looks reasonable. * We separate between memory used by the script framework (part of redis's used_memory), and memory used by the VM (not part of redis's used_memory) A full breakdown of `info memory` changes: * `used_memory_lua` and `used_memory_lua_human` was deprecated, `used_memory_vm_eval` has the same meaning as `used_memory_lua` * `used_memory_scripts` was renamed to `used_memory_scripts_eval` * `used_memory_scripts` and `used_memory_scripts_human` were repurposed and now return the total memory used by functions and eval (not including vm memory, only code cache, and structs). * `used_memory_vm_function` was added and represents the total memory used by functions vm's * `used_memory_functions` was added and represents the total memory by functions (not including vm memory, only code cache, and structs) * `used_memory_vm_total` and `used_memory_vm_total_human` was added and represents the total memory used by vm's (functions and eval combined) ### `functions.caches` `functions.caches` field was added to `memory stats`, representing the memory used by engines that are not functions (this memory includes data structures like dictionaries, arrays, ...) ## New API ### FUNCTION CREATE Usage: FUNCTION CREATE `ENGINE` `NAME` `[REPLACE]` `[DESC <DESCRIPTION>]` `<CODE>` * `ENGINE` - The name of the engine to use to create the script. * `NAME` - the name of the function that can be used later to call the function using `FUNCTION CALL` command. * `REPLACE` - if given, replace the given function with the existing function (if exists). * `DESCRIPTION` - optional argument describing the function and what it does * `CODE` - function code. The command will return `OK` if created successfully or error in the following cases: * The given engine name does not exist * The function name is already taken and `REPLACE` was not used. * The given function failed on the compilation. ### FCALL and FCALL_RO Usage: FCALL/FCALL_RO `NAME` `NUM_KEYS key1 key2` … ` arg1 arg2` Call and execute the function specified by `NAME`. The function will receive all arguments given after `NUM_KEYS`. The return value from the function will be returned to the user as a result. * `NAME` - Name of the function to run. * The rest is as today with EVALSHA command. The command will return an error in the following cases: * `NAME` does not exist * The function itself returned an error. The `FCALL_RO` is equivalent to `EVAL_RO` and allows only read-only commands to be invoked from the script. ### FUNCTION DELETE Usage: FUNCTION DELETE `NAME` Delete a function identified by `NAME`. Return `OK` on success or error on one of the following: * The given function does not exist ### FUNCTION INFO Usage: FUNCTION INFO `NAME` [WITHCODE] Return information about a function by function name: * Function name * Engine name * Description * Raw code (only if WITHCODE argument is given) ### FUNCTION LIST Usage: FUNCTION LIST Return general information about all the functions: * Function name * Engine name * Description ### FUNCTION STATS Usage: FUNCTION STATS Return information about the current running function: * Function name * Command that was used to invoke the function * Duration in MS that the function is already running If no function is currently running, this section is just a RESP nil. Additionally, return a list of all the available engines. ### FUNCTION KILL Usage: `FUNCTION KILL` Kill the currently executing function. The command will fail if the function already initiated a write command. ## Notes Note: Function creation/deletion is replicated to AOF but AOFRW is not implemented sense its going to be removed: #9794
Diffstat (limited to 'src')
-rw-r--r--src/Makefile2
-rw-r--r--src/acl.c2
-rw-r--r--src/aof.c4
-rw-r--r--src/config.c2
-rw-r--r--src/db.c12
-rw-r--r--src/debug.c2
-rw-r--r--src/defrag.c2
-rw-r--r--src/eval.c1636
-rw-r--r--src/evict.c3
-rw-r--r--src/function_lua.c183
-rw-r--r--src/functions.c538
-rw-r--r--src/functions.h126
-rw-r--r--src/module.c8
-rw-r--r--src/networking.c7
-rw-r--r--src/object.c14
-rw-r--r--src/rdb.c110
-rw-r--r--src/rdb.h4
-rw-r--r--src/replication.c16
-rw-r--r--src/script.c464
-rw-r--r--src/script.h101
-rw-r--r--src/script_lua.c1348
-rw-r--r--src/script_lua.h67
-rw-r--r--src/scripting.c3063
-rw-r--r--src/server.c125
-rw-r--r--src/server.h58
-rw-r--r--src/sort.c2
-rw-r--r--src/t_stream.c2
27 files changed, 4754 insertions, 3147 deletions
diff --git a/src/Makefile b/src/Makefile
index 34b5c3566..1bc6d86ed 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -309,7 +309,7 @@ endif
REDIS_SERVER_NAME=redis-server$(PROG_SUFFIX)
REDIS_SENTINEL_NAME=redis-sentinel$(PROG_SUFFIX)
-REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o scripting.o bio.o rio.o rand.o memtest.o crcspeed.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o lolwut.o lolwut5.o lolwut6.o acl.o tracking.o connection.o tls.o sha256.o timeout.o setcpuaffinity.o monotonic.o mt19937-64.o resp_parser.o call_reply.o
+REDIS_SERVER_OBJ=adlist.o quicklist.o ae.o anet.o dict.o server.o sds.o zmalloc.o lzf_c.o lzf_d.o pqsort.o zipmap.o sha1.o ziplist.o release.o networking.o util.o object.o db.o replication.o rdb.o t_string.o t_list.o t_set.o t_zset.o t_hash.o config.o aof.o pubsub.o multi.o debug.o sort.o intset.o syncio.o cluster.o crc16.o endianconv.o slowlog.o eval.o bio.o rio.o rand.o memtest.o crcspeed.o crc64.o bitops.o sentinel.o notify.o setproctitle.o blocked.o hyperloglog.o latency.o sparkline.o redis-check-rdb.o redis-check-aof.o geo.o lazyfree.o module.o evict.o expire.o geohash.o geohash_helper.o childinfo.o defrag.o siphash.o rax.o t_stream.o listpack.o localtime.o lolwut.o lolwut5.o lolwut6.o acl.o tracking.o connection.o tls.o sha256.o timeout.o setcpuaffinity.o monotonic.o mt19937-64.o resp_parser.o call_reply.o script_lua.o script.o functions.o function_lua.o
REDIS_CLI_NAME=redis-cli$(PROG_SUFFIX)
REDIS_CLI_OBJ=anet.o adlist.o dict.o redis-cli.o zmalloc.o release.o ae.o redisassert.o crcspeed.o crc64.o siphash.o crc16.o monotonic.o cli_common.o mt19937-64.o
REDIS_BENCHMARK_NAME=redis-benchmark$(PROG_SUFFIX)
diff --git a/src/acl.c b/src/acl.c
index 5d1598b39..b9efd5401 100644
--- a/src/acl.c
+++ b/src/acl.c
@@ -1897,7 +1897,7 @@ void addACLLogEntry(client *c, int reason, int context, int argpos, sds username
}
client *realclient = c;
- if (realclient->flags & CLIENT_LUA) realclient = server.lua_caller;
+ if (realclient->flags & CLIENT_SCRIPT) realclient = server.script_caller;
le->cinfo = catClientInfoString(sdsempty(),realclient);
le->context = context;
diff --git a/src/aof.c b/src/aof.c
index 4b9900ab2..8f609edc6 100644
--- a/src/aof.c
+++ b/src/aof.c
@@ -30,6 +30,7 @@
#include "server.h"
#include "bio.h"
#include "rio.h"
+#include "functions.h"
#include <signal.h>
#include <fcntl.h>
@@ -754,7 +755,8 @@ int loadAppendOnlyFile(char *filename) {
serverLog(LL_NOTICE,"Reading RDB preamble from AOF file...");
if (fseek(fp,0,SEEK_SET) == -1) goto readerr;
rioInitWithFile(&rdb,fp);
- if (rdbLoadRio(&rdb,RDBFLAGS_AOF_PREAMBLE,NULL,server.db) != C_OK) {
+
+ if (rdbLoadRio(&rdb,RDBFLAGS_AOF_PREAMBLE,NULL) != C_OK) {
serverLog(LL_WARNING,"Error reading the RDB preamble of the AOF file, AOF loading aborted");
goto readerr;
} else {
diff --git a/src/config.c b/src/config.c
index 2b2ff737d..939a5342c 100644
--- a/src/config.c
+++ b/src/config.c
@@ -2650,7 +2650,7 @@ standardConfig configs[] = {
createULongConfig("acllog-max-len", NULL, MODIFIABLE_CONFIG, 0, LONG_MAX, server.acllog_max_len, 128, INTEGER_CONFIG, NULL, NULL),
/* Long Long configs */
- createLongLongConfig("lua-time-limit", NULL, MODIFIABLE_CONFIG, 0, LONG_MAX, server.lua_time_limit, 5000, INTEGER_CONFIG, NULL, NULL),/* milliseconds */
+ createLongLongConfig("script-time-limit", "lua-time-limit", MODIFIABLE_CONFIG, 0, LONG_MAX, server.script_time_limit, 5000, INTEGER_CONFIG, NULL, NULL),/* milliseconds */
createLongLongConfig("cluster-node-timeout", NULL, MODIFIABLE_CONFIG, 0, LLONG_MAX, server.cluster_node_timeout, 15000, INTEGER_CONFIG, NULL, NULL),
createLongLongConfig("slowlog-log-slower-than", NULL, MODIFIABLE_CONFIG, -1, LLONG_MAX, server.slowlog_log_slower_than, 10000, INTEGER_CONFIG, NULL, NULL),
createLongLongConfig("latency-monitor-threshold", NULL, MODIFIABLE_CONFIG, 0, LLONG_MAX, server.latency_monitor_threshold, 0, INTEGER_CONFIG, NULL, NULL),
diff --git a/src/db.c b/src/db.c
index b98310cf3..d0c8b5903 100644
--- a/src/db.c
+++ b/src/db.c
@@ -31,6 +31,7 @@
#include "cluster.h"
#include "atomicvar.h"
#include "latency.h"
+#include "script.h"
#include <signal.h>
#include <ctype.h>
@@ -88,7 +89,7 @@ robj *lookupKey(redisDb *db, robj *key, int flags) {
* commands is to make writable replicas behave consistently. It
* shall not be used in readonly commands. Modules are accepted so
* that we don't break old modules. */
- client *c = server.in_eval ? server.lua_client : server.current_client;
+ client *c = server.in_script ? scriptGetClient() : server.current_client;
serverAssert(!c || !c->cmd || (c->cmd->flags & (CMD_WRITE|CMD_MODULE)));
}
if (expireIfNeeded(db, key, force_delete_expired)) {
@@ -1501,8 +1502,8 @@ int keyIsExpired(redisDb *db, robj *key) {
* only the first time it is accessed and not in the middle of the
* script execution, making propagation to slaves / AOF consistent.
* See issue #1525 on Github for more information. */
- if (server.lua_caller) {
- now = server.lua_time_snapshot;
+ if (server.script_caller) {
+ now = evalTimeSnapshot();
}
/* If we are in the middle of a command execution, we still want to use
* a reference time that does not change: in that case we just use the
@@ -1743,6 +1744,11 @@ int evalGetKeys(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *
return genericGetKeys(0, 2, 3, 1, argv, argc, result);
}
+int functionGetKeys(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result) {
+ UNUSED(cmd);
+ return genericGetKeys(0, 2, 3, 1, argv, argc, result);
+}
+
int lmpopGetKeys(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result) {
UNUSED(cmd);
return genericGetKeys(0, 1, 2, 1, argv, argc, result);
diff --git a/src/debug.c b/src/debug.c
index 8776de38f..d4f3f5dd2 100644
--- a/src/debug.c
+++ b/src/debug.c
@@ -919,7 +919,7 @@ NULL
addReplyStatus(c,"Apparently Redis did not crash: test passed");
} else if (!strcasecmp(c->argv[1]->ptr,"set-disable-deny-scripts") && c->argc == 3)
{
- server.lua_disable_deny_script = atoi(c->argv[2]->ptr);;
+ server.script_disable_deny_script = atoi(c->argv[2]->ptr);;
addReply(c,shared.ok);
} else if (!strcasecmp(c->argv[1]->ptr,"config-rewrite-force-all") && c->argc == 2)
{
diff --git a/src/defrag.c b/src/defrag.c
index 734174c3a..ffd6eaba6 100644
--- a/src/defrag.c
+++ b/src/defrag.c
@@ -939,7 +939,7 @@ long defragOtherGlobals() {
/* there are many more pointers to defrag (e.g. client argv, output / aof buffers, etc.
* but we assume most of these are short lived, we only need to defrag allocations
* that remain static for a long time */
- defragged += activeDefragSdsDict(server.lua_scripts, DEFRAG_SDS_DICT_VAL_IS_STROB);
+ defragged += activeDefragSdsDict(evalScriptsDict(), DEFRAG_SDS_DICT_VAL_IS_STROB);
defragged += activeDefragSdsListAndDict(server.repl_scriptcache_fifo, server.repl_scriptcache_dict, DEFRAG_SDS_DICT_NO_VAL);
defragged += moduleDefragGlobals();
return defragged;
diff --git a/src/eval.c b/src/eval.c
new file mode 100644
index 000000000..977cad721
--- /dev/null
+++ b/src/eval.c
@@ -0,0 +1,1636 @@
+/*
+ * Copyright (c) 2009-2012, Salvatore Sanfilippo <antirez at gmail dot com>
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * * Redistributions of source code must retain the above copyright notice,
+ * this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Redis nor the names of its contributors may be used
+ * to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "server.h"
+#include "sha1.h"
+#include "rand.h"
+#include "cluster.h"
+#include "monotonic.h"
+#include "resp_parser.h"
+#include "script_lua.h"
+
+#include <lua.h>
+#include <lauxlib.h>
+#include <lualib.h>
+#include <ctype.h>
+#include <math.h>
+
+void ldbInit(void);
+void ldbDisable(client *c);
+void ldbEnable(client *c);
+void evalGenericCommandWithDebugging(client *c, int evalsha);
+sds ldbCatStackValue(sds s, lua_State *lua, int idx);
+
+/* Lua context */
+struct luaCtx {
+ lua_State *lua; /* The Lua interpreter. We use just one for all clients */
+ client *lua_client; /* The "fake client" to query Redis from Lua */
+ char *lua_cur_script; /* SHA1 of the script currently running, or NULL */
+ dict *lua_scripts; /* A dictionary of SHA1 -> Lua scripts */
+ unsigned long long lua_scripts_mem; /* Cached scripts' memory + oh */
+ int lua_replicate_commands; /* True if we are doing single commands repl. */
+} lctx;
+
+/* Debugger shared state is stored inside this global structure. */
+#define LDB_BREAKPOINTS_MAX 64 /* Max number of breakpoints. */
+#define LDB_MAX_LEN_DEFAULT 256 /* Default len limit for replies / var dumps. */
+struct ldbState {
+ connection *conn; /* Connection of the debugging client. */
+ int active; /* Are we debugging EVAL right now? */
+ int forked; /* Is this a fork()ed debugging session? */
+ list *logs; /* List of messages to send to the client. */
+ list *traces; /* Messages about Redis commands executed since last stop.*/
+ list *children; /* All forked debugging sessions pids. */
+ int bp[LDB_BREAKPOINTS_MAX]; /* An array of breakpoints line numbers. */
+ int bpcount; /* Number of valid entries inside bp. */
+ int step; /* Stop at next line regardless of breakpoints. */
+ int luabp; /* Stop at next line because redis.breakpoint() was called. */
+ sds *src; /* Lua script source code split by line. */
+ int lines; /* Number of lines in 'src'. */
+ int currentline; /* Current line number. */
+ sds cbuf; /* Debugger client command buffer. */
+ size_t maxlen; /* Max var dump / reply length. */
+ int maxlen_hint_sent; /* Did we already hint about "set maxlen"? */
+} ldb;
+
+/* ---------------------------------------------------------------------------
+ * Utility functions.
+ * ------------------------------------------------------------------------- */
+
+/* Perform the SHA1 of the input string. We use this both for hashing script
+ * bodies in order to obtain the Lua function name, and in the implementation
+ * of redis.sha1().
+ *
+ * 'digest' should point to a 41 bytes buffer: 40 for SHA1 converted into an
+ * hexadecimal number, plus 1 byte for null term. */
+void sha1hex(char *digest, char *script, size_t len) {
+ SHA1_CTX ctx;
+ unsigned char hash[20];
+ char *cset = "0123456789abcdef";
+ int j;
+
+ SHA1Init(&ctx);
+ SHA1Update(&ctx,(unsigned char*)script,len);
+ SHA1Final(hash,&ctx);
+
+ for (j = 0; j < 20; j++) {
+ digest[j*2] = cset[((hash[j]&0xF0)>>4)];
+ digest[j*2+1] = cset[(hash[j]&0xF)];
+ }
+ digest[40] = '\0';
+}
+
+/* redis.breakpoint()
+ *
+ * Allows to stop execution during a debugging session from within
+ * the Lua code implementation, like if a breakpoint was set in the code
+ * immediately after the function. */
+int luaRedisBreakpointCommand(lua_State *lua) {
+ if (ldb.active) {
+ ldb.luabp = 1;
+ lua_pushboolean(lua,1);
+ } else {
+ lua_pushboolean(lua,0);
+ }
+ return 1;
+}
+
+/* redis.debug()
+ *
+ * Log a string message into the output console.
+ * Can take multiple arguments that will be separated by commas.
+ * Nothing is returned to the caller. */
+int luaRedisDebugCommand(lua_State *lua) {
+ if (!ldb.active) return 0;
+ int argc = lua_gettop(lua);
+ sds log = sdscatprintf(sdsempty(),"<debug> line %d: ", ldb.currentline);
+ while(argc--) {
+ log = ldbCatStackValue(log,lua,-1 - argc);
+ if (argc != 0) log = sdscatlen(log,", ",2);
+ }
+ ldbLog(log);
+ return 0;
+}
+
+/* redis.replicate_commands()
+ *
+ * Turn on single commands replication if the script never called
+ * a write command so far, and returns true. Otherwise if the script
+ * already started to write, returns false and stick to whole scripts
+ * replication, which is our default. */
+int luaRedisReplicateCommandsCommand(lua_State *lua) {
+ scriptRunCtx* rctx = luaGetFromRegistry(lua, REGISTRY_RUN_CTX_NAME);
+ if (rctx->flags & SCRIPT_WRITE_DIRTY) {
+ lua_pushboolean(lua,0);
+ } else {
+ lctx.lua_replicate_commands = 1;
+ rctx->flags &= ~SCRIPT_EVAL_REPLICATION;
+ /* When we switch to single commands replication, we can provide
+ * different math.random() sequences at every call, which is what
+ * the user normally expects. */
+ redisSrand48(rand());
+ lua_pushboolean(lua,1);
+ }
+ return 1;
+}
+
+/* Initialize the scripting environment.
+ *
+ * This function is called the first time at server startup with
+ * the 'setup' argument set to 1.
+ *
+ * It can be called again multiple times during the lifetime of the Redis
+ * process, with 'setup' set to 0, and following a scriptingRelease() call,
+ * in order to reset the Lua scripting environment.
+ *
+ * However it is simpler to just call scriptingReset() that does just that. */
+void scriptingInit(int setup) {
+ lua_State *lua = lua_open();
+
+ if (setup) {
+ lctx.lua_client = NULL;
+ server.script_caller = NULL;
+ lctx.lua_cur_script = NULL;
+ server.script_disable_deny_script = 0;
+ ldbInit();
+ }
+
+ /* Initialize a dictionary we use to map SHAs to scripts.
+ * This is useful for replication, as we need to replicate EVALSHA
+ * as EVAL, so we need to remember the associated script. */
+ lctx.lua_scripts = dictCreate(&shaScriptObjectDictType);
+ lctx.lua_scripts_mem = 0;
+
+ luaRegisterRedisAPI(lua);
+
+ /* register debug commands */
+ lua_getglobal(lua,"redis");
+
+ /* redis.breakpoint */
+ lua_pushstring(lua,"breakpoint");
+ lua_pushcfunction(lua,luaRedisBreakpointCommand);
+ lua_settable(lua,-3);
+
+ /* redis.debug */
+ lua_pushstring(lua,"debug");
+ lua_pushcfunction(lua,luaRedisDebugCommand);
+ lua_settable(lua,-3);
+
+ /* redis.replicate_commands */
+ lua_pushstring(lua, "replicate_commands");
+ lua_pushcfunction(lua, luaRedisReplicateCommandsCommand);
+ lua_settable(lua, -3);
+
+ lua_setglobal(lua,"redis");
+
+ /* Add a helper function that we use to sort the multi bulk output of non
+ * deterministic commands, when containing 'false' elements. */
+ {
+ char *compare_func = "function __redis__compare_helper(a,b)\n"
+ " if a == false then a = '' end\n"
+ " if b == false then b = '' end\n"
+ " return a<b\n"
+ "end\n";
+ luaL_loadbuffer(lua,compare_func,strlen(compare_func),"@cmp_func_def");
+ lua_pcall(lua,0,0,0);
+ }
+
+ /* Add a helper function we use for pcall error reporting.
+ * Note that when the error is in the C function we want to report the
+ * information about the caller, that's what makes sense from the point
+ * of view of the user debugging a script. */
+ {
+ char *errh_func = "local dbg = debug\n"
+ "function __redis__err__handler(err)\n"
+ " local i = dbg.getinfo(2,'nSl')\n"
+ " if i and i.what == 'C' then\n"
+ " i = dbg.getinfo(3,'nSl')\n"
+ " end\n"
+ " if i then\n"
+ " return i.source .. ':' .. i.currentline .. ': ' .. err\n"
+ " else\n"
+ " return err\n"
+ " end\n"
+ "end\n";
+ luaL_loadbuffer(lua,errh_func,strlen(errh_func),"@err_handler_def");
+ lua_pcall(lua,0,0,0);
+ }
+
+ /* Create the (non connected) client that we use to execute Redis commands
+ * inside the Lua interpreter.
+ * Note: there is no need to create it again when this function is called
+ * by scriptingReset(). */
+ if (lctx.lua_client == NULL) {
+ lctx.lua_client = createClient(NULL);
+ lctx.lua_client->flags |= CLIENT_SCRIPT;
+
+ /* We do not want to allow blocking commands inside Lua */
+ lctx.lua_client->flags |= CLIENT_DENY_BLOCKING;
+ }
+
+ /* Lua beginners often don't use "local", this is likely to introduce
+ * subtle bugs in their code. To prevent problems we protect accesses
+ * to global variables. */
+ luaEnableGlobalsProtection(lua, 1);
+
+ lctx.lua = lua;
+}
+
+/* Release resources related to Lua scripting.
+ * This function is used in order to reset the scripting environment. */
+void scriptingRelease(int async) {
+ if (async)
+ freeLuaScriptsAsync(lctx.lua_scripts);
+ else
+ dictRelease(lctx.lua_scripts);
+ lctx.lua_scripts_mem = 0;
+ lua_close(lctx.lua);
+}
+
+void scriptingReset(int async) {
+ scriptingRelease(async);
+ scriptingInit(0);
+}
+
+/* ---------------------------------------------------------------------------
+ * EVAL and SCRIPT commands implementation
+ * ------------------------------------------------------------------------- */
+
+/* Define a Lua function with the specified body.
+ * The function name will be generated in the following form:
+ *
+ * f_<hex sha1 sum>
+ *
+ * The function increments the reference count of the 'body' object as a
+ * side effect of a successful call.
+ *
+ * On success a pointer to an SDS string representing the function SHA1 of the
+ * just added function is returned (and will be valid until the next call
+ * to scriptingReset() function), otherwise NULL is returned.
+ *
+ * The function handles the fact of being called with a script that already
+ * exists, and in such a case, it behaves like in the success case.
+ *
+ * If 'c' is not NULL, on error the client is informed with an appropriate
+ * error describing the nature of the problem and the Lua interpreter error. */
+sds luaCreateFunction(client *c, robj *body) {
+ char funcname[43];
+ dictEntry *de;
+
+ funcname[0] = 'f';
+ funcname[1] = '_';
+ sha1hex(funcname+2,body->ptr,sdslen(body->ptr));
+
+ sds sha = sdsnewlen(funcname+2,40);
+ if ((de = dictFind(lctx.lua_scripts,sha)) != NULL) {
+ sdsfree(sha);
+ return dictGetKey(de);
+ }
+
+ sds funcdef = sdsempty();
+ funcdef = sdscat(funcdef,"function ");
+ funcdef = sdscatlen(funcdef,funcname,42);
+ funcdef = sdscatlen(funcdef,"() ",3);
+ funcdef = sdscatlen(funcdef,body->ptr,sdslen(body->ptr));
+ funcdef = sdscatlen(funcdef,"\nend",4);
+
+ if (luaL_loadbuffer(lctx.lua,funcdef,sdslen(funcdef),"@user_script")) {
+ if (c != NULL) {
+ addReplyErrorFormat(c,
+ "Error compiling script (new function): %s\n",
+ lua_tostring(lctx.lua,-1));
+ }
+ lua_pop(lctx.lua,1);
+ sdsfree(sha);
+ sdsfree(funcdef);
+ return NULL;
+ }
+ sdsfree(funcdef);
+
+ if (lua_pcall(lctx.lua,0,0,0)) {
+ if (c != NULL) {
+ addReplyErrorFormat(c,"Error running script (new function): %s\n",
+ lua_tostring(lctx.lua,-1));
+ }
+ lua_pop(lctx.lua,1);
+ sdsfree(sha);
+ return NULL;
+ }
+
+ /* We also save a SHA1 -> Original script map in a dictionary
+ * so that we can replicate / write in the AOF all the
+ * EVALSHA commands as EVAL using the original script. */
+ int retval = dictAdd(lctx.lua_scripts,sha,body);
+ serverAssertWithInfo(c ? c : lctx.lua_client,NULL,retval == DICT_OK);
+ lctx.lua_scripts_mem += sdsZmallocSize(sha) + getStringObjectSdsUsedMemory(body);
+ incrRefCount(body);
+ return sha;
+}
+
+void prepareLuaClient(void) {
+ /* Select the right DB in the context of the Lua client */
+ selectDb(lctx.lua_client,server.script_caller->db->id);
+ lctx.lua_client->resp = 2; /* Default is RESP2, scripts can change it. */
+
+ /* If we are in MULTI context, flag Lua client as CLIENT_MULTI. */
+ if (server.script_caller->flags & CLIENT_MULTI) {
+ lctx.lua_client->flags |= CLIENT_MULTI;
+ }
+}
+
+void resetLuaClient(void) {
+ /* After the script done, remove the MULTI state. */
+ lctx.lua_client->flags &= ~CLIENT_MULTI;
+}
+
+void evalGenericCommand(client *c, int evalsha) {
+ lua_State *lua = lctx.lua;
+ char funcname[43];
+ long long numkeys;
+ long long initial_server_dirty = server.dirty;
+
+ /* When we replicate whole scripts, we want the same PRNG sequence at
+ * every call so that our PRNG is not affected by external state. */
+ redisSrand48(0);
+
+ lctx.lua_replicate_commands = server.lua_always_replicate_commands;
+
+ /* Get the number of arguments that are keys */
+ if (getLongLongFromObjectOrReply(c,c->argv[2],&numkeys,NULL) != C_OK)
+ return;
+ if (numkeys > (c->argc - 3)) {
+ addReplyError(c,"Number of keys can't be greater than number of args");
+ return;
+ } else if (numkeys < 0) {
+ addReplyError(c,"Number of keys can't be negative");
+ return;
+ }
+
+ /* We obtain the script SHA1, then check if this function is already
+ * defined into the Lua state */
+ funcname[0] = 'f';
+ funcname[1] = '_';
+ if (!evalsha) {
+ /* Hash the code if this is an EVAL call */
+ sha1hex(funcname+2,c->argv[1]->ptr,sdslen(c->argv[1]->ptr));
+ } else {
+ /* We already have the SHA if it is an EVALSHA */
+ int j;
+ char *sha = c->argv[1]->ptr;
+
+ /* Convert to lowercase. We don't use tolower since the function
+ * managed to always show up in the profiler output consuming
+ * a non trivial amount of time. */
+ for (j = 0; j < 40; j++)
+ funcname[j+2] = (sha[j] >= 'A' && sha[j] <= 'Z') ?
+ sha[j]+('a'-'A') : sha[j];
+ funcname[42] = '\0';
+ }
+
+ /* Push the pcall error handler function on the stack. */
+ lua_getglobal(lua, "__redis__err__handler");
+
+ /* Try to lookup the Lua function */
+ lua_getglobal(lua, funcname);
+ if (lua_isnil(lua,-1)) {
+ lua_pop(lua,1); /* remove the nil from the stack */
+ /* Function not defined... let's define it if we have the
+ * body of the function. If this is an EVALSHA call we can just
+ * return an error. */
+ if (evalsha) {
+ lua_pop(lua,1); /* remove the error handler from the stack. */
+ addReplyErrorObject(c, shared.noscripterr);
+ return;
+ }
+ if (luaCreateFunction(c,c->argv[1]) == NULL) {
+ lua_pop(lua,1); /* remove the error handler from the stack. */
+ /* The error is sent to the client by luaCreateFunction()
+ * itself when it returns NULL. */
+ return;
+ }
+ /* Now the following is guaranteed to return non nil */
+ lua_getglobal(lua, funcname);
+ serverAssert(!lua_isnil(lua,-1));
+ }
+
+ lctx.lua_cur_script = funcname + 2;
+
+ scriptRunCtx rctx;
+ scriptPrepareForRun(&rctx, lctx.lua_client, c, lctx.lua_cur_script);
+ rctx.flags |= SCRIPT_EVAL_MODE; /* mark the current run as legacy so we
+ will get legacy error messages and logs */
+ if (!lctx.lua_replicate_commands) rctx.flags |= SCRIPT_EVAL_REPLICATION;
+ /* This check is for EVAL_RO, EVALSHA_RO. We want to allow only read only commands */
+ if ((server.script_caller->cmd->proc == evalRoCommand ||
+ server.script_caller->cmd->proc == evalShaRoCommand)) {
+ rctx.flags |= SCRIPT_READ_ONLY;
+ }
+
+ luaCallFunction(&rctx, lua, c->argv+3, numkeys, c->argv+3+numkeys, c->argc-3-numkeys, ldb.active);
+ lua_pop(lua,1); /* Remove the error handler. */
+ scriptResetRun(&rctx);
+
+ lctx.lua_cur_script = NULL;
+
+ /* EVALSHA should be propagated to Slave and AOF file as full EVAL, unless
+ * we are sure that the script was already in the context of all the
+ * attached slaves *and* the current AOF file if enabled.
+ *
+ * To do so we use a cache of SHA1s of scripts that we already propagated
+ * as full EVAL, that's called the Replication Script Cache.
+ *
+ * For replication, every time a new slave attaches to the master, we need to
+ * flush our cache of scripts that can be replicated as EVALSHA, while
+ * for AOF we need to do so every time we rewrite the AOF file. */
+ if (evalsha && !lctx.lua_replicate_commands) {
+ if (!replicationScriptCacheExists(c->argv[1]->ptr)) {
+ /* This script is not in our script cache, replicate it as
+ * EVAL, then add it into the script cache, as from now on
+ * slaves and AOF know about it. */
+ robj *script = dictFetchValue(lctx.lua_scripts,c->argv[1]->ptr);
+
+ replicationScriptCacheAdd(c->argv[1]->ptr);
+ serverAssertWithInfo(c,NULL,script != NULL);
+
+ /* If the script did not produce any changes in the dataset we want
+ * just to replicate it as SCRIPT LOAD, otherwise we risk running
+ * an aborted script on slaves (that may then produce results there)
+ * or just running a CPU costly read-only script on the slaves. */
+ if (server.dirty == initial_server_dirty) {
+ rewriteClientCommandVector(c,3,
+ shared.script,
+ shared.load,
+ script);
+ } else {
+ rewriteClientCommandArgument(c,0,shared.eval);
+ rewriteClientCommandArgument(c,1,script);
+ }
+ forceCommandPropagation(c,PROPAGATE_REPL|PROPAGATE_AOF);
+ }
+ }
+}
+
+void evalCommand(client *c) {
+ /* Explicitly feed monitor here so that lua commands appear after their
+ * script command. */
+ replicationFeedMonitors(c,server.monitors,c->db->id,c->argv,c->argc);
+ if (!(c->flags & CLIENT_LUA_DEBUG))
+ evalGenericCommand(c,0);
+ else
+ evalGenericCommandWithDebugging(c,0);
+}
+
+void evalRoCommand(client *c) {
+ evalCommand(c);
+}
+
+void evalShaCommand(client *c) {
+ /* Explicitly feed monitor here so that lua commands appear after their
+ * script command. */
+ replicationFeedMonitors(c,server.monitors,c->db->id,c->argv,c->argc);
+ if (sdslen(c->argv[1]->ptr) != 40) {
+ /* We know that a match is not possible if the provided SHA is
+ * not the right length. So we return an error ASAP, this way
+ * evalGenericCommand() can be implemented without string length
+ * sanity check */
+ addReplyErrorObject(c, shared.noscripterr);
+ return;
+ }
+ if (!(c->flags & CLIENT_LUA_DEBUG))
+ evalGenericCommand(c,1);
+ else {
+ addReplyError(c,"Please use EVAL instead of EVALSHA for debugging");
+ return;
+ }
+}
+
+void evalShaRoCommand(client *c) {
+ evalShaCommand(c);
+}
+
+void scriptCommand(client *c) {
+ if (c->argc == 2 && !strcasecmp(c->argv[1]->ptr,"help")) {
+ const char *help[] = {
+"DEBUG (YES|SYNC|NO)",
+" Set the debug mode for subsequent scripts executed.",
+"EXISTS <sha1> [<sha1> ...]",
+" Return information about the existence of the scripts in the script cache.",
+"FLUSH [ASYNC|SYNC]",
+" Flush the Lua scripts cache. Very dangerous on replicas.",
+" When called without the optional mode argument, the behavior is determined by the",
+" lazyfree-lazy-user-flush configuration directive. Valid modes are:",
+" * ASYNC: Asynchronously flush the scripts cache.",
+" * SYNC: Synchronously flush the scripts cache.",
+"KILL",
+" Kill the currently executing Lua script.",
+"LOAD <script>",
+" Load a script into the scripts cache without executing it.",
+NULL
+ };
+ addReplyHelp(c, help);
+ } else if (c->argc >= 2 && !strcasecmp(c->argv[1]->ptr,"flush")) {
+ int async = 0;
+ if (c->argc == 3 && !strcasecmp(c->argv[2]->ptr,"sync")) {
+ async = 0;
+ } else if (c->argc == 3 && !strcasecmp(c->argv[2]->ptr,"async")) {
+ async = 1;
+ } else if (c->argc == 2) {
+ async = server.lazyfree_lazy_user_flush ? 1 : 0;
+ } else {
+ addReplyError(c,"SCRIPT FLUSH only support SYNC|ASYNC option");
+ return;
+ }
+ scriptingReset(async);
+ addReply(c,shared.ok);
+ replicationScriptCacheFlush();
+ server.dirty++; /* Propagating this command is a good idea. */
+ } else if (c->argc >= 2 && !strcasecmp(c->argv[1]->ptr,"exists")) {
+ int j;
+
+ addReplyArrayLen(c, c->argc-2);
+ for (j = 2; j < c->argc; j++) {
+ if (dictFind(lctx.lua_scripts,c->argv[j]->ptr))
+ addReply(c,shared.cone);
+ else
+ addReply(c,shared.czero);
+ }
+ } else if (c->argc == 3 && !strcasecmp(c->argv[1]->ptr,"load")) {
+ sds sha = luaCreateFunction(c,c->argv[2]);
+ if (sha == NULL) return; /* The error was sent by luaCreateFunction(). */
+ addReplyBulkCBuffer(c,sha,40);
+ forceCommandPropagation(c,PROPAGATE_REPL|PROPAGATE_AOF);
+ } else if (c->argc == 2 && !strcasecmp(c->argv[1]->ptr,"kill")) {
+ scriptKill(c, 1);
+ } else if (c->argc == 3 && !strcasecmp(c->argv[1]->ptr,"debug")) {
+ if (clientHasPendingReplies(c)) {
+ addReplyError(c,"SCRIPT DEBUG must be called outside a pipeline");
+ return;
+ }
+ if (!strcasecmp(c->argv[2]->ptr,"no")) {
+ ldbDisable(c);
+ addReply(c,shared.ok);
+ } else if (!strcasecmp(c->argv[2]->ptr,"yes")) {
+ ldbEnable(c);
+ addReply(c,shared.ok);
+ } else if (!strcasecmp(c->argv[2]->ptr,"sync")) {
+ ldbEnable(c);
+ addReply(c,shared.ok);
+ c->flags |= CLIENT_LUA_DEBUG_SYNC;
+ } else {
+ addReplyError(c,"Use SCRIPT DEBUG YES/SYNC/NO");
+ return;
+ }
+ } else {
+ addReplySubcommandSyntaxError(c);
+ }
+}
+
+unsigned long evalMemory() {
+ return luaMemory(lctx.lua);
+}
+
+dict* evalScriptsDict() {
+ return lctx.lua_scripts;
+}
+
+unsigned long evalScriptsMemory() {
+ return lctx.lua_scripts_mem +
+ dictSize(lctx.lua_scripts) * sizeof(dictEntry) +
+ dictSlots(lctx.lua_scripts) * sizeof(dictEntry*);
+}
+
+/* Returns the time when the script invocation started */
+mstime_t evalTimeSnapshot() {
+ return scriptTimeSnapshot();
+}
+
+
+/* ---------------------------------------------------------------------------
+ * LDB: Redis Lua debugging facilities
+ * ------------------------------------------------------------------------- */
+
+/* Initialize Lua debugger data structures. */
+void ldbInit(void) {
+ ldb.conn = NULL;
+ ldb.active = 0;
+ ldb.logs = listCreate();
+ listSetFreeMethod(ldb.logs,(void (*)(void*))sdsfree);
+ ldb.children = listCreate();
+ ldb.src = NULL;
+ ldb.lines = 0;
+ ldb.cbuf = sdsempty();
+}
+
+/* Remove all the pending messages in the specified list. */
+void ldbFlushLog(list *log) {
+ listNode *ln;
+
+ while((ln = listFirst(log)) != NULL)
+ listDelNode(log,ln);
+}
+
+int ldbIsEnabled(){
+ return ldb.active && ldb.step;
+}
+
+/* Enable debug mode of Lua scripts for this client. */
+void ldbEnable(client *c) {
+ c->flags |= CLIENT_LUA_DEBUG;
+ ldbFlushLog(ldb.logs);
+ ldb.conn = c->conn;
+ ldb.step = 1;
+ ldb.bpcount = 0;
+ ldb.luabp = 0;
+ sdsfree(ldb.cbuf);
+ ldb.cbuf = sdsempty();
+ ldb.maxlen = LDB_MAX_LEN_DEFAULT;
+ ldb.maxlen_hint_sent = 0;
+}
+
+/* Exit debugging mode from the POV of client. This function is not enough
+ * to properly shut down a client debugging session, see ldbEndSession()
+ * for more information. */
+void ldbDisable(client *c) {
+ c->flags &= ~(CLIENT_LUA_DEBUG|CLIENT_LUA_DEBUG_SYNC);
+}
+
+/* Append a log entry to the specified LDB log. */
+void ldbLog(sds entry) {
+ listAddNodeTail(ldb.logs,entry);
+}
+
+/* A version of ldbLog() which prevents producing logs greater than
+ * ldb.maxlen. The first time the limit is reached a hint is generated
+ * to inform the user that reply trimming can be disabled using the
+ * debugger "maxlen" command. */
+void ldbLogWithMaxLen(sds entry) {
+ int trimmed = 0;
+ if (ldb.maxlen && sdslen(entry) > ldb.maxlen) {
+ sdsrange(entry,0,ldb.maxlen-1);
+ entry = sdscatlen(entry," ...",4);
+ trimmed = 1;
+ }
+ ldbLog(entry);
+ if (trimmed && ldb.maxlen_hint_sent == 0) {
+ ldb.maxlen_hint_sent = 1;
+ ldbLog(sdsnew(
+ "<hint> The above reply was trimmed. Use 'maxlen 0' to disable trimming."));
+ }
+}
+
+/* Send ldb.logs to the debugging client as a multi-bulk reply
+ * consisting of simple strings. Log entries which include newlines have them
+ * replaced with spaces. The entries sent are also consumed. */
+void ldbSendLogs(void) {
+ sds proto = sdsempty();
+ proto = sdscatfmt(proto,"*%i\r\n", (int)listLength(ldb.logs));
+ while(listLength(ldb.logs)) {
+ listNode *ln = listFirst(ldb.logs);
+ proto = sdscatlen(proto,"+",1);
+ sdsmapchars(ln->value,"\r\n"," ",2);
+ proto = sdscatsds(proto,ln->value);
+ proto = sdscatlen(proto,"\r\n",2);
+ listDelNode(ldb.logs,ln);
+ }
+ if (connWrite(ldb.conn,proto,sdslen(proto)) == -1) {
+ /* Avoid warning. We don't check the return value of write()
+ * since the next read() will catch the I/O error and will
+ * close the debugging session. */
+ }
+ sdsfree(proto);
+}
+
+/* Start a debugging session before calling EVAL implementation.
+ * The technique we use is to capture the client socket file descriptor,
+ * in order to perform direct I/O with it from within Lua hooks. This
+ * way we don't have to re-enter Redis in order to handle I/O.
+ *
+ * The function returns 1 if the caller should proceed to call EVAL,
+ * and 0 if instead the caller should abort the operation (this happens
+ * for the parent in a forked session, since it's up to the children
+ * to continue, or when fork returned an error).
+ *
+ * The caller should call ldbEndSession() only if ldbStartSession()
+ * returned 1. */
+int ldbStartSession(client *c) {
+ ldb.forked = (c->flags & CLIENT_LUA_DEBUG_SYNC) == 0;
+ if (ldb.forked) {
+ pid_t cp = redisFork(CHILD_TYPE_LDB);
+ if (cp == -1) {
+ addReplyErrorFormat(c,"Fork() failed: can't run EVAL in debugging mode: %s", strerror(errno));
+ return 0;
+ } else if (cp == 0) {
+ /* Child. Let's ignore important signals handled by the parent. */
+ struct sigaction act;
+ sigemptyset(&act.sa_mask);
+ act.sa_flags = 0;
+ act.sa_handler = SIG_IGN;
+ sigaction(SIGTERM, &act, NULL);
+ sigaction(SIGINT, &act, NULL);
+
+ /* Log the creation of the child and close the listening
+ * socket to make sure if the parent crashes a reset is sent
+ * to the clients. */
+ serverLog(LL_WARNING,"Redis forked for debugging eval");
+ } else {
+ /* Parent */
+ listAddNodeTail(ldb.children,(void*)(unsigned long)cp);
+ freeClientAsync(c); /* Close the client in the parent side. */
+ return 0;
+ }
+ } else {
+ serverLog(LL_WARNING,
+ "Redis synchronous debugging eval session started");
+ }
+
+ /* Setup our debugging session. */
+ connBlock(ldb.conn);
+ connSendTimeout(ldb.conn,5000);
+ ldb.active = 1;
+
+ /* First argument of EVAL is the script itself. We split it into different
+ * lines since this is the way the debugger accesses the source code. */
+ sds srcstring = sdsdup(c->argv[1]->ptr);
+ size_t srclen = sdslen(srcstring);
+ while(srclen && (srcstring[srclen-1] == '\n' ||
+ srcstring[srclen-1] == '\r'))
+ {
+ srcstring[--srclen] = '\0';
+ }
+ sdssetlen(srcstring,srclen);
+ ldb.src = sdssplitlen(srcstring,sdslen(srcstring),"\n",1,&ldb.lines);
+ sdsfree(srcstring);
+ return 1;
+}
+
+/* End a debugging session after the EVAL call with debugging enabled
+ * returned. */
+void ldbEndSession(client *c) {
+ /* Emit the remaining logs and an <endsession> mark. */
+ ldbLog(sdsnew("<endsession>"));
+ ldbSendLogs();
+
+ /* If it's a fork()ed session, we just exit. */
+ if (ldb.forked) {
+ writeToClient(c,0);
+ serverLog(LL_WARNING,"Lua debugging session child exiting");
+ exitFromChild(0);
+ } else {
+ serverLog(LL_WARNING,
+ "Redis synchronous debugging eval session ended");
+ }
+
+ /* Otherwise let's restore client's state. */
+ connNonBlock(ldb.conn);
+ connSendTimeout(ldb.conn,0);
+
+ /* Close the client connection after sending the final EVAL reply
+ * in order to signal the end of the debugging session. */
+ c->flags |= CLIENT_CLOSE_AFTER_REPLY;
+
+ /* Cleanup. */
+ sdsfreesplitres(ldb.src,ldb.lines);
+ ldb.lines = 0;
+ ldb.active = 0;
+}
+
+/* If the specified pid is among the list of children spawned for
+ * forked debugging sessions, it is removed from the children list.
+ * If the pid was found non-zero is returned. */
+int ldbRemoveChild(pid_t pid) {
+ listNode *ln = listSearchKey(ldb.children,(void*)(unsigned long)pid);
+ if (ln) {
+ listDelNode(ldb.children,ln);
+ return 1;
+ }
+ return 0;
+}
+
+/* Return the number of children we still did not receive termination
+ * acknowledge via wait() in the parent process. */
+int ldbPendingChildren(void) {
+ return listLength(ldb.children);
+}
+
+/* Kill all the forked sessions. */
+void ldbKillForkedSessions(void) {
+ listIter li;
+ listNode *ln;
+
+ listRewind(ldb.children,&li);
+ while((ln = listNext(&li))) {
+ pid_t pid = (unsigned long) ln->value;
+ serverLog(LL_WARNING,"Killing debugging session %ld",(long)pid);
+ kill(pid,SIGKILL);
+ }
+ listRelease(ldb.children);
+ ldb.children = listCreate();
+}
+
+/* Wrapper for EVAL / EVALSHA that enables debugging, and makes sure
+ * that when EVAL returns, whatever happened, the session is ended. */
+void evalGenericCommandWithDebugging(client *c, int evalsha) {
+ if (ldbStartSession(c)) {
+ evalGenericCommand(c,evalsha);
+ ldbEndSession(c);
+ } else {
+ ldbDisable(c);
+ }
+}
+
+/* Return a pointer to ldb.src source code line, considering line to be
+ * one-based, and returning a special string for out of range lines. */
+char *ldbGetSourceLine(int line) {
+ int idx = line-1;
+ if (idx < 0 || idx >= ldb.lines) return "<out of range source code line>";
+ return ldb.src[idx];
+}
+
+/* Return true if there is a breakpoint in the specified line. */
+int ldbIsBreakpoint(int line) {
+ int j;
+
+ for (j = 0; j < ldb.bpcount; j++)
+ if (ldb.bp[j] == line) return 1;
+ return 0;
+}
+
+/* Add the specified breakpoint. Ignore it if we already reached the max.
+ * Returns 1 if the breakpoint was added (or was already set). 0 if there is
+ * no space for the breakpoint or if the line is invalid. */
+int ldbAddBreakpoint(int line) {
+ if (line <= 0 || line > ldb.lines) return 0;
+ if (!ldbIsBreakpoint(line) && ldb.bpcount != LDB_BREAKPOINTS_MAX) {
+ ldb.bp[ldb.bpcount++] = line;
+ return 1;
+ }
+ return 0;
+}
+
+/* Remove the specified breakpoint, returning 1 if the operation was
+ * performed or 0 if there was no such breakpoint. */
+int ldbDelBreakpoint(int line) {
+ int j;
+
+ for (j = 0; j < ldb.bpcount; j++) {
+ if (ldb.bp[j] == line) {
+ ldb.bpcount--;
+ memmove(ldb.bp+j,ldb.bp+j+1,ldb.bpcount-j);
+ return 1;
+ }
+ }
+ return 0;
+}
+
+/* Expect a valid multi-bulk command in the debugging client query buffer.
+ * On success the command is parsed and returned as an array of SDS strings,
+ * otherwise NULL is returned and there is to read more buffer. */
+sds *ldbReplParseCommand(int *argcp, char** err) {
+ static char* protocol_error = "protocol error";
+ sds *argv = NULL;
+ int argc = 0;
+ if (sdslen(ldb.cbuf) == 0) return NULL;
+
+ /* Working on a copy is simpler in this case. We can modify it freely
+ * for the sake of simpler parsing. */
+ sds copy = sdsdup(ldb.cbuf);
+ char *p = copy;
+
+ /* This Redis protocol parser is a joke... just the simplest thing that
+ * works in this context. It is also very forgiving regarding broken
+ * protocol. */
+
+ /* Seek and parse *<count>\r\n. */
+ p = strchr(p,'*'); if (!p) goto protoerr;
+ char *plen = p+1; /* Multi bulk len pointer. */
+ p = strstr(p,"\r\n"); if (!p) goto keep_reading;
+ *p = '\0'; p += 2;
+ *argcp = atoi(plen);
+ if (*argcp <= 0 || *argcp > 1024) goto protoerr;
+
+ /* Parse each argument. */
+ argv = zmalloc(sizeof(sds)*(*argcp));
+ argc = 0;
+ while(argc < *argcp) {
+ /* reached the end but there should be more data to read */
+ if (*p == '\0') goto keep_reading;
+
+ if (*p != '$') goto protoerr;
+ plen = p+1; /* Bulk string len pointer. */
+ p = strstr(p,"\r\n"); if (!p) goto keep_reading;
+ *p = '\0'; p += 2;
+ int slen = atoi(plen); /* Length of this arg. */
+ if (slen <= 0 || slen > 1024) goto protoerr;
+ if ((size_t)(p + slen + 2 - copy) > sdslen(copy) ) goto keep_reading;
+ argv[argc++] = sdsnewlen(p,slen);
+ p += slen; /* Skip the already parsed argument. */
+ if (p[0] != '\r' || p[1] != '\n') goto protoerr;
+ p += 2; /* Skip \r\n. */
+ }
+ sdsfree(copy);
+ return argv;
+
+protoerr:
+ *err = protocol_error;
+keep_reading:
+ sdsfreesplitres(argv,argc);
+ sdsfree(copy);
+ return NULL;
+}
+
+/* Log the specified line in the Lua debugger output. */
+void ldbLogSourceLine(int lnum) {
+ char *line = ldbGetSourceLine(lnum);
+ char *prefix;
+ int bp = ldbIsBreakpoint(lnum);
+ int current = ldb.currentline == lnum;
+
+ if (current && bp)
+ prefix = "->#";
+ else if (current)
+ prefix = "-> ";
+ else if (bp)
+ prefix = " #";
+ else
+ prefix = " ";
+ sds thisline = sdscatprintf(sdsempty(),"%s%-3d %s", prefix, lnum, line);
+ ldbLog(thisline);
+}
+
+/* Implement the "list" command of the Lua debugger. If around is 0
+ * the whole file is listed, otherwise only a small portion of the file
+ * around the specified line is shown. When a line number is specified
+ * the amount of context (lines before/after) is specified via the
+ * 'context' argument. */
+void ldbList(int around, int context) {
+ int j;
+
+ for (j = 1; j <= ldb.lines; j++) {
+ if (around != 0 && abs(around-j) > context) continue;
+ ldbLogSourceLine(j);
+ }
+}
+
+/* Append a human readable representation of the Lua value at position 'idx'
+ * on the stack of the 'lua' state, to the SDS string passed as argument.
+ * The new SDS string with the represented value attached is returned.
+ * Used in order to implement ldbLogStackValue().
+ *
+ * The element is not automatically removed from the stack, nor it is
+ * converted to a different type. */
+#define LDB_MAX_VALUES_DEPTH (LUA_MINSTACK/2)
+sds ldbCatStackValueRec(sds s, lua_State *lua, int idx, int level) {
+ int t = lua_type(lua,idx);
+
+ if (level++ == LDB_MAX_VALUES_DEPTH)
+ return sdscat(s,"<max recursion level reached! Nested table?>");
+
+ switch(t) {
+ case LUA_TSTRING:
+ {
+ size_t strl;
+ char *strp = (char*)lua_tolstring(lua,idx,&strl);
+ s = sdscatrepr(s,strp,strl);
+ }
+ break;
+ case LUA_TBOOLEAN:
+ s = sdscat(s,lua_toboolean(lua,idx) ? "true" : "false");
+ break;
+ case LUA_TNUMBER:
+ s = sdscatprintf(s,"%g",(double)lua_tonumber(lua,idx));
+ break;
+ case LUA_TNIL:
+ s = sdscatlen(s,"nil",3);
+ break;
+ case LUA_TTABLE:
+ {
+ int expected_index = 1; /* First index we expect in an array. */
+ int is_array = 1; /* Will be set to null if check fails. */
+ /* Note: we create two representations at the same time, one
+ * assuming the table is an array, one assuming it is not. At the
+ * end we know what is true and select the right one. */
+ sds repr1 = sdsempty();
+ sds repr2 = sdsempty();
+ lua_pushnil(lua); /* The first key to start the iteration is nil. */
+ while (lua_next(lua,idx-1)) {
+ /* Test if so far the table looks like an array. */
+ if (is_array &&
+ (lua_type(lua,-2) != LUA_TNUMBER ||
+ lua_tonumber(lua,-2) != expected_index)) is_array = 0;
+ /* Stack now: table, key, value */
+ /* Array repr. */
+ repr1 = ldbCatStackValueRec(repr1,lua,-1,level);
+ repr1 = sdscatlen(repr1,"; ",2);
+ /* Full repr. */
+ repr2 = sdscatlen(repr2,"[",1);
+ repr2 = ldbCatStackValueRec(repr2,lua,-2,level);
+ repr2 = sdscatlen(repr2,"]=",2);
+ repr2 = ldbCatStackValueRec(repr2,lua,-1,level);
+ repr2 = sdscatlen(repr2,"; ",2);
+ lua_pop(lua,1); /* Stack: table, key. Ready for next iteration. */
+ expected_index++;
+ }
+ /* Strip the last " ;" from both the representations. */
+ if (sdslen(repr1)) sdsrange(repr1,0,-3);
+ if (sdslen(repr2)) sdsrange(repr2,0,-3);
+ /* Select the right one and discard the other. */
+ s = sdscatlen(s,"{",1);
+ s = sdscatsds(s,is_array ? repr1 : repr2);
+ s = sdscatlen(s,"}",1);
+ sdsfree(repr1);
+ sdsfree(repr2);
+ }
+ break;
+ case LUA_TFUNCTION:
+ case LUA_TUSERDATA:
+ case LUA_TTHREAD:
+ case LUA_TLIGHTUSERDATA:
+ {
+ const void *p = lua_topointer(lua,idx);
+ char *typename = "unknown";
+ if (t == LUA_TFUNCTION) typename = "function";
+ else if (t == LUA_TUSERDATA) typename = "userdata";
+ else if (t == LUA_TTHREAD) typename = "thread";
+ else if (t == LUA_TLIGHTUSERDATA) typename = "light-userdata";
+ s = sdscatprintf(s,"\"%s@%p\"",typename,p);
+ }
+ break;
+ default:
+ s = sdscat(s,"\"<unknown-lua-type>\"");
+ break;
+ }
+ return s;
+}
+
+/* Higher level wrapper for ldbCatStackValueRec() that just uses an initial
+ * recursion level of '0'. */
+sds ldbCatStackValue(sds s, lua_State *lua, int idx) {
+ return ldbCatStackValueRec(s,lua,idx,0);
+}
+
+/* Produce a debugger log entry representing the value of the Lua object
+ * currently on the top of the stack. The element is not popped nor modified.
+ * Check ldbCatStackValue() for the actual implementation. */
+void ldbLogStackValue(lua_State *lua, char *prefix) {
+ sds s = sdsnew(prefix);
+ s = ldbCatStackValue(s,lua,-1);
+ ldbLogWithMaxLen(s);
+}
+
+char *ldbRedisProtocolToHuman_Int(sds *o, char *reply);
+char *ldbRedisProtocolToHuman_Bulk(sds *o, char *reply);
+char *ldbRedisProtocolToHuman_Status(sds *o, char *reply);
+char *ldbRedisProtocolToHuman_MultiBulk(sds *o, char *reply);
+char *ldbRedisProtocolToHuman_Set(sds *o, char *reply);
+char *ldbRedisProtocolToHuman_Map(sds *o, char *reply);
+char *ldbRedisProtocolToHuman_Null(sds *o, char *reply);
+char *ldbRedisProtocolToHuman_Bool(sds *o, char *reply);
+char *ldbRedisProtocolToHuman_Double(sds *o, char *reply);
+
+/* Get Redis protocol from 'reply' and appends it in human readable form to
+ * the passed SDS string 'o'.
+ *
+ * Note that the SDS string is passed by reference (pointer of pointer to
+ * char*) so that we can return a modified pointer, as for SDS semantics. */
+char *ldbRedisProtocolToHuman(sds *o, char *reply) {
+ char *p = reply;
+ switch(*p) {
+ case ':': p = ldbRedisProtocolToHuman_Int(o,reply); break;
+ case '$': p = ldbRedisProtocolToHuman_Bulk(o,reply); break;
+ case '+': p = ldbRedisProtocolToHuman_Status(o,reply); break;
+ case '-': p = ldbRedisProtocolToHuman_Status(o,reply); break;
+ case '*': p = ldbRedisProtocolToHuman_MultiBulk(o,reply); break;
+ case '~': p = ldbRedisProtocolToHuman_Set(o,reply); break;
+ case '%': p = ldbRedisProtocolToHuman_Map(o,reply); break;
+ case '_': p = ldbRedisProtocolToHuman_Null(o,reply); break;
+ case '#': p = ldbRedisProtocolToHuman_Bool(o,reply); break;
+ case ',': p = ldbRedisProtocolToHuman_Double(o,reply); break;
+ }
+ return p;
+}
+
+/* The following functions are helpers for ldbRedisProtocolToHuman(), each
+ * take care of a given Redis return type. */
+
+char *ldbRedisProtocolToHuman_Int(sds *o, char *reply) {
+ char *p = strchr(reply+1,'\r');
+ *o = sdscatlen(*o,reply+1,p-reply-1);
+ return p+2;
+}
+
+char *ldbRedisProtocolToHuman_Bulk(sds *o, char *reply) {
+ char *p = strchr(reply+1,'\r');
+ long long bulklen;
+
+ string2ll(reply+1,p-reply-1,&bulklen);
+ if (bulklen == -1) {
+ *o = sdscatlen(*o,"NULL",4);
+ return p+2;
+ } else {
+ *o = sdscatrepr(*o,p+2,bulklen);
+ return p+2+bulklen+2;
+ }
+}
+
+char *ldbRedisProtocolToHuman_Status(sds *o, char *reply) {
+ char *p = strchr(reply+1,'\r');
+
+ *o = sdscatrepr(*o,reply,p-reply);
+ return p+2;
+}
+
+char *ldbRedisProtocolToHuman_MultiBulk(sds *o, char *reply) {
+ char *p = strchr(reply+1,'\r');
+ long long mbulklen;
+ int j = 0;
+
+ string2ll(reply+1,p-reply-1,&mbulklen);
+ p += 2;
+ if (mbulklen == -1) {
+ *o = sdscatlen(*o,"NULL",4);
+ return p;
+ }
+ *o = sdscatlen(*o,"[",1);
+ for (j = 0; j < mbulklen; j++) {
+ p = ldbRedisProtocolToHuman(o,p);
+ if (j != mbulklen-1) *o = sdscatlen(*o,",",1);
+ }
+ *o = sdscatlen(*o,"]",1);
+ return p;
+}
+
+char *ldbRedisProtocolToHuman_Set(sds *o, char *reply) {
+ char *p = strchr(reply+1,'\r');
+ long long mbulklen;
+ int j = 0;
+
+ string2ll(reply+1,p-reply-1,&mbulklen);
+ p += 2;
+ *o = sdscatlen(*o,"~(",2);
+ for (j = 0; j < mbulklen; j++) {
+ p = ldbRedisProtocolToHuman(o,p);
+ if (j != mbulklen-1) *o = sdscatlen(*o,",",1);
+ }
+ *o = sdscatlen(*o,")",1);
+ return p;
+}
+
+char *ldbRedisProtocolToHuman_Map(sds *o, char *reply) {
+ char *p = strchr(reply+1,'\r');
+ long long mbulklen;
+ int j = 0;
+
+ string2ll(reply+1,p-reply-1,&mbulklen);
+ p += 2;
+ *o = sdscatlen(*o,"{",1);
+ for (j = 0; j < mbulklen; j++) {
+ p = ldbRedisProtocolToHuman(o,p);
+ *o = sdscatlen(*o," => ",4);
+ p = ldbRedisProtocolToHuman(o,p);
+ if (j != mbulklen-1) *o = sdscatlen(*o,",",1);
+ }
+ *o = sdscatlen(*o,"}",1);
+ return p;
+}
+
+char *ldbRedisProtocolToHuman_Null(sds *o, char *reply) {
+ char *p = strchr(reply+1,'\r');
+ *o = sdscatlen(*o,"(null)",6);
+ return p+2;
+}
+
+char *ldbRedisProtocolToHuman_Bool(sds *o, char *reply) {
+ char *p = strchr(reply+1,'\r');
+ if (reply[1] == 't')
+ *o = sdscatlen(*o,"#true",5);
+ else
+ *o = sdscatlen(*o,"#false",6);
+ return p+2;
+}
+
+char *ldbRedisProtocolToHuman_Double(sds *o, char *reply) {
+ char *p = strchr(reply+1,'\r');
+ *o = sdscatlen(*o,"(double) ",9);
+ *o = sdscatlen(*o,reply+1,p-reply-1);
+ return p+2;
+}
+
+/* Log a Redis reply as debugger output, in a human readable format.
+ * If the resulting string is longer than 'len' plus a few more chars
+ * used as prefix, it gets truncated. */
+void ldbLogRedisReply(char *reply) {
+ sds log = sdsnew("<reply> ");
+ ldbRedisProtocolToHuman(&log,reply);
+ ldbLogWithMaxLen(log);
+}
+
+/* Implements the "print <var>" command of the Lua debugger. It scans for Lua
+ * var "varname" starting from the current stack frame up to the top stack
+ * frame. The first matching variable is printed. */
+void ldbPrint(lua_State *lua, char *varname) {
+ lua_Debug ar;
+
+ int l = 0; /* Stack level. */
+ while (lua_getstack(lua,l,&ar) != 0) {
+ l++;
+ const char *name;
+ int i = 1; /* Variable index. */
+ while((name = lua_getlocal(lua,&ar,i)) != NULL) {
+ i++;
+ if (strcmp(varname,name) == 0) {
+ ldbLogStackValue(lua,"<value> ");
+ lua_pop(lua,1);
+ return;
+ } else {
+ lua_pop(lua,1); /* Discard the var name on the stack. */
+ }
+ }
+ }
+
+ /* Let's try with global vars in two selected cases */
+ if (!strcmp(varname,"ARGV") || !strcmp(varname,"KEYS")) {
+ lua_getglobal(lua, varname);
+ ldbLogStackValue(lua,"<value> ");
+ lua_pop(lua,1);
+ } else {
+ ldbLog(sdsnew("No such variable."));
+ }
+}
+
+/* Implements the "print" command (without arguments) of the Lua debugger.
+ * Prints all the variables in the current stack frame. */
+void ldbPrintAll(lua_State *lua) {
+ lua_Debug ar;
+ int vars = 0;
+
+ if (lua_getstack(lua,0,&ar) != 0) {
+ const char *name;
+ int i = 1; /* Variable index. */
+ while((name = lua_getlocal(lua,&ar,i)) != NULL) {
+ i++;
+ if (!strstr(name,"(*temporary)")) {
+ sds prefix = sdscatprintf(sdsempty(),"<value> %s = ",name);
+ ldbLogStackValue(lua,prefix);
+ sdsfree(prefix);
+ vars++;
+ }
+ lua_pop(lua,1);
+ }
+ }
+
+ if (vars == 0) {
+ ldbLog(sdsnew("No local variables in the current context."));
+ }
+}
+
+/* Implements the break command to list, add and remove breakpoints. */
+void ldbBreak(sds *argv, int argc) {
+ if (argc == 1) {
+ if (ldb.bpcount == 0) {
+ ldbLog(sdsnew("No breakpoints set. Use 'b <line>' to add one."));
+ return;
+ } else {
+ ldbLog(sdscatfmt(sdsempty(),"%i breakpoints set:",ldb.bpcount));
+ int j;
+ for (j = 0; j < ldb.bpcount; j++)
+ ldbLogSourceLine(ldb.bp[j]);
+ }
+ } else {
+ int j;
+ for (j = 1; j < argc; j++) {
+ char *arg = argv[j];
+ long line;
+ if (!string2l(arg,sdslen(arg),&line)) {
+ ldbLog(sdscatfmt(sdsempty(),"Invalid argument:'%s'",arg));
+ } else {
+ if (line == 0) {
+ ldb.bpcount = 0;
+ ldbLog(sdsnew("All breakpoints removed."));
+ } else if (line > 0) {
+ if (ldb.bpcount == LDB_BREAKPOINTS_MAX) {
+ ldbLog(sdsnew("Too many breakpoints set."));
+ } else if (ldbAddBreakpoint(line)) {
+ ldbList(line,1);
+ } else {
+ ldbLog(sdsnew("Wrong line number."));
+ }
+ } else if (line < 0) {
+ if (ldbDelBreakpoint(-line))
+ ldbLog(sdsnew("Breakpoint removed."));
+ else
+ ldbLog(sdsnew("No breakpoint in the specified line."));
+ }
+ }
+ }
+ }
+}
+
+/* Implements the Lua debugger "eval" command. It just compiles the user
+ * passed fragment of code and executes it, showing the result left on
+ * the stack. */
+void ldbEval(lua_State *lua, sds *argv, int argc) {
+ /* Glue the script together if it is composed of multiple arguments. */
+ sds code = sdsjoinsds(argv+1,argc-1," ",1);
+ sds expr = sdscatsds(sdsnew("return "),code);
+
+ /* Try to compile it as an expression, prepending "return ". */
+ if (luaL_loadbuffer(lua,expr,sdslen(expr),"@ldb_eval")) {
+ lua_pop(lua,1);
+ /* Failed? Try as a statement. */
+ if (luaL_loadbuffer(lua,code,sdslen(code),"@ldb_eval")) {
+ ldbLog(sdscatfmt(sdsempty(),"<error> %s",lua_tostring(lua,-1)));
+ lua_pop(lua,1);
+ sdsfree(code);
+ sdsfree(expr);
+ return;
+ }
+ }
+
+ /* Call it. */
+ sdsfree(code);
+ sdsfree(expr);
+ if (lua_pcall(lua,0,1,0)) {
+ ldbLog(sdscatfmt(sdsempty(),"<error> %s",lua_tostring(lua,-1)));
+ lua_pop(lua,1);
+ return;
+ }
+ ldbLogStackValue(lua,"<retval> ");
+ lua_pop(lua,1);
+}
+
+/* Implement the debugger "redis" command. We use a trick in order to make
+ * the implementation very simple: we just call the Lua redis.call() command
+ * implementation, with ldb.step enabled, so as a side effect the Redis command
+ * and its reply are logged. */
+void ldbRedis(lua_State *lua, sds *argv, int argc) {
+ int j, saved_rc = lctx.lua_replicate_commands;
+
+ if (!lua_checkstack(lua, argc + 1)) {
+ /* Increase the Lua stack if needed to make sure there is enough room
+ * to push 'argc + 1' elements to the stack. On failure, return error.
+ * Notice that we need, in worst case, 'argc + 1' elements because we push all the arguments
+ * given by the user (without the first argument) and we also push the 'redis' global table and
+ * 'redis.call' function so:
+ * (1 (redis table)) + (1 (redis.call function)) + (argc - 1 (all arguments without the first)) = argc + 1*/
+ ldbLogRedisReply("max lua stack reached");
+ return;
+ }
+
+ lua_getglobal(lua,"redis");
+ lua_pushstring(lua,"call");
+ lua_gettable(lua,-2); /* Stack: redis, redis.call */
+ for (j = 1; j < argc; j++)
+ lua_pushlstring(lua,argv[j],sdslen(argv[j]));
+ ldb.step = 1; /* Force redis.call() to log. */
+ lctx.lua_replicate_commands = 1;
+ lua_pcall(lua,argc-1,1,0); /* Stack: redis, result */
+ ldb.step = 0; /* Disable logging. */
+ lctx.lua_replicate_commands = saved_rc;
+ lua_pop(lua,2); /* Discard the result and clean the stack. */
+}
+
+/* Implements "trace" command of the Lua debugger. It just prints a backtrace
+ * querying Lua starting from the current callframe back to the outer one. */
+void ldbTrace(lua_State *lua) {
+ lua_Debug ar;
+ int level = 0;
+
+ while(lua_getstack(lua,level,&ar)) {
+ lua_getinfo(lua,"Snl",&ar);
+ if(strstr(ar.short_src,"user_script") != NULL) {
+ ldbLog(sdscatprintf(sdsempty(),"%s %s:",
+ (level == 0) ? "In" : "From",
+ ar.name ? ar.name : "top level"));
+ ldbLogSourceLine(ar.currentline);
+ }
+ level++;
+ }
+ if (level == 0) {
+ ldbLog(sdsnew("<error> Can't retrieve Lua stack."));
+ }
+}
+
+/* Implements the debugger "maxlen" command. It just queries or sets the
+ * ldb.maxlen variable. */
+void ldbMaxlen(sds *argv, int argc) {
+ if (argc == 2) {
+ int newval = atoi(argv[1]);
+ ldb.maxlen_hint_sent = 1; /* User knows about this command. */
+ if (newval != 0 && newval <= 60) newval = 60;
+ ldb.maxlen = newval;
+ }
+ if (ldb.maxlen) {
+ ldbLog(sdscatprintf(sdsempty(),"<value> replies are truncated at %d bytes.",(int)ldb.maxlen));
+ } else {
+ ldbLog(sdscatprintf(sdsempty(),"<value> replies are unlimited."));
+ }
+}
+
+/* Read debugging commands from client.
+ * Return C_OK if the debugging session is continuing, otherwise
+ * C_ERR if the client closed the connection or is timing out. */
+int ldbRepl(lua_State *lua) {
+ sds *argv;
+ int argc;
+ char* err = NULL;
+
+ /* We continue processing commands until a command that should return
+ * to the Lua interpreter is found. */
+ while(1) {
+ while((argv = ldbReplParseCommand(&argc, &err)) == NULL) {
+ char buf[1024];
+ if (err) {
+ lua_pushstring(lua, err);
+ lua_error(lua);
+ }
+ int nread = connRead(ldb.conn,buf,sizeof(buf));
+ if (nread <= 0) {
+ /* Make sure the script runs without user input since the
+ * client is no longer connected. */
+ ldb.step = 0;
+ ldb.bpcount = 0;
+ return C_ERR;
+ }
+ ldb.cbuf = sdscatlen(ldb.cbuf,buf,nread);
+ /* after 1M we will exit with an error
+ * so that the client will not blow the memory
+ */
+ if (sdslen(ldb.cbuf) > 1<<20) {
+ sdsfree(ldb.cbuf);
+ ldb.cbuf = sdsempty();
+ lua_pushstring(lua, "max client buffer reached");
+ lua_error(lua);
+ }
+ }
+
+ /* Flush the old buffer. */
+ sdsfree(ldb.cbuf);
+ ldb.cbuf = sdsempty();
+
+ /* Execute the command. */
+ if (!strcasecmp(argv[0],"h") || !strcasecmp(argv[0],"help")) {
+ldbLog(sdsnew("Redis Lua debugger help:"));
+ldbLog(sdsnew("[h]elp Show this help."));
+ldbLog(sdsnew("[s]tep Run current line and stop again."));
+ldbLog(sdsnew("[n]ext Alias for step."));
+ldbLog(sdsnew("[c]ontinue Run till next breakpoint."));
+ldbLog(sdsnew("[l]ist List source code around current line."));
+ldbLog(sdsnew("[l]ist [line] List source code around [line]."));
+ldbLog(sdsnew(" line = 0 means: current position."));
+ldbLog(sdsnew("[l]ist [line] [ctx] In this form [ctx] specifies how many lines"));
+ldbLog(sdsnew(" to show before/after [line]."));
+ldbLog(sdsnew("[w]hole List all source code. Alias for 'list 1 1000000'."));
+ldbLog(sdsnew("[p]rint Show all the local variables."));
+ldbLog(sdsnew("[p]rint <var> Show the value of the specified variable."));
+ldbLog(sdsnew(" Can also show global vars KEYS and ARGV."));
+ldbLog(sdsnew("[b]reak Show all breakpoints."));
+ldbLog(sdsnew("[b]reak <line> Add a breakpoint to the specified line."));
+ldbLog(sdsnew("[b]reak -<line> Remove breakpoint from the specified line."));
+ldbLog(sdsnew("[b]reak 0 Remove all breakpoints."));
+ldbLog(sdsnew("[t]race Show a backtrace."));
+ldbLog(sdsnew("[e]val <code> Execute some Lua code (in a different callframe)."));
+ldbLog(sdsnew("[r]edis <cmd> Execute a Redis command."));
+ldbLog(sdsnew("[m]axlen [len] Trim logged Redis replies and Lua var dumps to len."));
+ldbLog(sdsnew(" Specifying zero as <len> means unlimited."));
+ldbLog(sdsnew("[a]bort Stop the execution of the script. In sync"));
+ldbLog(sdsnew(" mode dataset changes will be retained."));
+ldbLog(sdsnew(""));
+ldbLog(sdsnew("Debugger functions you can call from Lua scripts:"));
+ldbLog(sdsnew("redis.debug() Produce logs in the debugger console."));
+ldbLog(sdsnew("redis.breakpoint() Stop execution like if there was a breakpoint in the"));
+ldbLog(sdsnew(" next line of code."));
+ ldbSendLogs();
+ } else if (!strcasecmp(argv[0],"s") || !strcasecmp(argv[0],"step") ||
+ !strcasecmp(argv[0],"n") || !strcasecmp(argv[0],"next")) {
+ ldb.step = 1;
+ break;
+ } else if (!strcasecmp(argv[0],"c") || !strcasecmp(argv[0],"continue")){
+ break;
+ } else if (!strcasecmp(argv[0],"t") || !strcasecmp(argv[0],"trace")) {
+ ldbTrace(lua);
+ ldbSendLogs();
+ } else if (!strcasecmp(argv[0],"m") || !strcasecmp(argv[0],"maxlen")) {
+ ldbMaxlen(argv,argc);
+ ldbSendLogs();
+ } else if (!strcasecmp(argv[0],"b") || !strcasecmp(argv[0],"break")) {
+ ldbBreak(argv,argc);
+ ldbSendLogs();
+ } else if (!strcasecmp(argv[0],"e") || !strcasecmp(argv[0],"eval")) {
+ ldbEval(lua,argv,argc);
+ ldbSendLogs();
+ } else if (!strcasecmp(argv[0],"a") || !strcasecmp(argv[0],"abort")) {
+ lua_pushstring(lua, "script aborted for user request");
+ lua_error(lua);
+ } else if (argc > 1 &&
+ (!strcasecmp(argv[0],"r") || !strcasecmp(argv[0],"redis"))) {
+ ldbRedis(lua,argv,argc);
+ ldbSendLogs();
+ } else if ((!strcasecmp(argv[0],"p") || !strcasecmp(argv[0],"print"))) {
+ if (argc == 2)
+ ldbPrint(lua,argv[1]);
+ else
+ ldbPrintAll(lua);
+ ldbSendLogs();
+ } else if (!strcasecmp(argv[0],"l") || !strcasecmp(argv[0],"list")){
+ int around = ldb.currentline, ctx = 5;
+ if (argc > 1) {
+ int num = atoi(argv[1]);
+ if (num > 0) around = num;
+ }
+ if (argc > 2) ctx = atoi(argv[2]);
+ ldbList(around,ctx);
+ ldbSendLogs();
+ } else if (!strcasecmp(argv[0],"w") || !strcasecmp(argv[0],"whole")){
+ ldbList(1,1000000);
+ ldbSendLogs();
+ } else {
+ ldbLog(sdsnew("<error> Unknown Redis Lua debugger command or "
+ "wrong number of arguments."));
+ ldbSendLogs();
+ }
+
+ /* Free the command vector. */
+ sdsfreesplitres(argv,argc);
+ }
+
+ /* Free the current command argv if we break inside the while loop. */
+ sdsfreesplitres(argv,argc);
+ return C_OK;
+}
+
+/* This is the core of our Lua debugger, called each time Lua is about
+ * to start executing a new line. */
+void luaLdbLineHook(lua_State *lua, lua_Debug *ar) {
+ scriptRunCtx* rctx = luaGetFromRegistry(lua, REGISTRY_RUN_CTX_NAME);
+ lua_getstack(lua,0,ar);
+ lua_getinfo(lua,"Sl",ar);
+ ldb.currentline = ar->currentline;
+
+ int bp = ldbIsBreakpoint(ldb.currentline) || ldb.luabp;
+ int timeout = 0;
+
+ /* Events outside our script are not interesting. */
+ if(strstr(ar->short_src,"user_script") == NULL) return;
+
+ /* Check if a timeout occurred. */
+ if (ar->event == LUA_HOOKCOUNT && ldb.step == 0 && bp == 0) {
+ mstime_t elapsed = elapsedMs(rctx->start_time);
+ mstime_t timelimit = server.script_time_limit ?
+ server.script_time_limit : 5000;
+ if (elapsed >= timelimit) {
+ timeout = 1;
+ ldb.step = 1;
+ } else {
+ return; /* No timeout, ignore the COUNT event. */
+ }
+ }
+
+ if (ldb.step || bp) {
+ char *reason = "step over";
+ if (bp) reason = ldb.luabp ? "redis.breakpoint() called" :
+ "break point";
+ else if (timeout) reason = "timeout reached, infinite loop?";
+ ldb.step = 0;
+ ldb.luabp = 0;
+ ldbLog(sdscatprintf(sdsempty(),
+ "* Stopped at %d, stop reason = %s",
+ ldb.currentline, reason));
+ ldbLogSourceLine(ldb.currentline);
+ ldbSendLogs();
+ if (ldbRepl(lua) == C_ERR && timeout) {
+ /* If the client closed the connection and we have a timeout
+ * connection, let's kill the script otherwise the process
+ * will remain blocked indefinitely. */
+ lua_pushstring(lua, "timeout during Lua debugging with client closing connection");
+ lua_error(lua);
+ }
+ rctx->start_time = getMonotonicUs();
+ rctx->snapshot_time = mstime();
+ }
+}
diff --git a/src/evict.c b/src/evict.c
index 4186378a2..a10c2d20e 100644
--- a/src/evict.c
+++ b/src/evict.c
@@ -33,6 +33,7 @@
#include "server.h"
#include "bio.h"
#include "atomicvar.h"
+#include "script.h"
#include <math.h>
/* ----------------------------------------------------------------------------
@@ -472,7 +473,7 @@ static int evictionTimeProc(
static int isSafeToPerformEvictions(void) {
/* - There must be no script in timeout condition.
* - Nor we are loading data right now. */
- if (server.lua_timedout || server.loading) return 0;
+ if (scriptIsTimedout() || server.loading) return 0;
/* By default replicas should ignore maxmemory
* and just be masters exact copies. */
diff --git a/src/function_lua.c b/src/function_lua.c
new file mode 100644
index 000000000..864ced809
--- /dev/null
+++ b/src/function_lua.c
@@ -0,0 +1,183 @@
+/*
+ * Copyright (c) 2021, Redis Ltd.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * * Redistributions of source code must retain the above copyright notice,
+ * this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Redis nor the names of its contributors may be used
+ * to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/*
+ * function_lua.c unit provides the Lua engine functionality.
+ * Including registering the engine and implementing the engine
+ * callbacks:
+ * * Create a function from blob (usually text)
+ * * Invoke a function
+ * * Free function memory
+ * * Get memory usage
+ *
+ * Uses script_lua.c to run the Lua code.
+ */
+
+#include "functions.h"
+#include "script_lua.h"
+#include <lua.h>
+#include <lauxlib.h>
+#include <lualib.h>
+
+#define LUA_ENGINE_NAME "LUA"
+#define REGISTRY_ENGINE_CTX_NAME "__ENGINE_CTX__"
+#define REGISTRY_ERROR_HANDLER_NAME "__ERROR_HANDLER__"
+
+/* Lua engine ctx */
+typedef struct luaEngineCtx {
+ lua_State *lua;
+} luaEngineCtx;
+
+/* Lua function ctx */
+typedef struct luaFunctionCtx {
+ /* Special ID that allows getting the Lua function object from the Lua registry */
+ int lua_function_ref;
+} luaFunctionCtx;
+
+/*
+ * Compile a given blob and save it on the registry.
+ * Return a function ctx with Lua ref that allows to later retrieve the
+ * function from the registry.
+ *
+ * Return NULL on compilation error and set the error to the err variable
+ */
+static void* luaEngineCreate(void *engine_ctx, sds blob, sds *err) {
+ luaEngineCtx *lua_engine_ctx = engine_ctx;
+ lua_State *lua = lua_engine_ctx->lua;
+ if (luaL_loadbuffer(lua, blob, sdslen(blob), "@user_function")) {
+ *err = sdsempty();
+ *err = sdscatprintf(*err, "Error compiling function: %s",
+ lua_tostring(lua, -1));
+ lua_pop(lua, 1);
+ return NULL;
+ }
+
+ serverAssert(lua_isfunction(lua, -1));
+
+ int lua_function_ref = luaL_ref(lua, LUA_REGISTRYINDEX);
+
+ luaFunctionCtx *f_ctx = zmalloc(sizeof(*f_ctx));
+ *f_ctx = (luaFunctionCtx ) { .lua_function_ref = lua_function_ref, };
+
+ return f_ctx;
+}
+
+/*
+ * Invole the give function with the given keys and args
+ */
+static void luaEngineCall(scriptRunCtx *run_ctx,
+ void *engine_ctx,
+ void *compiled_function,
+ robj **keys,
+ size_t nkeys,
+ robj **args,
+ size_t nargs)
+{
+ luaEngineCtx *lua_engine_ctx = engine_ctx;
+ lua_State *lua = lua_engine_ctx->lua;
+ luaFunctionCtx *f_ctx = compiled_function;
+
+ /* Push error handler */
+ lua_pushstring(lua, REGISTRY_ERROR_HANDLER_NAME);
+ lua_gettable(lua, LUA_REGISTRYINDEX);
+
+ lua_rawgeti(lua, LUA_REGISTRYINDEX, f_ctx->lua_function_ref);
+
+ serverAssert(lua_isfunction(lua, -1));
+
+ luaCallFunction(run_ctx, lua, keys, nkeys, args, nargs, 0);
+ lua_pop(lua, 1); /* Pop error handler */
+}
+
+static size_t luaEngineGetUsedMemoy(void *engine_ctx) {
+ luaEngineCtx *lua_engine_ctx = engine_ctx;
+ return luaMemory(lua_engine_ctx->lua);
+}
+
+static size_t luaEngineFunctionMemoryOverhead(void *compiled_function) {
+ return zmalloc_size(compiled_function);
+}
+
+static size_t luaEngineMemoryOverhead(void *engine_ctx) {
+ luaEngineCtx *lua_engine_ctx = engine_ctx;
+ return zmalloc_size(lua_engine_ctx);
+}
+
+static void luaEngineFreeFunction(void *engine_ctx, void *compiled_function) {
+ luaEngineCtx *lua_engine_ctx = engine_ctx;
+ lua_State *lua = lua_engine_ctx->lua;
+ luaFunctionCtx *f_ctx = compiled_function;
+ lua_unref(lua, f_ctx->lua_function_ref);
+ zfree(f_ctx);
+}
+
+/* Initialize Lua engine, should be called once on start. */
+int luaEngineInitEngine() {
+ luaEngineCtx *lua_engine_ctx = zmalloc(sizeof(*lua_engine_ctx));
+ lua_engine_ctx->lua = lua_open();
+
+ luaRegisterRedisAPI(lua_engine_ctx->lua);
+
+ /* Save error handler to registry */
+ lua_pushstring(lua_engine_ctx->lua, REGISTRY_ERROR_HANDLER_NAME);
+ char *errh_func = "local dbg = debug\n"
+ "local error_handler = function (err)\n"
+ " local i = dbg.getinfo(2,'nSl')\n"
+ " if i and i.what == 'C' then\n"
+ " i = dbg.getinfo(3,'nSl')\n"
+ " end\n"
+ " if i then\n"
+ " return i.source .. ':' .. i.currentline .. ': ' .. err\n"
+ " else\n"
+ " return err\n"
+ " end\n"
+ "end\n"
+ "return error_handler";
+ luaL_loadbuffer(lua_engine_ctx->lua, errh_func, strlen(errh_func), "@err_handler_def");
+ lua_pcall(lua_engine_ctx->lua,0,1,0);
+ lua_settable(lua_engine_ctx->lua, LUA_REGISTRYINDEX);
+
+ /* save the engine_ctx on the registry so we can get it from the Lua interpreter */
+ luaSaveOnRegistry(lua_engine_ctx->lua, REGISTRY_ENGINE_CTX_NAME, lua_engine_ctx);
+
+ luaEnableGlobalsProtection(lua_engine_ctx->lua, 0);
+
+
+ engine *lua_engine = zmalloc(sizeof(*lua_engine));
+ *lua_engine = (engine) {
+ .engine_ctx = lua_engine_ctx,
+ .create = luaEngineCreate,
+ .call = luaEngineCall,
+ .get_used_memory = luaEngineGetUsedMemoy,
+ .get_function_memory_overhead = luaEngineFunctionMemoryOverhead,
+ .get_engine_memory_overhead = luaEngineMemoryOverhead,
+ .free_function = luaEngineFreeFunction,
+ };
+ return functionsRegisterEngine(LUA_ENGINE_NAME, lua_engine);
+}
diff --git a/src/functions.c b/src/functions.c
new file mode 100644
index 000000000..b0d7a2305
--- /dev/null
+++ b/src/functions.c
@@ -0,0 +1,538 @@
+/*
+ * Copyright (c) 2021, Redis Ltd.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * * Redistributions of source code must retain the above copyright notice,
+ * this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Redis nor the names of its contributors may be used
+ * to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "functions.h"
+#include "sds.h"
+#include "dict.h"
+#include "adlist.h"
+#include "atomicvar.h"
+
+static size_t engine_cache_memory = 0;
+
+/* Forward declaration */
+static void engineFunctionDispose(dict *d, void *obj);
+
+struct functionsCtx {
+ dict *functions; /* Function name -> Function object that can be used to run the function */
+ size_t cache_memory /* Overhead memory (structs, dictionaries, ..) used by all the functions */;
+};
+
+dictType engineDictType = {
+ dictSdsCaseHash, /* hash function */
+ dictSdsDup, /* key dup */
+ NULL, /* val dup */
+ dictSdsKeyCaseCompare, /* key compare */
+ dictSdsDestructor, /* key destructor */
+ NULL, /* val destructor */
+ NULL /* allow to expand */
+};
+
+dictType functionDictType = {
+ dictSdsHash, /* hash function */
+ dictSdsDup, /* key dup */
+ NULL, /* val dup */
+ dictSdsKeyCompare, /* key compare */
+ dictSdsDestructor, /* key destructor */
+ engineFunctionDispose,/* val destructor */
+ NULL /* allow to expand */
+};
+
+/* Dictionary of engines */
+static dict *engines = NULL;
+
+/* Functions Ctx.
+ * Contains the dictionary that map a function name to
+ * function object and the cache memory used by all the functions */
+static functionsCtx *functions_ctx = NULL;
+
+static size_t functionMallocSize(functionInfo *fi) {
+ return zmalloc_size(fi) + sdsZmallocSize(fi->name)
+ + (fi->desc ? sdsZmallocSize(fi->desc) : 0)
+ + sdsZmallocSize(fi->code)
+ + fi->ei->engine->get_function_memory_overhead(fi->function);
+}
+
+/* Dispose function memory */
+static void engineFunctionDispose(dict *d, void *obj) {
+ UNUSED(d);
+ functionInfo *fi = obj;
+ sdsfree(fi->code);
+ sdsfree(fi->name);
+ if (fi->desc) {
+ sdsfree(fi->desc);
+ }
+ engine *engine = fi->ei->engine;
+ engine->free_function(engine->engine_ctx, fi->function);
+ zfree(fi);
+}
+
+/* Free function memory and detele it from the functions dictionary */
+static void engineFunctionFree(functionInfo *fi, functionsCtx *functions) {
+ functions->cache_memory -= functionMallocSize(fi);
+
+ dictDelete(functions->functions, fi->name);
+}
+
+/* Clear all the functions from the given functions ctx */
+void functionsCtxClear(functionsCtx *functions_ctx) {
+ dictEmpty(functions_ctx->functions, NULL);
+ functions_ctx->cache_memory = 0;
+}
+
+/* Free the given functions ctx */
+void functionsCtxFree(functionsCtx *functions_ctx) {
+ functionsCtxClear(functions_ctx);
+ dictRelease(functions_ctx->functions);
+ zfree(functions_ctx);
+}
+
+/* Swap the current functions ctx with the given one.
+ * Free the old functions ctx. */
+void functionsCtxSwapWithCurrent(functionsCtx *new_functions_ctx) {
+ functionsCtxFree(functions_ctx);
+ functions_ctx = new_functions_ctx;
+}
+
+/* return the current functions ctx */
+functionsCtx* functionsCtxGetCurrent() {
+ return functions_ctx;
+}
+
+/* Create a new functions ctx */
+functionsCtx* functionsCtxCreate() {
+ functionsCtx *ret = zmalloc(sizeof(functionsCtx));
+ ret->functions = dictCreate(&functionDictType);
+ ret->cache_memory = 0;
+ return ret;
+}
+
+/*
+ * Register a function info to functions dictionary
+ * 1. Set the function client
+ * 2. Add function to functions dictionary
+ * 3. update cache memory
+ */
+static void engineFunctionRegister(functionInfo *fi, functionsCtx *functions) {
+ int res = dictAdd(functions->functions, fi->name, fi);
+ serverAssert(res == DICT_OK);
+
+ functions->cache_memory += functionMallocSize(fi);
+}
+
+/*
+ * Creating a function info object and register it.
+ * Return the created object
+ */
+static functionInfo* engineFunctionCreate(sds name, void *function, engineInfo *ei,
+ sds desc, sds code, functionsCtx *functions)
+{
+
+ functionInfo *fi = zmalloc(sizeof(*fi));
+ *fi = (functionInfo ) {
+ .name = sdsdup(name),
+ .function = function,
+ .ei = ei,
+ .code = sdsdup(code),
+ .desc = desc ? sdsdup(desc) : NULL,
+ };
+
+ engineFunctionRegister(fi, functions);
+
+ return fi;
+}
+
+/* Register an engine, should be called once by the engine on startup and give the following:
+ *
+ * - engine_name - name of the engine to register
+ * - engine_ctx - the engine ctx that should be used by Redis to interact with the engine */
+int functionsRegisterEngine(const char *engine_name, engine *engine) {
+ sds engine_name_sds = sdsnew(engine_name);
+ if (dictFetchValue(engines, engine_name_sds)) {
+ serverLog(LL_WARNING, "Same engine was registered twice");
+ sdsfree(engine_name_sds);
+ return C_ERR;
+ }
+
+ client *c = createClient(NULL);
+ c->flags |= (CLIENT_DENY_BLOCKING | CLIENT_SCRIPT);
+ engineInfo *ei = zmalloc(sizeof(*ei));
+ *ei = (engineInfo ) { .name = engine_name_sds, .engine = engine, .c = c,};
+
+ dictAdd(engines, engine_name_sds, ei);
+
+ engine_cache_memory += zmalloc_size(ei) + sdsZmallocSize(ei->name) +
+ zmalloc_size(engine) +
+ engine->get_engine_memory_overhead(engine->engine_ctx);
+
+ return C_OK;
+}
+
+/*
+ * FUNCTION STATS
+ */
+void functionsStatsCommand(client *c) {
+ if (scriptIsRunning() && scriptIsEval()) {
+ addReplyErrorObject(c, shared.slowevalerr);
+ return;
+ }
+
+ addReplyMapLen(c, 2);
+
+ addReplyBulkCString(c, "running_script");
+ if (!scriptIsRunning()) {
+ addReplyNull(c);
+ } else {
+ addReplyMapLen(c, 3);
+ addReplyBulkCString(c, "name");
+ addReplyBulkCString(c, scriptCurrFunction());
+ addReplyBulkCString(c, "command");
+ client *script_client = scriptGetCaller();
+ addReplyArrayLen(c, script_client->argc);
+ for (int i = 0 ; i < script_client->argc ; ++i) {
+ addReplyBulkCBuffer(c, script_client->argv[i]->ptr, sdslen(script_client->argv[i]->ptr));
+ }
+ addReplyBulkCString(c, "duration_ms");
+ addReplyLongLong(c, scriptRunDuration());
+ }
+
+ addReplyBulkCString(c, "engines");
+ addReplyArrayLen(c, dictSize(engines));
+ dictIterator *iter = dictGetIterator(engines);
+ dictEntry *entry = NULL;
+ while ((entry = dictNext(iter))) {
+ engineInfo *ei = dictGetVal(entry);
+ addReplyBulkCString(c, ei->name);
+ }
+ dictReleaseIterator(iter);
+}
+
+/*
+ * FUNCTION LIST
+ */
+void functionsListCommand(client *c) {
+ /* general information on all the functions */
+ addReplyArrayLen(c, dictSize(functions_ctx->functions));
+ dictIterator *iter = dictGetIterator(functions_ctx->functions);
+ dictEntry *entry = NULL;
+ while ((entry = dictNext(iter))) {
+ functionInfo *fi = dictGetVal(entry);
+ addReplyMapLen(c, 3);
+ addReplyBulkCString(c, "name");
+ addReplyBulkCBuffer(c, fi->name, sdslen(fi->name));
+ addReplyBulkCString(c, "engine");
+ addReplyBulkCBuffer(c, fi->ei->name, sdslen(fi->ei->name));
+ addReplyBulkCString(c, "description");
+ if (fi->desc) {
+ addReplyBulkCBuffer(c, fi->desc, sdslen(fi->desc));
+ } else {
+ addReplyNull(c);
+ }
+ }
+ dictReleaseIterator(iter);
+}
+
+/*
+ * FUNCTION INFO <FUNCTION NAME> [WITHCODE]
+ */
+void functionsInfoCommand(client *c) {
+ if (c->argc > 4) {
+ addReplyErrorFormat(c,"wrong number of arguments for '%s' command or subcommand", c->cmd->name);
+ return;
+ }
+ /* dedicated information on specific function */
+ robj *function_name = c->argv[2];
+ int with_code = 0;
+ if (c->argc == 4) {
+ robj *with_code_arg = c->argv[3];
+ if (!strcasecmp(with_code_arg->ptr, "withcode")) {
+ with_code = 1;
+ }
+ }
+
+ functionInfo *fi = dictFetchValue(functions_ctx->functions, function_name->ptr);
+ if (!fi) {
+ addReplyError(c, "Function does not exists");
+ return;
+ }
+ addReplyMapLen(c, with_code? 4 : 3);
+ addReplyBulkCString(c, "name");
+ addReplyBulkCBuffer(c, fi->name, sdslen(fi->name));
+ addReplyBulkCString(c, "engine");
+ addReplyBulkCBuffer(c, fi->ei->name, sdslen(fi->ei->name));
+ addReplyBulkCString(c, "description");
+ if (fi->desc) {
+ addReplyBulkCBuffer(c, fi->desc, sdslen(fi->desc));
+ } else {
+ addReplyNull(c);
+ }
+ if (with_code) {
+ addReplyBulkCString(c, "code");
+ addReplyBulkCBuffer(c, fi->code, sdslen(fi->code));
+ }
+}
+
+/*
+ * FUNCTION DELETE <FUNCTION NAME>
+ */
+void functionsDeleteCommand(client *c) {
+ if (server.masterhost && server.repl_slave_ro && !(c->flags & CLIENT_MASTER)) {
+ addReplyError(c, "Can not delete a function on a read only replica");
+ return;
+ }
+
+ robj *function_name = c->argv[2];
+ functionInfo *fi = dictFetchValue(functions_ctx->functions, function_name->ptr);
+ if (!fi) {
+ addReplyError(c, "Function not found");
+ return;
+ }
+
+ engineFunctionFree(fi, functions_ctx);
+ forceCommandPropagation(c, PROPAGATE_REPL | PROPAGATE_AOF);
+ addReply(c, shared.ok);
+}
+
+void functionsKillCommand(client *c) {
+ scriptKill(c, 0);
+}
+
+static void fcallCommandGeneric(client *c, int ro) {
+ robj *function_name = c->argv[1];
+ functionInfo *fi = dictFetchValue(functions_ctx->functions, function_name->ptr);
+ if (!fi) {
+ addReplyError(c, "Function not found");
+ return;
+ }
+ engine *engine = fi->ei->engine;
+
+ long long numkeys;
+ /* Get the number of arguments that are keys */
+ if (getLongLongFromObject(c->argv[2], &numkeys) != C_OK) {
+ addReplyError(c, "Bad number of keys provided");
+ return;
+ }
+ if (numkeys > (c->argc - 3)) {
+ addReplyError(c, "Number of keys can't be greater than number of args");
+ return;
+ } else if (numkeys < 0) {
+ addReplyError(c, "Number of keys can't be negative");
+ return;
+ }
+
+ scriptRunCtx run_ctx;
+
+ scriptPrepareForRun(&run_ctx, fi->ei->c, c, fi->name);
+ if (ro) {
+ run_ctx.flags |= SCRIPT_READ_ONLY;
+ }
+ engine->call(&run_ctx, engine->engine_ctx, fi->function, c->argv + 3, numkeys,
+ c->argv + 3 + numkeys, c->argc - 3 - numkeys);
+ scriptResetRun(&run_ctx);
+}
+
+/*
+ * FCALL <FUNCTION NAME> nkeys <key1 .. keyn> <arg1 .. argn>
+ */
+void fcallCommand(client *c) {
+ fcallCommandGeneric(c, 0);
+}
+
+/*
+ * FCALL_RO <FUNCTION NAME> nkeys <key1 .. keyn> <arg1 .. argn>
+ */
+void fcallCommandReadOnly(client *c) {
+ fcallCommandGeneric(c, 1);
+}
+
+void functionsHelpCommand(client *c) {
+ const char *help[] = {
+"CREATE <ENGINE NAME> <FUNCTION NAME> [REPLACE] [DESC <FUNCTION DESCRIPTION>] <FUNCTION CODE>",
+" Create a new function with the given function name and code.",
+"DELETE <FUNCTION NAME>",
+" Delete the given function.",
+"INFO <FUNCTION NAME> [WITHCODE]",
+" For each function, print the following information about the function:",
+" * Function name",
+" * The engine used to run the function",
+" * Function description",
+" * Function code (only if WITHCODE is given)",
+"LIST",
+" Return general information on all the functions:",
+" * Function name",
+" * The engine used to run the function",
+" * Function description",
+"STATS",
+" Return information about the current function running:",
+" * Function name",
+" * Command used to run the function",
+" * Duration in MS that the function is running",
+" If not function is running, return nil",
+" In addition, returns a list of available engines.",
+"KILL",
+" Kill the current running function.",
+NULL };
+ addReplyHelp(c, help);
+}
+
+/* Compile and save the given function, return C_OK on success and C_ERR on failure.
+ * In case on failure the err out param is set with relevant error message */
+int functionsCreateWithFunctionCtx(sds function_name,sds engine_name, sds desc, sds code,
+ int replace, sds* err, functionsCtx *functions) {
+ engineInfo *ei = dictFetchValue(engines, engine_name);
+ if (!ei) {
+ *err = sdsnew("Engine not found");
+ return C_ERR;
+ }
+ engine *engine = ei->engine;
+
+ functionInfo *fi = dictFetchValue(functions->functions, function_name);
+ if (fi && !replace) {
+ *err = sdsnew("Function already exists");
+ return C_ERR;
+ }
+
+ void *function = engine->create(engine->engine_ctx, code, err);
+ if (*err) {
+ return C_ERR;
+ }
+
+ if (fi) {
+ /* free the already existing function as we are going to replace it */
+ engineFunctionFree(fi, functions);
+ }
+
+ engineFunctionCreate(function_name, function, ei, desc, code, functions);
+
+ return C_OK;
+}
+
+/*
+ * FUNCTION CREATE <ENGINE NAME> <FUNCTION NAME>
+ * [REPLACE] [DESC <FUNCTION DESCRIPTION>] <FUNCTION CODE>
+ *
+ * ENGINE NAME - name of the engine to use the run the function
+ * FUNCTION NAME - name to use to invoke the function
+ * REPLACE - optional, replace existing function
+ * DESCRIPTION - optional, function description
+ * FUNCTION CODE - function code to pass to the engine
+ */
+void functionsCreateCommand(client *c) {
+
+ if (server.masterhost && server.repl_slave_ro && !(c->flags & CLIENT_MASTER)) {
+ addReplyError(c, "Can not create a function on a read only replica");
+ return;
+ }
+
+ robj *engine_name = c->argv[2];
+ robj *function_name = c->argv[3];
+
+ int replace = 0;
+ int argc_pos = 4;
+ sds desc = NULL;
+ while (argc_pos < c->argc - 1) {
+ robj *next_arg = c->argv[argc_pos++];
+ if (!strcasecmp(next_arg->ptr, "replace")) {
+ replace = 1;
+ continue;
+ }
+ if (!strcasecmp(next_arg->ptr, "description")) {
+ if (argc_pos >= c->argc) {
+ addReplyError(c, "Bad function description");
+ return;
+ }
+ desc = c->argv[argc_pos++]->ptr;
+ continue;
+ }
+ }
+
+ if (argc_pos >= c->argc) {
+ addReplyError(c, "Function code is missing");
+ return;
+ }
+
+ robj *code = c->argv[argc_pos];
+ sds err = NULL;
+ if (functionsCreateWithFunctionCtx(function_name->ptr, engine_name->ptr,
+ desc, code->ptr, replace, &err, functions_ctx) != C_OK)
+ {
+ addReplyErrorSds(c, err);
+ return;
+ }
+ forceCommandPropagation(c, PROPAGATE_REPL | PROPAGATE_AOF);
+ addReply(c, shared.ok);
+}
+
+/* Return memory usage of all the engines combine */
+unsigned long functionsMemory() {
+ dictIterator *iter = dictGetIterator(engines);
+ dictEntry *entry = NULL;
+ size_t engines_nemory = 0;
+ while ((entry = dictNext(iter))) {
+ engineInfo *ei = dictGetVal(entry);
+ engine *engine = ei->engine;
+ engines_nemory += engine->get_used_memory(engine->engine_ctx);
+ }
+ dictReleaseIterator(iter);
+
+ return engines_nemory;
+}
+
+/* Return memory overhead of all the engines combine */
+unsigned long functionsMemoryOverhead() {
+ size_t memory_overhead = dictSize(engines) * sizeof(dictEntry) +
+ dictSlots(engines) * sizeof(dictEntry*);
+ memory_overhead += dictSize(functions_ctx->functions) * sizeof(dictEntry) +
+ dictSlots(functions_ctx->functions) * sizeof(dictEntry*) + sizeof(functionsCtx);
+ memory_overhead += functions_ctx->cache_memory;
+ memory_overhead += engine_cache_memory;
+
+ return memory_overhead;
+}
+
+/* Returns the number of functions */
+unsigned long functionsNum() {
+ return dictSize(functions_ctx->functions);
+}
+
+dict* functionsGet() {
+ return functions_ctx->functions;
+}
+
+/* Initialize engine data structures.
+ * Should be called once on server initialization */
+int functionsInit() {
+ engines = dictCreate(&engineDictType);
+ functions_ctx = functionsCtxCreate();
+
+ if (luaEngineInitEngine() != C_OK) {
+ return C_ERR;
+ }
+
+ return C_OK;
+}
diff --git a/src/functions.h b/src/functions.h
new file mode 100644
index 000000000..8675883df
--- /dev/null
+++ b/src/functions.h
@@ -0,0 +1,126 @@
+/*
+ * Copyright (c) 2021, Redis Ltd.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * * Redistributions of source code must retain the above copyright notice,
+ * this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Redis nor the names of its contributors may be used
+ * to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef __FUNCTIONS_H_
+#define __FUNCTIONS_H_
+
+/*
+ * functions.c unit provides the Redis Functions API:
+ * * FUNCTION CREATE
+ * * FUNCTION CALL
+ * * FUNCTION DELETE
+ * * FUNCTION KILL
+ * * FUNCTION INFO
+ *
+ * Also contains implementation for:
+ * * Save/Load function from rdb
+ * * Register engines
+ */
+
+#include "server.h"
+#include "script.h"
+#include "redismodule.h"
+
+typedef struct engine {
+ /* engine specific context */
+ void *engine_ctx;
+
+ /* Create function callback, get the engine_ctx, and function code.
+ * returns NULL on error and set sds to be the error message */
+ void* (*create)(void *engine_ctx, sds code, sds *err);
+
+ /* Invoking a function, r_ctx is an opaque object (from engine POV).
+ * The r_ctx should be used by the engine to interaction with Redis,
+ * such interaction could be running commands, set resp, or set
+ * replication mode
+ */
+ void (*call)(scriptRunCtx *r_ctx, void *engine_ctx, void *compiled_function,
+ robj **keys, size_t nkeys, robj **args, size_t nargs);
+
+ /* get current used memory by the engine */
+ size_t (*get_used_memory)(void *engine_ctx);
+
+ /* Return memory overhead for a given function,
+ * such memory is not counted as engine memory but as general
+ * structs memory that hold different information */
+ size_t (*get_function_memory_overhead)(void *compiled_function);
+
+ /* Return memory overhead for engine (struct size holding the engine)*/
+ size_t (*get_engine_memory_overhead)(void *engine_ctx);
+
+ /* free the given function */
+ void (*free_function)(void *engine_ctx, void *compiled_function);
+} engine;
+
+/* Hold information about an engine.
+ * Used on rdb.c so it must be declared here. */
+typedef struct engineInfo {
+ sds name; /* Name of the engine */
+ engine *engine; /* engine callbacks that allows to interact with the engine */
+ client *c; /* Client that is used to run commands */
+} engineInfo;
+
+/* Hold information about the specific function.
+ * Used on rdb.c so it must be declared here. */
+typedef struct functionInfo {
+ sds name; /* Function name */
+ void *function; /* Opaque object that set by the function's engine and allow it
+ to run the function, usually it's the function compiled code. */
+ engineInfo *ei; /* Pointer to the function engine */
+ sds code; /* Function code */
+ sds desc; /* Function description */
+} functionInfo;
+
+int functionsRegisterEngine(const char *engine_name, engine *engine_ctx);
+int functionsCreateWithFunctionCtx(sds function_name, sds engine_name, sds desc, sds code,
+ int replace, sds* err, functionsCtx *functions);
+void functionsCreateCommand(client *c);
+void fcallCommand(client *c);
+void fcallCommandReadOnly(client *c);
+void functionsDeleteCommand(client *c);
+void functionsKillCommand(client *c);
+void functionsStatsCommand(client *c);
+void functionsInfoCommand(client *c);
+void functionsListCommand(client *c);
+void functionsHelpCommand(client *c);
+unsigned long functionsMemory();
+unsigned long functionsMemoryOverhead();
+int functionsLoad(rio *rdb, int ver);
+unsigned long functionsNum();
+dict* functionsGet();
+functionsCtx* functionsCtxGetCurrent();
+functionsCtx* functionsCtxCreate();
+void functionsCtxFree(functionsCtx *functions_ctx);
+void functionsCtxClear(functionsCtx *functions_ctx);
+void functionsCtxSwapWithCurrent(functionsCtx *functions_ctx);
+
+int luaEngineInitEngine();
+int functionsInit();
+
+#endif /* __FUNCTIONS_H_ */
diff --git a/src/module.c b/src/module.c
index 7c1cc054b..bc7599b75 100644
--- a/src/module.c
+++ b/src/module.c
@@ -629,7 +629,7 @@ void moduleHandlePropagationAfterCommandCallback(RedisModuleCtx *ctx) {
/* If this command is executed from with Lua or MULTI/EXEC we do not
* need to propagate EXEC */
- if (server.in_eval || server.in_exec) return;
+ if (server.in_script || server.in_exec) return;
/* Handle the replication of the final EXEC, since whatever a command
* emits is always wrapped around MULTI/EXEC. */
@@ -2333,7 +2333,7 @@ int RM_ReplyWithLongDouble(RedisModuleCtx *ctx, long double ld) {
void moduleReplicateMultiIfNeeded(RedisModuleCtx *ctx) {
/* Skip this if client explicitly wrap the command with MULTI, or if
* the module command was called by a script. */
- if (server.in_eval || server.in_exec) return;
+ if (server.in_script || server.in_exec) return;
/* If we already emitted MULTI return ASAP. */
if (server.propagate_in_transaction) return;
/* If this is a thread safe context, we do not want to wrap commands
@@ -2709,7 +2709,7 @@ int RM_GetContextFlags(RedisModuleCtx *ctx) {
}
}
- if (server.in_eval)
+ if (server.in_script)
flags |= REDISMODULE_CTX_FLAGS_LUA;
if (server.in_exec)
@@ -6215,7 +6215,7 @@ void unblockClientFromModule(client *c) {
*/
RedisModuleBlockedClient *moduleBlockClient(RedisModuleCtx *ctx, RedisModuleCmdFunc reply_callback, RedisModuleCmdFunc timeout_callback, void (*free_privdata)(RedisModuleCtx*,void*), long long timeout_ms, RedisModuleString **keys, int numkeys, void *privdata) {
client *c = ctx->client;
- int islua = server.in_eval;
+ int islua = server.in_script;
int ismulti = server.in_exec;
c->bpop.module_blocked_handle = zmalloc(sizeof(RedisModuleBlockedClient));
diff --git a/src/networking.c b/src/networking.c
index 1f1a2e169..4275d62fb 100644
--- a/src/networking.c
+++ b/src/networking.c
@@ -30,6 +30,7 @@
#include "server.h"
#include "atomicvar.h"
#include "cluster.h"
+#include "script.h"
#include <sys/socket.h>
#include <sys/uio.h>
#include <math.h>
@@ -260,7 +261,7 @@ void clientInstallWriteHandler(client *c) {
int prepareClientToWrite(client *c) {
/* If it's the Lua client we always return ok without installing any
* handler since there is no socket at all. */
- if (c->flags & (CLIENT_LUA|CLIENT_MODULE)) return C_OK;
+ if (c->flags & (CLIENT_SCRIPT|CLIENT_MODULE)) return C_OK;
/* If CLIENT_CLOSE_ASAP flag is set, we need not write anything. */
if (c->flags & CLIENT_CLOSE_ASAP) return C_ERR;
@@ -1491,7 +1492,7 @@ void freeClientAsync(client *c) {
* may access the list while Redis uses I/O threads. All the other accesses
* are in the context of the main thread while the other threads are
* idle. */
- if (c->flags & CLIENT_CLOSE_ASAP || c->flags & CLIENT_LUA) return;
+ if (c->flags & CLIENT_CLOSE_ASAP || c->flags & CLIENT_SCRIPT) return;
c->flags |= CLIENT_CLOSE_ASAP;
if (server.io_threads_num == 1) {
/* no need to bother with locking if there's just one thread (the main thread) */
@@ -2199,7 +2200,7 @@ int processInputBuffer(client *c) {
* condition on the slave. We want just to accumulate the replication
* stream (instead of replying -BUSY like we do with other clients) and
* later resume the processing. */
- if (server.lua_timedout && c->flags & CLIENT_MASTER) break;
+ if (scriptIsTimedout() && c->flags & CLIENT_MASTER) break;
/* CLIENT_CLOSE_AFTER_REPLY closes the connection once the reply is
* written to the client. Make sure to not let the reply grow after
diff --git a/src/object.c b/src/object.c
index 5831f196d..7a5563ccb 100644
--- a/src/object.c
+++ b/src/object.c
@@ -29,6 +29,7 @@
*/
#include "server.h"
+#include "functions.h"
#include <math.h>
#include <ctype.h>
@@ -1203,9 +1204,7 @@ struct redisMemOverhead *getMemoryOverheadData(void) {
mh->aof_buffer = mem;
mem_total+=mem;
- mem = server.lua_scripts_mem;
- mem += dictSize(server.lua_scripts) * sizeof(dictEntry) +
- dictSlots(server.lua_scripts) * sizeof(dictEntry*);
+ mem = evalScriptsMemory();
mem += dictSize(server.repl_scriptcache_dict) * sizeof(dictEntry) +
dictSlots(server.repl_scriptcache_dict) * sizeof(dictEntry*);
if (listLength(server.repl_scriptcache_fifo) > 0) {
@@ -1214,6 +1213,8 @@ struct redisMemOverhead *getMemoryOverheadData(void) {
}
mh->lua_caches = mem;
mem_total+=mem;
+ mh->functions_caches = functionsMemoryOverhead();
+ mem_total+=mh->functions_caches;
for (j = 0; j < server.dbnum; j++) {
redisDb *db = server.db+j;
@@ -1325,7 +1326,7 @@ sds getMemoryDoctorReport(void) {
}
/* Too many scripts are cached? */
- if (dictSize(server.lua_scripts) > 1000) {
+ if (dictSize(evalScriptsDict()) > 1000) {
many_scripts = 1;
num_reports++;
}
@@ -1529,7 +1530,7 @@ NULL
} else if (!strcasecmp(c->argv[1]->ptr,"stats") && c->argc == 2) {
struct redisMemOverhead *mh = getMemoryOverheadData();
- addReplyMapLen(c,25+mh->num_dbs);
+ addReplyMapLen(c,26+mh->num_dbs);
addReplyBulkCString(c,"peak.allocated");
addReplyLongLong(c,mh->peak_allocated);
@@ -1555,6 +1556,9 @@ NULL
addReplyBulkCString(c,"lua.caches");
addReplyLongLong(c,mh->lua_caches);
+ addReplyBulkCString(c,"functions.caches");
+ addReplyLongLong(c,mh->functions_caches);
+
for (size_t j = 0; j < mh->num_dbs; j++) {
char dbname[32];
snprintf(dbname,sizeof(dbname),"db.%zd",mh->db[j].dbid);
diff --git a/src/rdb.c b/src/rdb.c
index 4a65f2cc9..28b29d65d 100644
--- a/src/rdb.c
+++ b/src/rdb.c
@@ -32,6 +32,7 @@
#include "zipmap.h"
#include "endianconv.h"
#include "stream.h"
+#include "functions.h"
#include <math.h>
#include <fcntl.h>
@@ -1239,6 +1240,25 @@ int rdbSaveRio(rio *rdb, int *error, int rdbflags, rdbSaveInfo *rsi) {
if (rdbSaveInfoAuxFields(rdb,rdbflags,rsi) == -1) goto werr;
if (rdbSaveModulesAux(rdb, REDISMODULE_AUX_BEFORE_RDB) == -1) goto werr;
+ /* save functions */
+ dict *functions = functionsGet();
+ dictIterator *iter = dictGetIterator(functions);
+ dictEntry *entry = NULL;
+ while ((entry = dictNext(iter))) {
+ rdbSaveType(rdb, RDB_OPCODE_FUNCTION);
+ functionInfo* fi = dictGetVal(entry);
+ if (rdbSaveRawString(rdb, (unsigned char *) fi->name, sdslen(fi->name)) == -1) goto werr;
+ if (rdbSaveRawString(rdb, (unsigned char *) fi->ei->name, sdslen(fi->ei->name)) == -1) goto werr;
+ if (fi->desc) {
+ if (rdbSaveLen(rdb, 1) == -1) goto werr; /* desc exists */
+ if (rdbSaveRawString(rdb, (unsigned char *) fi->desc, sdslen(fi->desc)) == -1) goto werr;
+ } else {
+ if (rdbSaveLen(rdb, 0) == -1) goto werr; /* desc not exists */
+ }
+ if (rdbSaveRawString(rdb, (unsigned char *) fi->code, sdslen(fi->code)) == -1) goto werr;
+ }
+ dictReleaseIterator(iter);
+
for (j = 0; j < server.dbnum; j++) {
redisDb *db = server.db+j;
dict *d = db->dict;
@@ -1303,8 +1323,8 @@ int rdbSaveRio(rio *rdb, int *error, int rdbflags, rdbSaveInfo *rsi) {
* the script cache as well: on successful PSYNC after a restart, we need
* to be able to process any EVALSHA inside the replication backlog the
* master will send us. */
- if (rsi && dictSize(server.lua_scripts)) {
- di = dictGetIterator(server.lua_scripts);
+ if (rsi && dictSize(evalScriptsDict())) {
+ di = dictGetIterator(evalScriptsDict());
while((de = dictNext(di)) != NULL) {
robj *body = dictGetVal(de);
if (rdbSaveAuxField(rdb,"lua",3,body->ptr,sdslen(body->ptr)) == -1)
@@ -2687,12 +2707,80 @@ void rdbLoadProgressCallback(rio *r, const void *buf, size_t len) {
}
}
+static int rdbFunctionLoad(rio *rdb, int ver, functionsCtx* functions_ctx) {
+ UNUSED(ver);
+ sds name = NULL;
+ sds engine_name = NULL;
+ sds desc = NULL;
+ sds blob = NULL;
+ sds err = NULL;
+ uint64_t has_desc;
+ int res = C_ERR;
+ if (!(name = rdbGenericLoadStringObject(rdb, RDB_LOAD_SDS, NULL))) {
+ serverLog(LL_WARNING, "Failed loading function name");
+ goto error;
+ }
+
+ if (!(engine_name = rdbGenericLoadStringObject(rdb, RDB_LOAD_SDS, NULL))) {
+ serverLog(LL_WARNING, "Failed loading engine name");
+ goto error;
+ }
+
+ if ((has_desc = rdbLoadLen(rdb, NULL)) == RDB_LENERR) {
+ serverLog(LL_WARNING, "Failed loading function desc indicator");
+ goto error;
+ }
+
+ if (has_desc && !(desc = rdbGenericLoadStringObject(rdb, RDB_LOAD_SDS, NULL))) {
+ serverLog(LL_WARNING, "Failed loading function desc");
+ goto error;
+ }
+
+ if (!(blob = rdbGenericLoadStringObject(rdb, RDB_LOAD_SDS, NULL))) {
+ serverLog(LL_WARNING, "Failed loading function blob");
+ goto error;
+ }
+
+ if (functionsCreateWithFunctionCtx(name, engine_name, desc, blob, 0, &err, functions_ctx) != C_OK) {
+ serverLog(LL_WARNING, "Failed compiling and saving the function %s", err);
+ goto error;
+ }
+
+ res = C_OK;
+
+error:
+ if (name) sdsfree(name);
+ if (engine_name) sdsfree(engine_name);
+ if (desc) sdsfree(desc);
+ if (blob) sdsfree(blob);
+ if (err) sdsfree(err);
+ return res;
+}
+
/* Load an RDB file from the rio stream 'rdb'. On success C_OK is returned,
* otherwise C_ERR is returned and 'errno' is set accordingly. */
-int rdbLoadRio(rio *rdb, int rdbflags, rdbSaveInfo *rsi, redisDb *dbarray) {
+int rdbLoadRio(rio *rdb, int rdbflags, rdbSaveInfo *rsi) {
+ functionsCtx* functions_ctx = functionsCtxGetCurrent();
+ functionsCtxClear(functions_ctx);
+ rdbLoadingCtx loading_ctx = { .dbarray = server.db, .functions_ctx = functions_ctx };
+ int retval = rdbLoadRioWithLoadingCtx(rdb,rdbflags,rsi,&loading_ctx);
+ if (retval != C_OK) {
+ /* Loading failed, clear the function ctx */
+ functionsCtxClear(functions_ctx);
+ }
+ return retval;
+}
+
+
+/* Load an RDB file from the rio stream 'rdb'. On success C_OK is returned,
+ * otherwise C_ERR is returned and 'errno' is set accordingly.
+ * The rdb_loading_ctx argument holds objects to which the rdb will be loaded to,
+ * currently it only allow to set db object and functionsCtx to which the data
+ * will be loaded (in the future it might contains more such objects). */
+int rdbLoadRioWithLoadingCtx(rio *rdb, int rdbflags, rdbSaveInfo *rsi, rdbLoadingCtx *rdb_loading_ctx) {
uint64_t dbid = 0;
int type, rdbver;
- redisDb *db = dbarray+0;
+ redisDb *db = rdb_loading_ctx->dbarray+0;
char buf[1024];
int error;
long long empty_keys_skipped = 0;
@@ -2764,7 +2852,7 @@ int rdbLoadRio(rio *rdb, int rdbflags, rdbSaveInfo *rsi, redisDb *dbarray) {
"databases. Exiting\n", server.dbnum);
exit(1);
}
- db = dbarray+dbid;
+ db = rdb_loading_ctx->dbarray+dbid;
continue; /* Read next opcode. */
} else if (type == RDB_OPCODE_RESIZEDB) {
/* RESIZEDB: Hint about the size of the keys in the currently
@@ -2808,7 +2896,7 @@ int rdbLoadRio(rio *rdb, int rdbflags, rdbSaveInfo *rsi, redisDb *dbarray) {
if (rsi) rsi->repl_offset = strtoll(auxval->ptr,NULL,10);
} else if (!strcasecmp(auxkey->ptr,"lua")) {
/* Load the script back in memory. */
- if (luaCreateFunction(NULL,server.lua,auxval) == NULL) {
+ if (luaCreateFunction(NULL, auxval) == NULL) {
rdbReportCorruptRDB(
"Can't load Lua script from RDB file! "
"BODY: %s", (char*)auxval->ptr);
@@ -2895,6 +2983,12 @@ int rdbLoadRio(rio *rdb, int rdbflags, rdbSaveInfo *rsi, redisDb *dbarray) {
decrRefCount(aux);
continue; /* Read next opcode. */
}
+ } else if (type == RDB_OPCODE_FUNCTION) {
+ if (rdbFunctionLoad(rdb, rdbver, rdb_loading_ctx->functions_ctx) != C_OK) {
+ serverLog(LL_WARNING,"Failed loading function");
+ goto eoferr;
+ }
+ continue;
}
/* Read key */
@@ -3044,7 +3138,9 @@ int rdbLoad(char *filename, rdbSaveInfo *rsi, int rdbflags) {
if ((fp = fopen(filename,"r")) == NULL) return C_ERR;
startLoadingFile(fp, filename,rdbflags);
rioInitWithFile(&rdb,fp);
- retval = rdbLoadRio(&rdb,rdbflags,rsi,server.db);
+
+ retval = rdbLoadRio(&rdb,rdbflags,rsi);
+
fclose(fp);
stopLoading(retval==C_OK);
return retval;
diff --git a/src/rdb.h b/src/rdb.h
index f150bcb0d..66496bcec 100644
--- a/src/rdb.h
+++ b/src/rdb.h
@@ -100,6 +100,7 @@
#define rdbIsObjectType(t) ((t >= 0 && t <= 7) || (t >= 9 && t <= 18))
/* Special RDB opcodes (saved/loaded with rdbSaveType/rdbLoadType). */
+#define RDB_OPCODE_FUNCTION 246 /* engine data */
#define RDB_OPCODE_MODULE_AUX 247 /* Module auxiliary data. */
#define RDB_OPCODE_IDLE 248 /* LRU idle time. */
#define RDB_OPCODE_FREQ 249 /* LFU frequency. */
@@ -166,7 +167,8 @@ int rdbSaveBinaryDoubleValue(rio *rdb, double val);
int rdbLoadBinaryDoubleValue(rio *rdb, double *val);
int rdbSaveBinaryFloatValue(rio *rdb, float val);
int rdbLoadBinaryFloatValue(rio *rdb, float *val);
-int rdbLoadRio(rio *rdb, int rdbflags, rdbSaveInfo *rsi, redisDb *db);
+int rdbLoadRio(rio *rdb, int rdbflags, rdbSaveInfo *rsi);
+int rdbLoadRioWithLoadingCtx(rio *rdb, int rdbflags, rdbSaveInfo *rsi, rdbLoadingCtx *rdb_loading_ctx);
int rdbSaveRio(rio *rdb, int *error, int rdbflags, rdbSaveInfo *rsi);
rdbSaveInfo *rdbPopulateSaveInfo(rdbSaveInfo *rsi);
diff --git a/src/replication.c b/src/replication.c
index 1a4aa4c2d..66483d934 100644
--- a/src/replication.c
+++ b/src/replication.c
@@ -32,6 +32,7 @@
#include "server.h"
#include "cluster.h"
#include "bio.h"
+#include "functions.h"
#include <memory.h>
#include <sys/time.h>
@@ -546,7 +547,7 @@ void replicationFeedMonitors(client *c, list *monitors, int dictid, robj **argv,
gettimeofday(&tv,NULL);
cmdrepr = sdscatprintf(cmdrepr,"%ld.%06ld ",(long)tv.tv_sec,(long)tv.tv_usec);
- if (c->flags & CLIENT_LUA) {
+ if (c->flags & CLIENT_SCRIPT) {
cmdrepr = sdscatprintf(cmdrepr,"[%d lua] ",dictid);
} else if (c->flags & CLIENT_UNIX_SOCKET) {
cmdrepr = sdscatprintf(cmdrepr,"[%d unix:%s] ",dictid,server.unixsocket);
@@ -1738,6 +1739,7 @@ void readSyncBulkPayload(connection *conn) {
ssize_t nread, readlen, nwritten;
int use_diskless_load = useDisklessLoad();
redisDb *diskless_load_tempDb = NULL;
+ functionsCtx* temp_functions_ctx = NULL;
int empty_db_flags = server.repl_slave_lazy_flush ? EMPTYDB_ASYNC :
EMPTYDB_NO_FLAGS;
off_t left;
@@ -1913,6 +1915,7 @@ void readSyncBulkPayload(connection *conn) {
if (use_diskless_load && server.repl_diskless_load == REPL_DISKLESS_LOAD_SWAPDB) {
/* Initialize empty tempDb dictionaries. */
diskless_load_tempDb = disklessLoadInitTempDb();
+ temp_functions_ctx = functionsCtxCreate();
moduleFireServerEvent(REDISMODULE_EVENT_REPL_ASYNC_LOAD,
REDISMODULE_SUBEVENT_REPL_ASYNC_LOAD_STARTED,
@@ -1935,6 +1938,7 @@ void readSyncBulkPayload(connection *conn) {
if (use_diskless_load) {
rio rdb;
redisDb *dbarray;
+ functionsCtx* functions_ctx;
int asyncLoading = 0;
if (server.repl_diskless_load == REPL_DISKLESS_LOAD_SWAPDB) {
@@ -1947,8 +1951,11 @@ void readSyncBulkPayload(connection *conn) {
asyncLoading = 1;
}
dbarray = diskless_load_tempDb;
+ functions_ctx = temp_functions_ctx;
} else {
dbarray = server.db;
+ functions_ctx = functionsCtxGetCurrent();
+ functionsCtxClear(functions_ctx);
}
rioInitWithConn(&rdb,conn,server.repl_transfer_size);
@@ -1960,7 +1967,8 @@ void readSyncBulkPayload(connection *conn) {
startLoading(server.repl_transfer_size, RDBFLAGS_REPLICATION, asyncLoading);
int loadingFailed = 0;
- if (rdbLoadRio(&rdb,RDBFLAGS_REPLICATION,&rsi,dbarray) != C_OK) {
+ rdbLoadingCtx loadingCtx = { .dbarray = dbarray, .functions_ctx = functions_ctx };
+ if (rdbLoadRioWithLoadingCtx(&rdb,RDBFLAGS_REPLICATION,&rsi,&loadingCtx) != C_OK) {
/* RDB loading failed. */
serverLog(LL_WARNING,
"Failed trying to load the MASTER synchronization DB "
@@ -1988,6 +1996,7 @@ void readSyncBulkPayload(connection *conn) {
NULL);
disklessLoadDiscardTempDb(diskless_load_tempDb);
+ functionsCtxFree(temp_functions_ctx);
serverLog(LL_NOTICE, "MASTER <-> REPLICA sync: Discarding temporary DB in background");
} else {
/* Remove the half-loaded data in case we started with an empty replica. */
@@ -2010,6 +2019,9 @@ void readSyncBulkPayload(connection *conn) {
serverLog(LL_NOTICE, "MASTER <-> REPLICA sync: Swapping active DB with loaded DB");
swapMainDbWithTempDb(diskless_load_tempDb);
+ /* swap existing functions ctx with the temporary one */
+ functionsCtxSwapWithCurrent(temp_functions_ctx);
+
moduleFireServerEvent(REDISMODULE_EVENT_REPL_ASYNC_LOAD,
REDISMODULE_SUBEVENT_REPL_ASYNC_LOAD_COMPLETED,
NULL);
diff --git a/src/script.c b/src/script.c
new file mode 100644
index 000000000..34e4953d3
--- /dev/null
+++ b/src/script.c
@@ -0,0 +1,464 @@
+/*
+ * Copyright (c) 2009-2021, Redis Ltd.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * * Redistributions of source code must retain the above copyright notice,
+ * this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Redis nor the names of its contributors may be used
+ * to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "server.h"
+#include "script.h"
+#include "cluster.h"
+
+/* On script invocation, holding the current run context */
+static scriptRunCtx *curr_run_ctx = NULL;
+
+static void exitScriptTimedoutMode(scriptRunCtx *run_ctx) {
+ serverAssert(run_ctx == curr_run_ctx);
+ serverAssert(scriptIsTimedout());
+ run_ctx->flags &= ~SCRIPT_TIMEDOUT;
+ blockingOperationEnds();
+ /* if we are a replica and we have an active master, set it for continue processing */
+ if (server.masterhost && server.master) queueClientForReprocessing(server.master);
+}
+
+static void enterScriptTimedoutMode(scriptRunCtx *run_ctx) {
+ serverAssert(run_ctx == curr_run_ctx);
+ serverAssert(!scriptIsTimedout());
+ /* Mark script as timedout */
+ run_ctx->flags |= SCRIPT_TIMEDOUT;
+ blockingOperationStarts();
+}
+
+int scriptIsTimedout() {
+ return scriptIsRunning() && (curr_run_ctx->flags & SCRIPT_TIMEDOUT);
+}
+
+client* scriptGetClient() {
+ serverAssert(scriptIsRunning());
+ return curr_run_ctx->c;
+}
+
+client* scriptGetCaller() {
+ serverAssert(scriptIsRunning());
+ return curr_run_ctx->original_client;
+}
+
+/* interrupt function for scripts, should be call
+ * from time to time to reply some special command (like ping)
+ * and also check if the run should be terminated. */
+int scriptInterrupt(scriptRunCtx *run_ctx) {
+ if (run_ctx->flags & SCRIPT_TIMEDOUT) {
+ /* script already timedout
+ we just need to precess some events and return */
+ processEventsWhileBlocked();
+ return (run_ctx->flags & SCRIPT_KILLED) ? SCRIPT_KILL : SCRIPT_CONTINUE;
+ }
+
+ long long elapsed = elapsedMs(run_ctx->start_time);
+ if (elapsed < server.script_time_limit) {
+ return SCRIPT_CONTINUE;
+ }
+
+ serverLog(LL_WARNING,
+ "Slow script detected: still in execution after %lld milliseconds. "
+ "You can try killing the script using the %s command.",
+ elapsed, (run_ctx->flags & SCRIPT_EVAL_MODE) ? "SCRIPT KILL" : "FUNCTION KILL");
+
+ enterScriptTimedoutMode(run_ctx);
+ /* Once the script timeouts we reenter the event loop to permit others
+ * some commands execution. For this reason
+ * we need to mask the client executing the script from the event loop.
+ * If we don't do that the client may disconnect and could no longer be
+ * here when the EVAL command will return. */
+ protectClient(run_ctx->original_client);
+
+ processEventsWhileBlocked();
+
+ return (run_ctx->flags & SCRIPT_KILLED) ? SCRIPT_KILL : SCRIPT_CONTINUE;
+}
+
+/* Prepare the given run ctx for execution */
+void scriptPrepareForRun(scriptRunCtx *run_ctx, client *engine_client, client *caller, const char *funcname) {
+ serverAssert(!curr_run_ctx);
+ /* set the curr_run_ctx so we can use it to kill the script if needed */
+ curr_run_ctx = run_ctx;
+
+ run_ctx->c = engine_client;
+ run_ctx->original_client = caller;
+ run_ctx->funcname = funcname;
+
+ client *script_client = run_ctx->c;
+ client *curr_client = run_ctx->original_client;
+ server.script_caller = curr_client;
+
+ /* Select the right DB in the context of the Lua client */
+ selectDb(script_client, curr_client->db->id);
+ script_client->resp = 2; /* Default is RESP2, scripts can change it. */
+
+ /* If we are in MULTI context, flag Lua client as CLIENT_MULTI. */
+ if (curr_client->flags & CLIENT_MULTI) {
+ script_client->flags |= CLIENT_MULTI;
+ }
+
+ server.in_script = 1;
+
+ run_ctx->start_time = getMonotonicUs();
+ run_ctx->snapshot_time = mstime();
+
+ run_ctx->flags = 0;
+ run_ctx->repl_flags = PROPAGATE_AOF | PROPAGATE_REPL;
+}
+
+/* Reset the given run ctx after execution */
+void scriptResetRun(scriptRunCtx *run_ctx) {
+ serverAssert(curr_run_ctx);
+
+ /* After the script done, remove the MULTI state. */
+ run_ctx->c->flags &= ~CLIENT_MULTI;
+
+ server.in_script = 0;
+ server.script_caller = NULL;
+
+ if (scriptIsTimedout()) {
+ exitScriptTimedoutMode(run_ctx);
+ /* Restore the client that was protected when the script timeout
+ * was detected. */
+ unprotectClient(run_ctx->original_client);
+ }
+
+ if (!(run_ctx->flags & SCRIPT_EVAL_REPLICATION)) {
+ preventCommandPropagation(run_ctx->original_client);
+ if (run_ctx->flags & SCRIPT_MULTI_EMMITED) {
+ execCommandPropagateExec(run_ctx->original_client->db->id);
+ }
+ }
+
+ /* unset curr_run_ctx so we will know there is no running script */
+ curr_run_ctx = NULL;
+}
+
+/* return true if a script is currently running */
+int scriptIsRunning() {
+ return curr_run_ctx != NULL;
+}
+
+const char* scriptCurrFunction() {
+ serverAssert(scriptIsRunning());
+ return curr_run_ctx->funcname;
+}
+
+int scriptIsEval() {
+ serverAssert(scriptIsRunning());
+ return curr_run_ctx->flags & SCRIPT_EVAL_MODE;
+}
+
+/* Kill the current running script */
+void scriptKill(client *c, int is_eval) {
+ if (!curr_run_ctx) {
+ addReplyError(c, "-NOTBUSY No scripts in execution right now.");
+ return;
+ }
+ if (curr_run_ctx->original_client->flags & CLIENT_MASTER) {
+ addReplyError(c,
+ "-UNKILLABLE The busy script was sent by a master instance in the context of replication and cannot be killed.");
+ }
+ if (curr_run_ctx->flags & SCRIPT_WRITE_DIRTY) {
+ addReplyError(c,
+ "-UNKILLABLE Sorry the script already executed write "
+ "commands against the dataset. You can either wait the "
+ "script termination or kill the server in a hard way "
+ "using the SHUTDOWN NOSAVE command.");
+ return;
+ }
+ if (is_eval && !(curr_run_ctx->flags & SCRIPT_EVAL_MODE)) {
+ /* Kill a function with 'SCRIPT KILL' is not allow */
+ addReplyErrorObject(c, shared.slowscripterr);
+ return;
+ }
+ if (!is_eval && (curr_run_ctx->flags & SCRIPT_EVAL_MODE)) {
+ /* Kill an eval with 'FUNCTION KILL' is not allow */
+ addReplyErrorObject(c, shared.slowevalerr);
+ return;
+ }
+ curr_run_ctx->flags |= SCRIPT_KILLED;
+ addReply(c, shared.ok);
+}
+
+static int scriptVerifyCommandArity(struct redisCommand *cmd, int argc, sds *err) {
+ if (!cmd || ((cmd->arity > 0 && cmd->arity != argc) || (argc < cmd->arity))) {
+ if (cmd)
+ *err = sdsnew("Wrong number of args calling Redis command from script");
+ else
+ *err = sdsnew("Unknown Redis command called from script");
+ return C_ERR;
+ }
+ return C_OK;
+}
+
+static int scriptVerifyACL(client *c, sds *err) {
+ /* Check the ACLs. */
+ int acl_errpos;
+ int acl_retval = ACLCheckAllPerm(c, &acl_errpos);
+ if (acl_retval != ACL_OK) {
+ addACLLogEntry(c,acl_retval,ACL_LOG_CTX_LUA,acl_errpos,NULL,NULL);
+ switch (acl_retval) {
+ case ACL_DENIED_CMD:
+ *err = sdsnew("The user executing the script can't run this "
+ "command or subcommand");
+ break;
+ case ACL_DENIED_KEY:
+ *err = sdsnew("The user executing the script can't access "
+ "at least one of the keys mentioned in the "
+ "command arguments");
+ break;
+ case ACL_DENIED_CHANNEL:
+ *err = sdsnew("The user executing the script can't publish "
+ "to the channel mentioned in the command");
+ break;
+ default:
+ *err = sdsnew("The user executing the script is lacking the "
+ "permissions for the command");
+ break;
+ }
+ return C_ERR;
+ }
+ return C_OK;
+}
+
+static int scriptVerifyWriteCommandAllow(scriptRunCtx *run_ctx, char **err) {
+ if (!(run_ctx->c->cmd->flags & CMD_WRITE)) {
+ return C_OK;
+ }
+
+ if (run_ctx->flags & SCRIPT_READ_ONLY) {
+ /* We know its a write command, on a read only run we do not allow it. */
+ *err = sdsnew("Write commands are not allowed from read-only scripts.");
+ return C_ERR;
+ }
+
+ if ((run_ctx->flags & SCRIPT_RANDOM_DIRTY) && (run_ctx->flags & SCRIPT_EVAL_REPLICATION)) {
+ *err = sdsnew("Write commands not allowed after non deterministic commands. Call redis.replicate_commands() at the start of your script in order to switch to single commands replication mode.");
+ return C_ERR;
+ }
+
+ /* Write commands are forbidden against read-only slaves, or if a
+ * command marked as non-deterministic was already called in the context
+ * of this script. */
+ int deny_write_type = writeCommandsDeniedByDiskError();
+
+ if (server.masterhost && server.repl_slave_ro && run_ctx->original_client->flags != CLIENT_ID_AOF
+ && !(run_ctx->original_client->flags & CLIENT_MASTER))
+ {
+ *err = sdsdup(shared.roslaveerr->ptr);
+ return C_ERR;
+ }
+
+ if (deny_write_type != DISK_ERROR_TYPE_NONE) {
+ if (deny_write_type == DISK_ERROR_TYPE_RDB) {
+ *err = sdsdup(shared.bgsaveerr->ptr);
+ } else {
+ *err = sdsempty();
+ *err = sdscatfmt(*err,
+ "MISCONF Errors writing to the AOF file: %s\r\n",
+ strerror(server.aof_last_write_errno));
+ }
+ return C_ERR;
+ }
+
+ return C_OK;
+}
+
+static int scriptVerifyOOM(scriptRunCtx *run_ctx, char **err) {
+ /* If we reached the memory limit configured via maxmemory, commands that
+ * could enlarge the memory usage are not allowed, but only if this is the
+ * first write in the context of this script, otherwise we can't stop
+ * in the middle. */
+
+ if (server.maxmemory && /* Maxmemory is actually enabled. */
+ run_ctx->original_client->id != CLIENT_ID_AOF && /* Don't care about mem if loading from AOF. */
+ !server.masterhost && /* Slave must execute the script. */
+ !(run_ctx->flags & SCRIPT_WRITE_DIRTY) && /* Script had no side effects so far. */
+ server.script_oom && /* Detected OOM when script start. */
+ (run_ctx->c->cmd->flags & CMD_DENYOOM))
+ {
+ *err = sdsdup(shared.oomerr->ptr);
+ return C_ERR;
+ }
+
+ return C_OK;
+}
+
+static int scriptVerifyClusterState(client *c, client *original_c, sds *err) {
+ if (!server.cluster_enabled || original_c->id == CLIENT_ID_AOF || (original_c->flags & CLIENT_MASTER)) {
+ return C_OK;
+ }
+ /* If this is a Redis Cluster node, we need to make sure the script is not
+ * trying to access non-local keys, with the exception of commands
+ * received from our master or when loading the AOF back in memory. */
+ int error_code;
+ /* Duplicate relevant flags in the script client. */
+ c->flags &= ~(CLIENT_READONLY | CLIENT_ASKING);
+ c->flags |= original_c->flags & (CLIENT_READONLY | CLIENT_ASKING);
+ if (getNodeByQuery(c, c->cmd, c->argv, c->argc, NULL, &error_code) != server.cluster->myself) {
+ if (error_code == CLUSTER_REDIR_DOWN_RO_STATE) {
+ *err = sdsnew(
+ "Script attempted to execute a write command while the "
+ "cluster is down and readonly");
+ } else if (error_code == CLUSTER_REDIR_DOWN_STATE) {
+ *err = sdsnew("Script attempted to execute a command while the "
+ "cluster is down");
+ } else {
+ *err = sdsnew("Script attempted to access a non local key in a "
+ "cluster node");
+ }
+ return C_ERR;
+ }
+ return C_OK;
+}
+
+static void scriptEmitMultiIfNeeded(scriptRunCtx *run_ctx) {
+ /* If we are using single commands replication, we need to wrap what
+ * we propagate into a MULTI/EXEC block, so that it will be atomic like
+ * a Lua script in the context of AOF and slaves. */
+ client *c = run_ctx->c;
+ if (!(run_ctx->flags & SCRIPT_EVAL_REPLICATION)
+ && !(run_ctx->flags & SCRIPT_MULTI_EMMITED)
+ && !(run_ctx->original_client->flags & CLIENT_MULTI)
+ && (run_ctx->flags & SCRIPT_WRITE_DIRTY)
+ && ((run_ctx->repl_flags & PROPAGATE_AOF)
+ || (run_ctx->repl_flags & PROPAGATE_REPL)))
+ {
+ execCommandPropagateMulti(run_ctx->original_client->db->id);
+ run_ctx->flags |= SCRIPT_MULTI_EMMITED;
+ /* Now we are in the MULTI context, the lua_client should be
+ * flag as CLIENT_MULTI. */
+ c->flags |= CLIENT_MULTI;
+ }
+}
+
+/* set RESP for a given run_ctx */
+int scriptSetResp(scriptRunCtx *run_ctx, int resp) {
+ if (resp != 2 && resp != 3) {
+ return C_ERR;
+ }
+
+ run_ctx->c->resp = resp;
+ return C_OK;
+}
+
+/* set Repl for a given run_ctx
+ * either: PROPAGATE_AOF | PROPAGATE_REPL*/
+int scriptSetRepl(scriptRunCtx *run_ctx, int repl) {
+ if ((repl & ~(PROPAGATE_AOF | PROPAGATE_REPL)) != 0) {
+ return C_ERR;
+ }
+ run_ctx->repl_flags = repl;
+ return C_OK;
+}
+
+/* Call a Redis command.
+ * The reply is written to the run_ctx client and it is
+ * up to the engine to take and parse.
+ * The err out variable is set only if error occurs and describe the error.
+ * If err is set on reply is written to the run_ctx client. */
+void scriptCall(scriptRunCtx *run_ctx, robj* *argv, int argc, sds *err) {
+ client *c = run_ctx->c;
+
+ /* Setup our fake client for command execution */
+ c->argv = argv;
+ c->argc = argc;
+ c->user = run_ctx->original_client->user;
+
+ /* Process module hooks */
+ moduleCallCommandFilters(c);
+ argv = c->argv;
+ argc = c->argc;
+
+ struct redisCommand *cmd = lookupCommand(argv, argc);
+ if (scriptVerifyCommandArity(cmd, argc, err) != C_OK) {
+ return;
+ }
+
+ c->cmd = c->lastcmd = cmd;
+
+ /* There are commands that are not allowed inside scripts. */
+ if (!server.script_disable_deny_script && (cmd->flags & CMD_NOSCRIPT)) {
+ *err = sdsnew("This Redis command is not allowed from script");
+ return;
+ }
+
+ if (scriptVerifyACL(c, err) != C_OK) {
+ return;
+ }
+
+ if (scriptVerifyWriteCommandAllow(run_ctx, err) != C_OK) {
+ return;
+ }
+
+ if (scriptVerifyOOM(run_ctx, err) != C_OK) {
+ return;
+ }
+
+ if (cmd->flags & CMD_WRITE) {
+ /* signify that we already change the data in this execution */
+ run_ctx->flags |= SCRIPT_WRITE_DIRTY;
+ }
+
+ if (cmd->flags & CMD_RANDOM) {
+ /* signify that we already perform a random command in this execution */
+ run_ctx->flags |= SCRIPT_RANDOM_DIRTY;
+ }
+
+ if (scriptVerifyClusterState(c, run_ctx->original_client, err) != C_OK) {
+ return;
+ }
+
+ scriptEmitMultiIfNeeded(run_ctx);
+
+ int call_flags = CMD_CALL_SLOWLOG | CMD_CALL_STATS;
+ if (!(run_ctx->flags & SCRIPT_EVAL_REPLICATION)) {
+ if (run_ctx->repl_flags & PROPAGATE_AOF) {
+ call_flags |= CMD_CALL_PROPAGATE_AOF;
+ }
+ if (run_ctx->repl_flags & PROPAGATE_REPL) {
+ call_flags |= CMD_CALL_PROPAGATE_REPL;
+ }
+ }
+ call(c, call_flags);
+ serverAssert((c->flags & CLIENT_BLOCKED) == 0);
+}
+
+/* Returns the time when the script invocation started */
+mstime_t scriptTimeSnapshot() {
+ serverAssert(!curr_run_ctx);
+ return curr_run_ctx->snapshot_time;
+}
+
+long long scriptRunDuration() {
+ serverAssert(scriptIsRunning());
+ return elapsedMs(curr_run_ctx->start_time);
+}
+
+
diff --git a/src/script.h b/src/script.h
new file mode 100644
index 000000000..aeed72456
--- /dev/null
+++ b/src/script.h
@@ -0,0 +1,101 @@
+/*
+ * Copyright (c) 2009-2021, Redis Ltd.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * * Redistributions of source code must retain the above copyright notice,
+ * this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Redis nor the names of its contributors may be used
+ * to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef __SCRIPT_H_
+#define __SCRIPT_H_
+
+/*
+ * Script.c unit provides an API for functions and eval
+ * to interact with Redis. Interaction includes mostly
+ * executing commands, but also functionalities like calling
+ * Redis back on long scripts or check if the script was killed.
+ *
+ * The interaction is done using a scriptRunCtx object that
+ * need to be created by the user and initialized using scriptPrepareForRun.
+ *
+ * Detailed list of functionalities expose by the unit:
+ * 1. Calling commands (including all the validation checks such as
+ * acl, cluster, read only run, ...)
+ * 2. Set Resp
+ * 3. Set Replication method (AOF/REPLICATION/NONE)
+ * 4. Call Redis back to on long running scripts to allow Redis reply
+ * to clients and perform script kill
+ */
+
+/*
+ * scriptInterrupt function will return one of those value,
+ *
+ * - SCRIPT_KILL - kill the current running script.
+ * - SCRIPT_CONTINUE - keep running the current script.
+ */
+#define SCRIPT_KILL 1
+#define SCRIPT_CONTINUE 2
+
+/* runCtx flags */
+#define SCRIPT_WRITE_DIRTY (1ULL<<0) /* indicate that the current script already performed a write command */
+#define SCRIPT_RANDOM_DIRTY (1ULL<<1) /* indicate that the current script already performed a random reply command.
+ Thanks to this flag we'll raise an error every time a write command
+ is called after a random command and prevent none deterministic
+ replication or AOF. */
+#define SCRIPT_MULTI_EMMITED (1ULL<<2) /* indicate that we already wrote a multi command to replication/aof */
+#define SCRIPT_TIMEDOUT (1ULL<<3) /* indicate that the current script timedout */
+#define SCRIPT_KILLED (1ULL<<4) /* indicate that the current script was marked to be killed */
+#define SCRIPT_READ_ONLY (1ULL<<5) /* indicate that the current script should only perform read commands */
+#define SCRIPT_EVAL_REPLICATION (1ULL<<6) /* mode for eval, indicate that we replicate the
+ script invocation and not the effects */
+#define SCRIPT_EVAL_MODE (1ULL<<7) /* Indicate that the current script called from legacy Lua */
+typedef struct scriptRunCtx scriptRunCtx;
+
+struct scriptRunCtx {
+ const char *funcname;
+ client *c;
+ client *original_client;
+ int flags;
+ int repl_flags;
+ monotime start_time;
+ mstime_t snapshot_time;
+};
+
+void scriptPrepareForRun(scriptRunCtx *r_ctx, client *engine_client, client *caller, const char *funcname);
+void scriptResetRun(scriptRunCtx *r_ctx);
+int scriptSetResp(scriptRunCtx *r_ctx, int resp);
+int scriptSetRepl(scriptRunCtx *r_ctx, int repl);
+void scriptCall(scriptRunCtx *r_ctx, robj **argv, int argc, sds *err);
+int scriptInterrupt(scriptRunCtx *r_ctx);
+void scriptKill(client *c, int is_eval);
+int scriptIsRunning();
+const char* scriptCurrFunction();
+int scriptIsEval();
+int scriptIsTimedout();
+client* scriptGetClient();
+client* scriptGetCaller();
+mstime_t scriptTimeSnapshot();
+long long scriptRunDuration();
+
+#endif /* __SCRIPT_H_ */
diff --git a/src/script_lua.c b/src/script_lua.c
new file mode 100644
index 000000000..aa73c5fb3
--- /dev/null
+++ b/src/script_lua.c
@@ -0,0 +1,1348 @@
+/*
+ * Copyright (c) 2009-2021, Redis Ltd.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * * Redistributions of source code must retain the above copyright notice,
+ * this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Redis nor the names of its contributors may be used
+ * to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "script_lua.h"
+
+#include "server.h"
+#include "sha1.h"
+#include "rand.h"
+#include "cluster.h"
+#include "monotonic.h"
+#include "resp_parser.h"
+#include <lauxlib.h>
+#include <lualib.h>
+#include <ctype.h>
+#include <math.h>
+
+static int redis_math_random (lua_State *L);
+static int redis_math_randomseed (lua_State *L);
+static void redisProtocolToLuaType_Int(void *ctx, long long val, const char *proto, size_t proto_len);
+static void redisProtocolToLuaType_BulkString(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len);
+static void redisProtocolToLuaType_NullBulkString(void *ctx, const char *proto, size_t proto_len);
+static void redisProtocolToLuaType_NullArray(void *ctx, const char *proto, size_t proto_len);
+static void redisProtocolToLuaType_Status(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len);
+static void redisProtocolToLuaType_Error(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len);
+static void redisProtocolToLuaType_Array(struct ReplyParser *parser, void *ctx, size_t len, const char *proto);
+static void redisProtocolToLuaType_Map(struct ReplyParser *parser, void *ctx, size_t len, const char *proto);
+static void redisProtocolToLuaType_Set(struct ReplyParser *parser, void *ctx, size_t len, const char *proto);
+static void redisProtocolToLuaType_Null(void *ctx, const char *proto, size_t proto_len);
+static void redisProtocolToLuaType_Bool(void *ctx, int val, const char *proto, size_t proto_len);
+static void redisProtocolToLuaType_Double(void *ctx, double d, const char *proto, size_t proto_len);
+static void redisProtocolToLuaType_BigNumber(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len);
+static void redisProtocolToLuaType_VerbatimString(void *ctx, const char *format, const char *str, size_t len, const char *proto, size_t proto_len);
+static void redisProtocolToLuaType_Attribute(struct ReplyParser *parser, void *ctx, size_t len, const char *proto);
+static void luaReplyToRedisReply(client *c, client* script_client, lua_State *lua);
+
+/*
+ * Save the give pointer on Lua registry, used to save the Lua context and
+ * function context so we can retrieve them from lua_State.
+ */
+void luaSaveOnRegistry(lua_State* lua, const char* name, void* ptr) {
+ lua_pushstring(lua, name);
+ if (ptr) {
+ lua_pushlightuserdata(lua, ptr);
+ } else {
+ lua_pushnil(lua);
+ }
+ lua_settable(lua, LUA_REGISTRYINDEX);
+}
+
+/*
+ * Get a saved pointer from registry
+ */
+void* luaGetFromRegistry(lua_State* lua, const char* name) {
+ lua_pushstring(lua, name);
+ lua_gettable(lua, LUA_REGISTRYINDEX);
+
+ /* must be light user data */
+ serverAssert(lua_islightuserdata(lua, -1));
+
+ void* ptr = (void*) lua_topointer(lua, -1);
+ serverAssert(ptr);
+
+ /* pops the value */
+ lua_pop(lua, 1);
+
+ return ptr;
+}
+
+/* ---------------------------------------------------------------------------
+ * Redis reply to Lua type conversion functions.
+ * ------------------------------------------------------------------------- */
+
+/* Take a Redis reply in the Redis protocol format and convert it into a
+ * Lua type. Thanks to this function, and the introduction of not connected
+ * clients, it is trivial to implement the redis() lua function.
+ *
+ * Basically we take the arguments, execute the Redis command in the context
+ * of a non connected client, then take the generated reply and convert it
+ * into a suitable Lua type. With this trick the scripting feature does not
+ * need the introduction of a full Redis internals API. The script
+ * is like a normal client that bypasses all the slow I/O paths.
+ *
+ * Note: in this function we do not do any sanity check as the reply is
+ * generated by Redis directly. This allows us to go faster.
+ *
+ * Errors are returned as a table with a single 'err' field set to the
+ * error string.
+ */
+
+static const ReplyParserCallbacks DefaultLuaTypeParserCallbacks = {
+ .null_array_callback = redisProtocolToLuaType_NullArray,
+ .bulk_string_callback = redisProtocolToLuaType_BulkString,
+ .null_bulk_string_callback = redisProtocolToLuaType_NullBulkString,
+ .error_callback = redisProtocolToLuaType_Error,
+ .simple_str_callback = redisProtocolToLuaType_Status,
+ .long_callback = redisProtocolToLuaType_Int,
+ .array_callback = redisProtocolToLuaType_Array,
+ .set_callback = redisProtocolToLuaType_Set,
+ .map_callback = redisProtocolToLuaType_Map,
+ .bool_callback = redisProtocolToLuaType_Bool,
+ .double_callback = redisProtocolToLuaType_Double,
+ .null_callback = redisProtocolToLuaType_Null,
+ .big_number_callback = redisProtocolToLuaType_BigNumber,
+ .verbatim_string_callback = redisProtocolToLuaType_VerbatimString,
+ .attribute_callback = redisProtocolToLuaType_Attribute,
+ .error = NULL,
+};
+
+static void redisProtocolToLuaType(lua_State *lua, char* reply) {
+ ReplyParser parser = {.curr_location = reply, .callbacks = DefaultLuaTypeParserCallbacks};
+
+ parseReply(&parser, lua);
+}
+
+static void redisProtocolToLuaType_Int(void *ctx, long long val, const char *proto, size_t proto_len) {
+ UNUSED(proto);
+ UNUSED(proto_len);
+ if (!ctx) {
+ return;
+ }
+
+ lua_State *lua = ctx;
+ if (!lua_checkstack(lua, 1)) {
+ /* Increase the Lua stack if needed, to make sure there is enough room
+ * to push elements to the stack. On failure, exit with panic. */
+ serverPanic("lua stack limit reach when parsing redis.call reply");
+ }
+ lua_pushnumber(lua,(lua_Number)val);
+}
+
+static void redisProtocolToLuaType_NullBulkString(void *ctx, const char *proto, size_t proto_len) {
+ UNUSED(proto);
+ UNUSED(proto_len);
+ if (!ctx) {
+ return;
+ }
+
+ lua_State *lua = ctx;
+ if (!lua_checkstack(lua, 1)) {
+ /* Increase the Lua stack if needed, to make sure there is enough room
+ * to push elements to the stack. On failure, exit with panic. */
+ serverPanic("lua stack limit reach when parsing redis.call reply");
+ }
+ lua_pushboolean(lua,0);
+}
+
+static void redisProtocolToLuaType_NullArray(void *ctx, const char *proto, size_t proto_len) {
+ UNUSED(proto);
+ UNUSED(proto_len);
+ if (!ctx) {
+ return;
+ }
+ lua_State *lua = ctx;
+ if (!lua_checkstack(lua, 1)) {
+ /* Increase the Lua stack if needed, to make sure there is enough room
+ * to push elements to the stack. On failure, exit with panic. */
+ serverPanic("lua stack limit reach when parsing redis.call reply");
+ }
+ lua_pushboolean(lua,0);
+}
+
+
+static void redisProtocolToLuaType_BulkString(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len) {
+ UNUSED(proto);
+ UNUSED(proto_len);
+ if (!ctx) {
+ return;
+ }
+
+ lua_State *lua = ctx;
+ if (!lua_checkstack(lua, 1)) {
+ /* Increase the Lua stack if needed, to make sure there is enough room
+ * to push elements to the stack. On failure, exit with panic. */
+ serverPanic("lua stack limit reach when parsing redis.call reply");
+ }
+ lua_pushlstring(lua,str,len);
+}
+
+static void redisProtocolToLuaType_Status(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len) {
+ UNUSED(proto);
+ UNUSED(proto_len);
+ if (!ctx) {
+ return;
+ }
+
+ lua_State *lua = ctx;
+ if (!lua_checkstack(lua, 3)) {
+ /* Increase the Lua stack if needed, to make sure there is enough room
+ * to push elements to the stack. On failure, exit with panic. */
+ serverPanic("lua stack limit reach when parsing redis.call reply");
+ }
+ lua_newtable(lua);
+ lua_pushstring(lua,"ok");
+ lua_pushlstring(lua,str,len);
+ lua_settable(lua,-3);
+}
+
+static void redisProtocolToLuaType_Error(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len) {
+ UNUSED(proto);
+ UNUSED(proto_len);
+ if (!ctx) {
+ return;
+ }
+
+ lua_State *lua = ctx;
+ if (!lua_checkstack(lua, 3)) {
+ /* Increase the Lua stack if needed, to make sure there is enough room
+ * to push elements to the stack. On failure, exit with panic. */
+ serverPanic("lua stack limit reach when parsing redis.call reply");
+ }
+ lua_newtable(lua);
+ lua_pushstring(lua,"err");
+ lua_pushlstring(lua,str,len);
+ lua_settable(lua,-3);
+}
+
+static void redisProtocolToLuaType_Map(struct ReplyParser *parser, void *ctx, size_t len, const char *proto) {
+ UNUSED(proto);
+ lua_State *lua = ctx;
+ if (lua) {
+ if (!lua_checkstack(lua, 3)) {
+ /* Increase the Lua stack if needed, to make sure there is enough room
+ * to push elements to the stack. On failure, exit with panic. */
+ serverPanic("lua stack limit reach when parsing redis.call reply");
+ }
+ lua_newtable(lua);
+ lua_pushstring(lua, "map");
+ lua_newtable(lua);
+ }
+ for (size_t j = 0; j < len; j++) {
+ parseReply(parser,lua);
+ parseReply(parser,lua);
+ if (lua) lua_settable(lua,-3);
+ }
+ if (lua) lua_settable(lua,-3);
+}
+
+static void redisProtocolToLuaType_Set(struct ReplyParser *parser, void *ctx, size_t len, const char *proto) {
+ UNUSED(proto);
+
+ lua_State *lua = ctx;
+ if (lua) {
+ if (!lua_checkstack(lua, 3)) {
+ /* Increase the Lua stack if needed, to make sure there is enough room
+ * to push elements to the stack. On failure, exit with panic. */
+ serverPanic("lua stack limit reach when parsing redis.call reply");
+ }
+ lua_newtable(lua);
+ lua_pushstring(lua, "set");
+ lua_newtable(lua);
+ }
+ for (size_t j = 0; j < len; j++) {
+ parseReply(parser,lua);
+ if (lua) {
+ if (!lua_checkstack(lua, 1)) {
+ /* Increase the Lua stack if needed, to make sure there is enough room
+ * to push elements to the stack. On failure, exit with panic.
+ * Notice that here we need to check the stack again because the recursive
+ * call to redisProtocolToLuaType might have use the room allocated in the stack*/
+ serverPanic("lua stack limit reach when parsing redis.call reply");
+ }
+ lua_pushboolean(lua,1);
+ lua_settable(lua,-3);
+ }
+ }
+ if (lua) lua_settable(lua,-3);
+}
+
+static void redisProtocolToLuaType_Array(struct ReplyParser *parser, void *ctx, size_t len, const char *proto) {
+ UNUSED(proto);
+
+ lua_State *lua = ctx;
+ if (lua){
+ if (!lua_checkstack(lua, 2)) {
+ /* Increase the Lua stack if needed, to make sure there is enough room
+ * to push elements to the stack. On failure, exit with panic. */
+ serverPanic("lua stack limit reach when parsing redis.call reply");
+ }
+ lua_newtable(lua);
+ }
+ for (size_t j = 0; j < len; j++) {
+ if (lua) lua_pushnumber(lua,j+1);
+ parseReply(parser,lua);
+ if (lua) lua_settable(lua,-3);
+ }
+}
+
+static void redisProtocolToLuaType_Attribute(struct ReplyParser *parser, void *ctx, size_t len, const char *proto) {
+ UNUSED(proto);
+
+ /* Parse the attribute reply.
+ * Currently, we do not expose the attribute to the Lua script so
+ * we just need to continue parsing and ignore it (the NULL ensures that the
+ * reply will be ignored). */
+ for (size_t j = 0; j < len; j++) {
+ parseReply(parser,NULL);
+ parseReply(parser,NULL);
+ }
+
+ /* Parse the reply itself. */
+ parseReply(parser,ctx);
+}
+
+static void redisProtocolToLuaType_VerbatimString(void *ctx, const char *format, const char *str, size_t len, const char *proto, size_t proto_len) {
+ UNUSED(proto);
+ UNUSED(proto_len);
+ if (!ctx) {
+ return;
+ }
+
+ lua_State *lua = ctx;
+ if (!lua_checkstack(lua, 5)) {
+ /* Increase the Lua stack if needed, to make sure there is enough room
+ * to push elements to the stack. On failure, exit with panic. */
+ serverPanic("lua stack limit reach when parsing redis.call reply");
+ }
+ lua_newtable(lua);
+ lua_pushstring(lua,"verbatim_string");
+ lua_newtable(lua);
+ lua_pushstring(lua,"string");
+ lua_pushlstring(lua,str,len);
+ lua_settable(lua,-3);
+ lua_pushstring(lua,"format");
+ lua_pushlstring(lua,format,3);
+ lua_settable(lua,-3);
+ lua_settable(lua,-3);
+}
+
+static void redisProtocolToLuaType_BigNumber(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len) {
+ UNUSED(proto);
+ UNUSED(proto_len);
+ if (!ctx) {
+ return;
+ }
+
+ lua_State *lua = ctx;
+ if (!lua_checkstack(lua, 3)) {
+ /* Increase the Lua stack if needed, to make sure there is enough room
+ * to push elements to the stack. On failure, exit with panic. */
+ serverPanic("lua stack limit reach when parsing redis.call reply");
+ }
+ lua_newtable(lua);
+ lua_pushstring(lua,"big_number");
+ lua_pushlstring(lua,str,len);
+ lua_settable(lua,-3);
+}
+
+static void redisProtocolToLuaType_Null(void *ctx, const char *proto, size_t proto_len) {
+ UNUSED(proto);
+ UNUSED(proto_len);
+ if (!ctx) {
+ return;
+ }
+
+ lua_State *lua = ctx;
+ if (!lua_checkstack(lua, 1)) {
+ /* Increase the Lua stack if needed, to make sure there is enough room
+ * to push elements to the stack. On failure, exit with panic. */
+ serverPanic("lua stack limit reach when parsing redis.call reply");
+ }
+ lua_pushnil(lua);
+}
+
+static void redisProtocolToLuaType_Bool(void *ctx, int val, const char *proto, size_t proto_len) {
+ UNUSED(proto);
+ UNUSED(proto_len);
+ if (!ctx) {
+ return;
+ }
+
+ lua_State *lua = ctx;
+ if (!lua_checkstack(lua, 1)) {
+ /* Increase the Lua stack if needed, to make sure there is enough room
+ * to push elements to the stack. On failure, exit with panic. */
+ serverPanic("lua stack limit reach when parsing redis.call reply");
+ }
+ lua_pushboolean(lua,val);
+}
+
+static void redisProtocolToLuaType_Double(void *ctx, double d, const char *proto, size_t proto_len) {
+ UNUSED(proto);
+ UNUSED(proto_len);
+ if (!ctx) {
+ return;
+ }
+
+ lua_State *lua = ctx;
+ if (!lua_checkstack(lua, 3)) {
+ /* Increase the Lua stack if needed, to make sure there is enough room
+ * to push elements to the stack. On failure, exit with panic. */
+ serverPanic("lua stack limit reach when parsing redis.call reply");
+ }
+ lua_newtable(lua);
+ lua_pushstring(lua,"double");
+ lua_pushnumber(lua,d);
+ lua_settable(lua,-3);
+}
+
+/* This function is used in order to push an error on the Lua stack in the
+ * format used by redis.pcall to return errors, which is a lua table
+ * with a single "err" field set to the error string. Note that this
+ * table is never a valid reply by proper commands, since the returned
+ * tables are otherwise always indexed by integers, never by strings. */
+static void luaPushError(lua_State *lua, char *error) {
+ lua_Debug dbg;
+
+ /* If debugging is active and in step mode, log errors resulting from
+ * Redis commands. */
+ if (ldbIsEnabled()) {
+ ldbLog(sdscatprintf(sdsempty(),"<error> %s",error));
+ }
+
+ lua_newtable(lua);
+ lua_pushstring(lua,"err");
+
+ /* Attempt to figure out where this function was called, if possible */
+ if(lua_getstack(lua, 1, &dbg) && lua_getinfo(lua, "nSl", &dbg)) {
+ sds msg = sdscatprintf(sdsempty(), "%s: %d: %s",
+ dbg.source, dbg.currentline, error);
+ lua_pushstring(lua, msg);
+ sdsfree(msg);
+ } else {
+ lua_pushstring(lua, error);
+ }
+ lua_settable(lua,-3);
+}
+
+/* In case the error set into the Lua stack by luaPushError() was generated
+ * by the non-error-trapping version of redis.pcall(), which is redis.call(),
+ * this function will raise the Lua error so that the execution of the
+ * script will be halted. */
+static int luaRaiseError(lua_State *lua) {
+ lua_pushstring(lua,"err");
+ lua_gettable(lua,-2);
+ return lua_error(lua);
+}
+
+/* Sort the array currently in the stack. We do this to make the output
+ * of commands like KEYS or SMEMBERS something deterministic when called
+ * from Lua (to play well with AOf/replication).
+ *
+ * The array is sorted using table.sort itself, and assuming all the
+ * list elements are strings. */
+static void luaSortArray(lua_State *lua) {
+ /* Initial Stack: array */
+ lua_getglobal(lua,"table");
+ lua_pushstring(lua,"sort");
+ lua_gettable(lua,-2); /* Stack: array, table, table.sort */
+ lua_pushvalue(lua,-3); /* Stack: array, table, table.sort, array */
+ if (lua_pcall(lua,1,0,0)) {
+ /* Stack: array, table, error */
+
+ /* We are not interested in the error, we assume that the problem is
+ * that there are 'false' elements inside the array, so we try
+ * again with a slower function but able to handle this case, that
+ * is: table.sort(table, __redis__compare_helper) */
+ lua_pop(lua,1); /* Stack: array, table */
+ lua_pushstring(lua,"sort"); /* Stack: array, table, sort */
+ lua_gettable(lua,-2); /* Stack: array, table, table.sort */
+ lua_pushvalue(lua,-3); /* Stack: array, table, table.sort, array */
+ lua_getglobal(lua,"__redis__compare_helper");
+ /* Stack: array, table, table.sort, array, __redis__compare_helper */
+ lua_call(lua,2,0);
+ }
+ /* Stack: array (sorted), table */
+ lua_pop(lua,1); /* Stack: array (sorted) */
+}
+
+/* ---------------------------------------------------------------------------
+ * Lua reply to Redis reply conversion functions.
+ * ------------------------------------------------------------------------- */
+
+/* Reply to client 'c' converting the top element in the Lua stack to a
+ * Redis reply. As a side effect the element is consumed from the stack. */
+static void luaReplyToRedisReply(client *c, client* script_client, lua_State *lua) {
+ int t = lua_type(lua,-1);
+
+ if (!lua_checkstack(lua, 4)) {
+ /* Increase the Lua stack if needed to make sure there is enough room
+ * to push 4 elements to the stack. On failure, return error.
+ * Notice that we need, in the worst case, 4 elements because returning a map might
+ * require push 4 elements to the Lua stack.*/
+ addReplyErrorFormat(c, "reached lua stack limit");
+ lua_pop(lua,1); /* pop the element from the stack */
+ return;
+ }
+
+ switch(t) {
+ case LUA_TSTRING:
+ addReplyBulkCBuffer(c,(char*)lua_tostring(lua,-1),lua_strlen(lua,-1));
+ break;
+ case LUA_TBOOLEAN:
+ if (script_client->resp == 2)
+ addReply(c,lua_toboolean(lua,-1) ? shared.cone :
+ shared.null[c->resp]);
+ else
+ addReplyBool(c,lua_toboolean(lua,-1));
+ break;
+ case LUA_TNUMBER:
+ addReplyLongLong(c,(long long)lua_tonumber(lua,-1));
+ break;
+ case LUA_TTABLE:
+ /* We need to check if it is an array, an error, or a status reply.
+ * Error are returned as a single element table with 'err' field.
+ * Status replies are returned as single element table with 'ok'
+ * field. */
+
+ /* Handle error reply. */
+ /* we took care of the stack size on function start */
+ lua_pushstring(lua,"err");
+ lua_gettable(lua,-2);
+ t = lua_type(lua,-1);
+ if (t == LUA_TSTRING) {
+ addReplyErrorFormat(c,"-%s",lua_tostring(lua,-1));
+ lua_pop(lua,2);
+ return;
+ }
+ lua_pop(lua,1); /* Discard field name pushed before. */
+
+ /* Handle status reply. */
+ lua_pushstring(lua,"ok");
+ lua_gettable(lua,-2);
+ t = lua_type(lua,-1);
+ if (t == LUA_TSTRING) {
+ sds ok = sdsnew(lua_tostring(lua,-1));
+ sdsmapchars(ok,"\r\n"," ",2);
+ addReplySds(c,sdscatprintf(sdsempty(),"+%s\r\n",ok));
+ sdsfree(ok);
+ lua_pop(lua,2);
+ return;
+ }
+ lua_pop(lua,1); /* Discard field name pushed before. */
+
+ /* Handle double reply. */
+ lua_pushstring(lua,"double");
+ lua_gettable(lua,-2);
+ t = lua_type(lua,-1);
+ if (t == LUA_TNUMBER) {
+ addReplyDouble(c,lua_tonumber(lua,-1));
+ lua_pop(lua,2);
+ return;
+ }
+ lua_pop(lua,1); /* Discard field name pushed before. */
+
+ /* Handle big number reply. */
+ lua_pushstring(lua,"big_number");
+ lua_gettable(lua,-2);
+ t = lua_type(lua,-1);
+ if (t == LUA_TSTRING) {
+ sds big_num = sdsnewlen(lua_tostring(lua,-1), lua_strlen(lua,-1));
+ sdsmapchars(big_num,"\r\n"," ",2);
+ addReplyBigNum(c,big_num,sdslen(big_num));
+ sdsfree(big_num);
+ lua_pop(lua,2);
+ return;
+ }
+ lua_pop(lua,1); /* Discard field name pushed before. */
+
+ /* Handle verbatim reply. */
+ lua_pushstring(lua,"verbatim_string");
+ lua_gettable(lua,-2);
+ t = lua_type(lua,-1);
+ if (t == LUA_TTABLE) {
+ lua_pushstring(lua,"format");
+ lua_gettable(lua,-2);
+ t = lua_type(lua,-1);
+ if (t == LUA_TSTRING){
+ char* format = (char*)lua_tostring(lua,-1);
+ lua_pushstring(lua,"string");
+ lua_gettable(lua,-3);
+ t = lua_type(lua,-1);
+ if (t == LUA_TSTRING){
+ size_t len;
+ char* str = (char*)lua_tolstring(lua,-1,&len);
+ addReplyVerbatim(c, str, len, format);
+ lua_pop(lua,4);
+ return;
+ }
+ lua_pop(lua,1);
+ }
+ lua_pop(lua,1);
+ }
+ lua_pop(lua,1); /* Discard field name pushed before. */
+
+ /* Handle map reply. */
+ lua_pushstring(lua,"map");
+ lua_gettable(lua,-2);
+ t = lua_type(lua,-1);
+ if (t == LUA_TTABLE) {
+ int maplen = 0;
+ void *replylen = addReplyDeferredLen(c);
+ /* we took care of the stack size on function start */
+ lua_pushnil(lua); /* Use nil to start iteration. */
+ while (lua_next(lua,-2)) {
+ /* Stack now: table, key, value */
+ lua_pushvalue(lua,-2); /* Dup key before consuming. */
+ luaReplyToRedisReply(c, script_client, lua); /* Return key. */
+ luaReplyToRedisReply(c, script_client, lua); /* Return value. */
+ /* Stack now: table, key. */
+ maplen++;
+ }
+ setDeferredMapLen(c,replylen,maplen);
+ lua_pop(lua,2);
+ return;
+ }
+ lua_pop(lua,1); /* Discard field name pushed before. */
+
+ /* Handle set reply. */
+ lua_pushstring(lua,"set");
+ lua_gettable(lua,-2);
+ t = lua_type(lua,-1);
+ if (t == LUA_TTABLE) {
+ int setlen = 0;
+ void *replylen = addReplyDeferredLen(c);
+ /* we took care of the stack size on function start */
+ lua_pushnil(lua); /* Use nil to start iteration. */
+ while (lua_next(lua,-2)) {
+ /* Stack now: table, key, true */
+ lua_pop(lua,1); /* Discard the boolean value. */
+ lua_pushvalue(lua,-1); /* Dup key before consuming. */
+ luaReplyToRedisReply(c, script_client, lua); /* Return key. */
+ /* Stack now: table, key. */
+ setlen++;
+ }
+ setDeferredSetLen(c,replylen,setlen);
+ lua_pop(lua,2);
+ return;
+ }
+ lua_pop(lua,1); /* Discard field name pushed before. */
+
+ /* Handle the array reply. */
+ void *replylen = addReplyDeferredLen(c);
+ int j = 1, mbulklen = 0;
+ while(1) {
+ /* we took care of the stack size on function start */
+ lua_pushnumber(lua,j++);
+ lua_gettable(lua,-2);
+ t = lua_type(lua,-1);
+ if (t == LUA_TNIL) {
+ lua_pop(lua,1);
+ break;
+ }
+ luaReplyToRedisReply(c, script_client, lua);
+ mbulklen++;
+ }
+ setDeferredArrayLen(c,replylen,mbulklen);
+ break;
+ default:
+ addReplyNull(c);
+ }
+ lua_pop(lua,1);
+}
+
+/* ---------------------------------------------------------------------------
+ * Lua redis.* functions implementations.
+ * ------------------------------------------------------------------------- */
+
+#define LUA_CMD_OBJCACHE_SIZE 32
+#define LUA_CMD_OBJCACHE_MAX_LEN 64
+static int luaRedisGenericCommand(lua_State *lua, int raise_error) {
+ int j, argc = lua_gettop(lua);
+ scriptRunCtx* rctx = luaGetFromRegistry(lua, REGISTRY_RUN_CTX_NAME);
+ sds err = NULL;
+ client* c = rctx->c;
+ sds reply;
+
+ /* Cached across calls. */
+ static robj **argv = NULL;
+ static int argv_size = 0;
+ static robj *cached_objects[LUA_CMD_OBJCACHE_SIZE];
+ static size_t cached_objects_len[LUA_CMD_OBJCACHE_SIZE];
+ static int inuse = 0; /* Recursive calls detection. */
+
+ /* By using Lua debug hooks it is possible to trigger a recursive call
+ * to luaRedisGenericCommand(), which normally should never happen.
+ * To make this function reentrant is futile and makes it slower, but
+ * we should at least detect such a misuse, and abort. */
+ if (inuse) {
+ char *recursion_warning =
+ "luaRedisGenericCommand() recursive call detected. "
+ "Are you doing funny stuff with Lua debug hooks?";
+ serverLog(LL_WARNING,"%s",recursion_warning);
+ luaPushError(lua,recursion_warning);
+ return 1;
+ }
+ inuse++;
+
+ /* Require at least one argument */
+ if (argc == 0) {
+ luaPushError(lua,
+ "Please specify at least one argument for redis.call()");
+ inuse--;
+ return raise_error ? luaRaiseError(lua) : 1;
+ }
+
+ /* Build the arguments vector */
+ if (argv_size < argc) {
+ argv = zrealloc(argv,sizeof(robj*)*argc);
+ argv_size = argc;
+ }
+
+ for (j = 0; j < argc; j++) {
+ char *obj_s;
+ size_t obj_len;
+ char dbuf[64];
+
+ if (lua_type(lua,j+1) == LUA_TNUMBER) {
+ /* We can't use lua_tolstring() for number -> string conversion
+ * since Lua uses a format specifier that loses precision. */
+ lua_Number num = lua_tonumber(lua,j+1);
+
+ obj_len = snprintf(dbuf,sizeof(dbuf),"%.17g",(double)num);
+ obj_s = dbuf;
+ } else {
+ obj_s = (char*)lua_tolstring(lua,j+1,&obj_len);
+ if (obj_s == NULL) break; /* Not a string. */
+ }
+
+ /* Try to use a cached object. */
+ if (j < LUA_CMD_OBJCACHE_SIZE && cached_objects[j] &&
+ cached_objects_len[j] >= obj_len)
+ {
+ sds s = cached_objects[j]->ptr;
+ argv[j] = cached_objects[j];
+ cached_objects[j] = NULL;
+ memcpy(s,obj_s,obj_len+1);
+ sdssetlen(s, obj_len);
+ } else {
+ argv[j] = createStringObject(obj_s, obj_len);
+ }
+ }
+
+ /* Check if one of the arguments passed by the Lua script
+ * is not a string or an integer (lua_isstring() return true for
+ * integers as well). */
+ if (j != argc) {
+ j--;
+ while (j >= 0) {
+ decrRefCount(argv[j]);
+ j--;
+ }
+ luaPushError(lua,
+ "Lua redis() command arguments must be strings or integers");
+ inuse--;
+ return raise_error ? luaRaiseError(lua) : 1;
+ }
+
+ /* Pop all arguments from the stack, we do not need them anymore
+ * and this way we guaranty we will have room on the stack for the result. */
+ lua_pop(lua, argc);
+
+ /* Log the command if debugging is active. */
+ if (ldbIsEnabled()) {
+ sds cmdlog = sdsnew("<redis>");
+ for (j = 0; j < c->argc; j++) {
+ if (j == 10) {
+ cmdlog = sdscatprintf(cmdlog," ... (%d more)",
+ c->argc-j-1);
+ break;
+ } else {
+ cmdlog = sdscatlen(cmdlog," ",1);
+ cmdlog = sdscatsds(cmdlog,c->argv[j]->ptr);
+ }
+ }
+ ldbLog(cmdlog);
+ }
+
+
+ scriptCall(rctx, argv, argc, &err);
+ if (err) {
+ luaPushError(lua, err);
+ sdsfree(err);
+ goto cleanup;
+ }
+
+ /* Convert the result of the Redis command into a suitable Lua type.
+ * The first thing we need is to create a single string from the client
+ * output buffers. */
+ if (listLength(c->reply) == 0 && (size_t)c->bufpos < c->buf_usable_size) {
+ /* This is a fast path for the common case of a reply inside the
+ * client static buffer. Don't create an SDS string but just use
+ * the client buffer directly. */
+ c->buf[c->bufpos] = '\0';
+ reply = c->buf;
+ c->bufpos = 0;
+ } else {
+ reply = sdsnewlen(c->buf,c->bufpos);
+ c->bufpos = 0;
+ while(listLength(c->reply)) {
+ clientReplyBlock *o = listNodeValue(listFirst(c->reply));
+
+ reply = sdscatlen(reply,o->buf,o->used);
+ listDelNode(c->reply,listFirst(c->reply));
+ }
+ }
+ if (raise_error && reply[0] != '-') raise_error = 0;
+ redisProtocolToLuaType(lua,reply);
+
+ /* If the debugger is active, log the reply from Redis. */
+ if (ldbIsEnabled())
+ ldbLogRedisReply(reply);
+
+ /* Sort the output array if needed, assuming it is a non-null multi bulk
+ * reply as expected. */
+ if ((c->cmd->flags & CMD_SORT_FOR_SCRIPT) &&
+ (rctx->flags & SCRIPT_EVAL_REPLICATION) &&
+ (reply[0] == '*' && reply[1] != '-')) {
+ luaSortArray(lua);
+ }
+ if (reply != c->buf) sdsfree(reply);
+ c->reply_bytes = 0;
+
+cleanup:
+ /* Clean up. Command code may have changed argv/argc so we use the
+ * argv/argc of the client instead of the local variables. */
+ for (j = 0; j < c->argc; j++) {
+ robj *o = c->argv[j];
+
+ /* Try to cache the object in the cached_objects array.
+ * The object must be small, SDS-encoded, and with refcount = 1
+ * (we must be the only owner) for us to cache it. */
+ if (j < LUA_CMD_OBJCACHE_SIZE &&
+ o->refcount == 1 &&
+ (o->encoding == OBJ_ENCODING_RAW ||
+ o->encoding == OBJ_ENCODING_EMBSTR) &&
+ sdslen(o->ptr) <= LUA_CMD_OBJCACHE_MAX_LEN)
+ {
+ sds s = o->ptr;
+ if (cached_objects[j]) decrRefCount(cached_objects[j]);
+ cached_objects[j] = o;
+ cached_objects_len[j] = sdsalloc(s);
+ } else {
+ decrRefCount(o);
+ }
+ }
+
+ if (c->argv != argv) {
+ zfree(c->argv);
+ argv = NULL;
+ argv_size = 0;
+ }
+
+ c->user = NULL;
+ c->argv = NULL;
+ c->argc = 0;
+
+ if (raise_error) {
+ /* If we are here we should have an error in the stack, in the
+ * form of a table with an "err" field. Extract the string to
+ * return the plain error. */
+ inuse--;
+ return luaRaiseError(lua);
+ }
+ inuse--;
+ return 1;
+}
+
+/* redis.call() */
+static int luaRedisCallCommand(lua_State *lua) {
+ return luaRedisGenericCommand(lua,1);
+}
+
+/* redis.pcall() */
+static int luaRedisPCallCommand(lua_State *lua) {
+ return luaRedisGenericCommand(lua,0);
+}
+
+/* This adds redis.sha1hex(string) to Lua scripts using the same hashing
+ * function used for sha1ing lua scripts. */
+static int luaRedisSha1hexCommand(lua_State *lua) {
+ int argc = lua_gettop(lua);
+ char digest[41];
+ size_t len;
+ char *s;
+
+ if (argc != 1) {
+ lua_pushstring(lua, "wrong number of arguments");
+ return lua_error(lua);
+ }
+
+ s = (char*)lua_tolstring(lua,1,&len);
+ sha1hex(digest,s,len);
+ lua_pushstring(lua,digest);
+ return 1;
+}
+
+/* Returns a table with a single field 'field' set to the string value
+ * passed as argument. This helper function is handy when returning
+ * a Redis Protocol error or status reply from Lua:
+ *
+ * return redis.error_reply("ERR Some Error")
+ * return redis.status_reply("ERR Some Error")
+ */
+static int luaRedisReturnSingleFieldTable(lua_State *lua, char *field) {
+ if (lua_gettop(lua) != 1 || lua_type(lua,-1) != LUA_TSTRING) {
+ luaPushError(lua, "wrong number or type of arguments");
+ return 1;
+ }
+
+ lua_newtable(lua);
+ lua_pushstring(lua, field);
+ lua_pushvalue(lua, -3);
+ lua_settable(lua, -3);
+ return 1;
+}
+
+/* redis.error_reply() */
+static int luaRedisErrorReplyCommand(lua_State *lua) {
+ return luaRedisReturnSingleFieldTable(lua,"err");
+}
+
+/* redis.status_reply() */
+static int luaRedisStatusReplyCommand(lua_State *lua) {
+ return luaRedisReturnSingleFieldTable(lua,"ok");
+}
+
+/* redis.set_repl()
+ *
+ * Set the propagation of write commands executed in the context of the
+ * script to on/off for AOF and slaves. */
+static int luaRedisSetReplCommand(lua_State *lua) {
+ int argc = lua_gettop(lua);
+ int flags;
+
+ scriptRunCtx* rctx = luaGetFromRegistry(lua, REGISTRY_RUN_CTX_NAME);
+
+ if (rctx->flags & SCRIPT_EVAL_REPLICATION) {
+ lua_pushstring(lua, "You can set the replication behavior only after turning on single commands replication with redis.replicate_commands().");
+ return lua_error(lua);
+ } else if (argc != 1) {
+ lua_pushstring(lua, "redis.set_repl() requires two arguments.");
+ return lua_error(lua);
+ }
+
+ flags = lua_tonumber(lua,-1);
+ if ((flags & ~(PROPAGATE_AOF|PROPAGATE_REPL)) != 0) {
+ lua_pushstring(lua, "Invalid replication flags. Use REPL_AOF, REPL_REPLICA, REPL_ALL or REPL_NONE.");
+ return lua_error(lua);
+ }
+
+ scriptSetRepl(rctx, flags);
+ return 0;
+}
+
+/* redis.log() */
+static int luaLogCommand(lua_State *lua) {
+ int j, argc = lua_gettop(lua);
+ int level;
+ sds log;
+
+ if (argc < 2) {
+ lua_pushstring(lua, "redis.log() requires two arguments or more.");
+ return lua_error(lua);
+ } else if (!lua_isnumber(lua,-argc)) {
+ lua_pushstring(lua, "First argument must be a number (log level).");
+ return lua_error(lua);
+ }
+ level = lua_tonumber(lua,-argc);
+ if (level < LL_DEBUG || level > LL_WARNING) {
+ lua_pushstring(lua, "Invalid debug level.");
+ return lua_error(lua);
+ }
+ if (level < server.verbosity) return 0;
+
+ /* Glue together all the arguments */
+ log = sdsempty();
+ for (j = 1; j < argc; j++) {
+ size_t len;
+ char *s;
+
+ s = (char*)lua_tolstring(lua,(-argc)+j,&len);
+ if (s) {
+ if (j != 1) log = sdscatlen(log," ",1);
+ log = sdscatlen(log,s,len);
+ }
+ }
+ serverLogRaw(level,log);
+ sdsfree(log);
+ return 0;
+}
+
+/* redis.setresp() */
+static int luaSetResp(lua_State *lua) {
+ int argc = lua_gettop(lua);
+
+ if (argc != 1) {
+ lua_pushstring(lua, "redis.setresp() requires one argument.");
+ return lua_error(lua);
+ }
+
+ int resp = lua_tonumber(lua,-argc);
+ if (resp != 2 && resp != 3) {
+ lua_pushstring(lua, "RESP version must be 2 or 3.");
+ return lua_error(lua);
+ }
+ scriptRunCtx* rctx = luaGetFromRegistry(lua, REGISTRY_RUN_CTX_NAME);
+ scriptSetResp(rctx, resp);
+ return 0;
+}
+
+/* ---------------------------------------------------------------------------
+ * Lua engine initialization and reset.
+ * ------------------------------------------------------------------------- */
+
+static void luaLoadLib(lua_State *lua, const char *libname, lua_CFunction luafunc) {
+ lua_pushcfunction(lua, luafunc);
+ lua_pushstring(lua, libname);
+ lua_call(lua, 1, 0);
+}
+
+LUALIB_API int (luaopen_cjson) (lua_State *L);
+LUALIB_API int (luaopen_struct) (lua_State *L);
+LUALIB_API int (luaopen_cmsgpack) (lua_State *L);
+LUALIB_API int (luaopen_bit) (lua_State *L);
+
+static void luaLoadLibraries(lua_State *lua) {
+ luaLoadLib(lua, "", luaopen_base);
+ luaLoadLib(lua, LUA_TABLIBNAME, luaopen_table);
+ luaLoadLib(lua, LUA_STRLIBNAME, luaopen_string);
+ luaLoadLib(lua, LUA_MATHLIBNAME, luaopen_math);
+ luaLoadLib(lua, LUA_DBLIBNAME, luaopen_debug);
+ luaLoadLib(lua, "cjson", luaopen_cjson);
+ luaLoadLib(lua, "struct", luaopen_struct);
+ luaLoadLib(lua, "cmsgpack", luaopen_cmsgpack);
+ luaLoadLib(lua, "bit", luaopen_bit);
+
+#if 0 /* Stuff that we don't load currently, for sandboxing concerns. */
+ luaLoadLib(lua, LUA_LOADLIBNAME, luaopen_package);
+ luaLoadLib(lua, LUA_OSLIBNAME, luaopen_os);
+#endif
+}
+
+/* Remove a functions that we don't want to expose to the Redis scripting
+ * environment. */
+static void luaRemoveUnsupportedFunctions(lua_State *lua) {
+ lua_pushnil(lua);
+ lua_setglobal(lua,"loadfile");
+ lua_pushnil(lua);
+ lua_setglobal(lua,"dofile");
+}
+
+/* This function installs metamethods in the global table _G that prevent
+ * the creation of globals accidentally.
+ *
+ * It should be the last to be called in the scripting engine initialization
+ * sequence, because it may interact with creation of globals.
+ *
+ * On Legacy Lua (eval) we need to check 'w ~= \"main\"' otherwise we will not be able
+ * to create the global 'function <sha> ()' variable. On Lua engine we do not use this trick
+ * so its not needed. */
+void luaEnableGlobalsProtection(lua_State *lua, int is_eval) {
+ char *s[32];
+ sds code = sdsempty();
+ int j = 0;
+
+ /* strict.lua from: http://metalua.luaforge.net/src/lib/strict.lua.html.
+ * Modified to be adapted to Redis. */
+ s[j++]="local dbg=debug\n";
+ s[j++]="local mt = {}\n";
+ s[j++]="setmetatable(_G, mt)\n";
+ s[j++]="mt.__newindex = function (t, n, v)\n";
+ s[j++]=" if dbg.getinfo(2) then\n";
+ s[j++]=" local w = dbg.getinfo(2, \"S\").what\n";
+ s[j++]= is_eval ? " if w ~= \"main\" and w ~= \"C\" then\n" : " if w ~= \"C\" then\n";
+ s[j++]=" error(\"Script attempted to create global variable '\"..tostring(n)..\"'\", 2)\n";
+ s[j++]=" end\n";
+ s[j++]=" end\n";
+ s[j++]=" rawset(t, n, v)\n";
+ s[j++]="end\n";
+ s[j++]="mt.__index = function (t, n)\n";
+ s[j++]=" if dbg.getinfo(2) and dbg.getinfo(2, \"S\").what ~= \"C\" then\n";
+ s[j++]=" error(\"Script attempted to access nonexistent global variable '\"..tostring(n)..\"'\", 2)\n";
+ s[j++]=" end\n";
+ s[j++]=" return rawget(t, n)\n";
+ s[j++]="end\n";
+ s[j++]="debug = nil\n";
+ s[j++]=NULL;
+
+ for (j = 0; s[j] != NULL; j++) code = sdscatlen(code,s[j],strlen(s[j]));
+ luaL_loadbuffer(lua,code,sdslen(code),"@enable_strict_lua");
+ lua_pcall(lua,0,0,0);
+ sdsfree(code);
+}
+
+void luaRegisterRedisAPI(lua_State* lua) {
+ luaLoadLibraries(lua);
+ luaRemoveUnsupportedFunctions(lua);
+
+ /* Register the redis commands table and fields */
+ lua_newtable(lua);
+
+ /* redis.call */
+ lua_pushstring(lua,"call");
+ lua_pushcfunction(lua,luaRedisCallCommand);
+ lua_settable(lua,-3);
+
+ /* redis.pcall */
+ lua_pushstring(lua,"pcall");
+ lua_pushcfunction(lua,luaRedisPCallCommand);
+ lua_settable(lua,-3);
+
+ /* redis.log and log levels. */
+ lua_pushstring(lua,"log");
+ lua_pushcfunction(lua,luaLogCommand);
+ lua_settable(lua,-3);
+
+ /* redis.setresp */
+ lua_pushstring(lua,"setresp");
+ lua_pushcfunction(lua,luaSetResp);
+ lua_settable(lua,-3);
+
+ lua_pushstring(lua,"LOG_DEBUG");
+ lua_pushnumber(lua,LL_DEBUG);
+ lua_settable(lua,-3);
+
+ lua_pushstring(lua,"LOG_VERBOSE");
+ lua_pushnumber(lua,LL_VERBOSE);
+ lua_settable(lua,-3);
+
+ lua_pushstring(lua,"LOG_NOTICE");
+ lua_pushnumber(lua,LL_NOTICE);
+ lua_settable(lua,-3);
+
+ lua_pushstring(lua,"LOG_WARNING");
+ lua_pushnumber(lua,LL_WARNING);
+ lua_settable(lua,-3);
+
+ /* redis.sha1hex */
+ lua_pushstring(lua, "sha1hex");
+ lua_pushcfunction(lua, luaRedisSha1hexCommand);
+ lua_settable(lua, -3);
+
+ /* redis.error_reply and redis.status_reply */
+ lua_pushstring(lua, "error_reply");
+ lua_pushcfunction(lua, luaRedisErrorReplyCommand);
+ lua_settable(lua, -3);
+ lua_pushstring(lua, "status_reply");
+ lua_pushcfunction(lua, luaRedisStatusReplyCommand);
+ lua_settable(lua, -3);
+
+ /* redis.set_repl and associated flags. */
+ lua_pushstring(lua,"set_repl");
+ lua_pushcfunction(lua,luaRedisSetReplCommand);
+ lua_settable(lua,-3);
+
+ lua_pushstring(lua,"REPL_NONE");
+ lua_pushnumber(lua,PROPAGATE_NONE);
+ lua_settable(lua,-3);
+
+ lua_pushstring(lua,"REPL_AOF");
+ lua_pushnumber(lua,PROPAGATE_AOF);
+ lua_settable(lua,-3);
+
+ lua_pushstring(lua,"REPL_SLAVE");
+ lua_pushnumber(lua,PROPAGATE_REPL);
+ lua_settable(lua,-3);
+
+ lua_pushstring(lua,"REPL_REPLICA");
+ lua_pushnumber(lua,PROPAGATE_REPL);
+ lua_settable(lua,-3);
+
+ lua_pushstring(lua,"REPL_ALL");
+ lua_pushnumber(lua,PROPAGATE_AOF|PROPAGATE_REPL);
+
+ lua_settable(lua,-3);
+ /* Finally set the table as 'redis' global var. */
+ lua_setglobal(lua,"redis");
+
+ /* Replace math.random and math.randomseed with our implementations. */
+ lua_getglobal(lua,"math");
+
+ lua_pushstring(lua,"random");
+ lua_pushcfunction(lua,redis_math_random);
+ lua_settable(lua,-3);
+
+ lua_pushstring(lua,"randomseed");
+ lua_pushcfunction(lua,redis_math_randomseed);
+ lua_settable(lua,-3);
+
+ lua_setglobal(lua,"math");
+}
+
+/* Set an array of Redis String Objects as a Lua array (table) stored into a
+ * global variable. */
+static void luaSetGlobalArray(lua_State *lua, char *var, robj **elev, int elec) {
+ int j;
+
+ lua_newtable(lua);
+ for (j = 0; j < elec; j++) {
+ lua_pushlstring(lua,(char*)elev[j]->ptr,sdslen(elev[j]->ptr));
+ lua_rawseti(lua,-2,j+1);
+ }
+ lua_setglobal(lua,var);
+}
+
+/* ---------------------------------------------------------------------------
+ * Redis provided math.random
+ * ------------------------------------------------------------------------- */
+
+/* We replace math.random() with our implementation that is not affected
+ * by specific libc random() implementations and will output the same sequence
+ * (for the same seed) in every arch. */
+
+/* The following implementation is the one shipped with Lua itself but with
+ * rand() replaced by redisLrand48(). */
+static int redis_math_random (lua_State *L) {
+ /* the `%' avoids the (rare) case of r==1, and is needed also because on
+ some systems (SunOS!) `rand()' may return a value larger than RAND_MAX */
+ lua_Number r = (lua_Number)(redisLrand48()%REDIS_LRAND48_MAX) /
+ (lua_Number)REDIS_LRAND48_MAX;
+ switch (lua_gettop(L)) { /* check number of arguments */
+ case 0: { /* no arguments */
+ lua_pushnumber(L, r); /* Number between 0 and 1 */
+ break;
+ }
+ case 1: { /* only upper limit */
+ int u = luaL_checkint(L, 1);
+ luaL_argcheck(L, 1<=u, 1, "interval is empty");
+ lua_pushnumber(L, floor(r*u)+1); /* int between 1 and `u' */
+ break;
+ }
+ case 2: { /* lower and upper limits */
+ int l = luaL_checkint(L, 1);
+ int u = luaL_checkint(L, 2);
+ luaL_argcheck(L, l<=u, 2, "interval is empty");
+ lua_pushnumber(L, floor(r*(u-l+1))+l); /* int between `l' and `u' */
+ break;
+ }
+ default: return luaL_error(L, "wrong number of arguments");
+ }
+ return 1;
+}
+
+static int redis_math_randomseed (lua_State *L) {
+ redisSrand48(luaL_checkint(L, 1));
+ return 0;
+}
+
+/* This is the Lua script "count" hook that we use to detect scripts timeout. */
+static void luaMaskCountHook(lua_State *lua, lua_Debug *ar) {
+ UNUSED(ar);
+ scriptRunCtx* rctx = luaGetFromRegistry(lua, REGISTRY_RUN_CTX_NAME);
+ if (scriptInterrupt(rctx) == SCRIPT_KILL) {
+ serverLog(LL_WARNING,"Lua script killed by user with SCRIPT KILL.");
+
+ /*
+ * Set the hook to invoke all the time so the user
+         * will not be able to catch the error with pcall and invoke
+         * pcall again which will prevent the script from ever been killed
+ */
+ lua_sethook(lua, luaMaskCountHook, LUA_MASKLINE, 0);
+
+ lua_pushstring(lua,"Script killed by user with SCRIPT KILL...");
+ lua_error(lua);
+ }
+}
+
+void luaCallFunction(scriptRunCtx* run_ctx, lua_State *lua, robj** keys, size_t nkeys, robj** args, size_t nargs, int debug_enabled) {
+ client* c = run_ctx->original_client;
+ int delhook = 0;
+
+ /* We must set it before we set the Lua hook, theoretically the
+ * Lua hook might be called wheneven we run any Lua instruction
+ * such as 'luaSetGlobalArray' and we want the run_ctx to be available
+ * each time the Lua hook is invoked. */
+ luaSaveOnRegistry(lua, REGISTRY_RUN_CTX_NAME, run_ctx);
+
+ if (server.script_time_limit > 0 && !debug_enabled) {
+ lua_sethook(lua,luaMaskCountHook,LUA_MASKCOUNT,100000);
+ delhook = 1;
+ } else if (debug_enabled) {
+ lua_sethook(lua,luaLdbLineHook,LUA_MASKLINE|LUA_MASKCOUNT,100000);
+ delhook = 1;
+ }
+
+ /* Populate the argv and keys table accordingly to the arguments that
+ * EVAL received. */
+ luaSetGlobalArray(lua,"KEYS",keys,nkeys);
+ luaSetGlobalArray(lua,"ARGV",args,nargs);
+
+ /* At this point whether this script was never seen before or if it was
+ * already defined, we can call it. We have zero arguments and expect
+ * a single return value. */
+ int err = lua_pcall(lua,0,1,-2);
+
+ /* Call the Lua garbage collector from time to time to avoid a
+ * full cycle performed by Lua, which adds too latency.
+ *
+ * The call is performed every LUA_GC_CYCLE_PERIOD executed commands
+ * (and for LUA_GC_CYCLE_PERIOD collection steps) because calling it
+ * for every command uses too much CPU. */
+ #define LUA_GC_CYCLE_PERIOD 50
+ {
+ static long gc_count = 0;
+
+ gc_count++;
+ if (gc_count == LUA_GC_CYCLE_PERIOD) {
+ lua_gc(lua,LUA_GCSTEP,LUA_GC_CYCLE_PERIOD);
+ gc_count = 0;
+ }
+ }
+
+ if (err) {
+ addReplyErrorFormat(c,"Error running script (call to %s): %s\n",
+ run_ctx->funcname, lua_tostring(lua,-1));
+ lua_pop(lua,1); /* Consume the Lua reply and remove error handler. */
+ } else {
+ /* On success convert the Lua return value into Redis protocol, and
+ * send it to * the client. */
+ luaReplyToRedisReply(c, run_ctx->c, lua); /* Convert and consume the reply. */
+ }
+
+ /* Perform some cleanup that we need to do both on error and success. */
+ if (delhook) lua_sethook(lua,NULL,0,0); /* Disable hook */
+
+ /* remove run_ctx from registry, its only applicable for the current script. */
+ luaSaveOnRegistry(lua, REGISTRY_RUN_CTX_NAME, NULL);
+}
+
+unsigned long luaMemory(lua_State *lua) {
+ return lua_gc(lua, LUA_GCCOUNT, 0) * 1024LL;
+}
diff --git a/src/script_lua.h b/src/script_lua.h
new file mode 100644
index 000000000..40cdd00b5
--- /dev/null
+++ b/src/script_lua.h
@@ -0,0 +1,67 @@
+/*
+ * Copyright (c) 2009-2021, Redis Ltd.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions are met:
+ *
+ * * Redistributions of source code must retain the above copyright notice,
+ * this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in the
+ * documentation and/or other materials provided with the distribution.
+ * * Neither the name of Redis nor the names of its contributors may be used
+ * to endorse or promote products derived from this software without
+ * specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef __SCRIPT_LUA_H_
+#define __SCRIPT_LUA_H_
+
+/*
+ * script_lua.c unit provides shared functionality between
+ * eval.c and function_lua.c. Functionality provided:
+ *
+ * * Execute Lua code, assuming that the code is located on
+ * the top of the Lua stack. In addition, parsing the execution
+ * result and convert it to the resp and reply ot the client.
+ *
+ * * Run Redis commands from within the Lua code (Including
+ * parsing the reply and create a Lua object out of it).
+ *
+ * * Register Redis API to the Lua interpreter. Only shared
+ * API are registered (API that is only relevant on eval.c
+ * (like debugging) are registered on eval.c).
+ *
+ * Uses script.c for interaction back with Redis.
+ */
+
+#include "server.h"
+#include "script.h"
+#include <lua.h>
+#include <lauxlib.h>
+#include <lualib.h>
+
+#define REGISTRY_RUN_CTX_NAME "__RUN_CTX__"
+
+void luaRegisterRedisAPI(lua_State* lua);
+void luaEnableGlobalsProtection(lua_State *lua, int is_eval);
+void luaSaveOnRegistry(lua_State* lua, const char* name, void* ptr);
+void* luaGetFromRegistry(lua_State* lua, const char* name);
+void luaCallFunction(scriptRunCtx* r_ctx, lua_State *lua, robj** keys, size_t nkeys, robj** args, size_t nargs, int debug_enabled);
+unsigned long luaMemory(lua_State *lua);
+
+
+#endif /* __SCRIPT_LUA_H_ */
diff --git a/src/scripting.c b/src/scripting.c
deleted file mode 100644
index f27840956..000000000
--- a/src/scripting.c
+++ /dev/null
@@ -1,3063 +0,0 @@
-/*
- * Copyright (c) 2009-2012, Salvatore Sanfilippo <antirez at gmail dot com>
- * All rights reserved.
- *
- * Redistribution and use in source and binary forms, with or without
- * modification, are permitted provided that the following conditions are met:
- *
- * * Redistributions of source code must retain the above copyright notice,
- * this list of conditions and the following disclaimer.
- * * Redistributions in binary form must reproduce the above copyright
- * notice, this list of conditions and the following disclaimer in the
- * documentation and/or other materials provided with the distribution.
- * * Neither the name of Redis nor the names of its contributors may be used
- * to endorse or promote products derived from this software without
- * specific prior written permission.
- *
- * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
- * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
- * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
- * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
- * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
- * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
- * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
- * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
- * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
- * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
- * POSSIBILITY OF SUCH DAMAGE.
- */
-
-#include "server.h"
-#include "sha1.h"
-#include "rand.h"
-#include "cluster.h"
-#include "monotonic.h"
-#include "resp_parser.h"
-
-#include <lua.h>
-#include <lauxlib.h>
-#include <lualib.h>
-#include <ctype.h>
-#include <math.h>
-
-static void redisProtocolToLuaType_Int(void *ctx, long long val, const char *proto, size_t proto_len);
-static void redisProtocolToLuaType_BulkString(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len);
-static void redisProtocolToLuaType_NullBulkString(void *ctx, const char *proto, size_t proto_len);
-static void redisProtocolToLuaType_NullArray(void *ctx, const char *proto, size_t proto_len);
-static void redisProtocolToLuaType_Status(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len);
-static void redisProtocolToLuaType_Error(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len);
-static void redisProtocolToLuaType_Array(struct ReplyParser *parser, void *ctx, size_t len, const char *proto);
-static void redisProtocolToLuaType_Map(struct ReplyParser *parser, void *ctx, size_t len, const char *proto);
-static void redisProtocolToLuaType_Set(struct ReplyParser *parser, void *ctx, size_t len, const char *proto);
-static void redisProtocolToLuaType_Null(void *ctx, const char *proto, size_t proto_len);
-static void redisProtocolToLuaType_Bool(void *ctx, int val, const char *proto, size_t proto_len);
-static void redisProtocolToLuaType_Double(void *ctx, double d, const char *proto, size_t proto_len);
-static void redisProtocolToLuaType_BigNumber(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len);
-static void redisProtocolToLuaType_VerbatimString(void *ctx, const char *format, const char *str, size_t len, const char *proto, size_t proto_len);
-static void redisProtocolToLuaType_Attribute(struct ReplyParser *parser, void *ctx, size_t len, const char *proto);
-int redis_math_random (lua_State *L);
-int redis_math_randomseed (lua_State *L);
-void ldbInit(void);
-void ldbDisable(client *c);
-void ldbEnable(client *c);
-void evalGenericCommandWithDebugging(client *c, int evalsha);
-void luaLdbLineHook(lua_State *lua, lua_Debug *ar);
-void ldbLog(sds entry);
-void ldbLogRedisReply(char *reply);
-sds ldbCatStackValue(sds s, lua_State *lua, int idx);
-
-/* Debugger shared state is stored inside this global structure. */
-#define LDB_BREAKPOINTS_MAX 64 /* Max number of breakpoints. */
-#define LDB_MAX_LEN_DEFAULT 256 /* Default len limit for replies / var dumps. */
-struct ldbState {
- connection *conn; /* Connection of the debugging client. */
- int active; /* Are we debugging EVAL right now? */
- int forked; /* Is this a fork()ed debugging session? */
- list *logs; /* List of messages to send to the client. */
- list *traces; /* Messages about Redis commands executed since last stop.*/
- list *children; /* All forked debugging sessions pids. */
- int bp[LDB_BREAKPOINTS_MAX]; /* An array of breakpoints line numbers. */
- int bpcount; /* Number of valid entries inside bp. */
- int step; /* Stop at next line regardless of breakpoints. */
- int luabp; /* Stop at next line because redis.breakpoint() was called. */
- sds *src; /* Lua script source code split by line. */
- int lines; /* Number of lines in 'src'. */
- int currentline; /* Current line number. */
- sds cbuf; /* Debugger client command buffer. */
- size_t maxlen; /* Max var dump / reply length. */
- int maxlen_hint_sent; /* Did we already hint about "set maxlen"? */
-} ldb;
-
-/* ---------------------------------------------------------------------------
- * Utility functions.
- * ------------------------------------------------------------------------- */
-
-/* Perform the SHA1 of the input string. We use this both for hashing script
- * bodies in order to obtain the Lua function name, and in the implementation
- * of redis.sha1().
- *
- * 'digest' should point to a 41 bytes buffer: 40 for SHA1 converted into an
- * hexadecimal number, plus 1 byte for null term. */
-void sha1hex(char *digest, char *script, size_t len) {
- SHA1_CTX ctx;
- unsigned char hash[20];
- char *cset = "0123456789abcdef";
- int j;
-
- SHA1Init(&ctx);
- SHA1Update(&ctx,(unsigned char*)script,len);
- SHA1Final(hash,&ctx);
-
- for (j = 0; j < 20; j++) {
- digest[j*2] = cset[((hash[j]&0xF0)>>4)];
- digest[j*2+1] = cset[(hash[j]&0xF)];
- }
- digest[40] = '\0';
-}
-
-/* ---------------------------------------------------------------------------
- * Redis reply to Lua type conversion functions.
- * ------------------------------------------------------------------------- */
-
-/* Take a Redis reply in the Redis protocol format and convert it into a
- * Lua type. Thanks to this function, and the introduction of not connected
- * clients, it is trivial to implement the redis() lua function.
- *
- * Basically we take the arguments, execute the Redis command in the context
- * of a non connected client, then take the generated reply and convert it
- * into a suitable Lua type. With this trick the scripting feature does not
- * need the introduction of a full Redis internals API. The script
- * is like a normal client that bypasses all the slow I/O paths.
- *
- * Note: in this function we do not do any sanity check as the reply is
- * generated by Redis directly. This allows us to go faster.
- *
- * Errors are returned as a table with a single 'err' field set to the
- * error string.
- */
-
-static const ReplyParserCallbacks DefaultLuaTypeParserCallbacks = {
- .null_array_callback = redisProtocolToLuaType_NullArray,
- .bulk_string_callback = redisProtocolToLuaType_BulkString,
- .null_bulk_string_callback = redisProtocolToLuaType_NullBulkString,
- .error_callback = redisProtocolToLuaType_Error,
- .simple_str_callback = redisProtocolToLuaType_Status,
- .long_callback = redisProtocolToLuaType_Int,
- .array_callback = redisProtocolToLuaType_Array,
- .set_callback = redisProtocolToLuaType_Set,
- .map_callback = redisProtocolToLuaType_Map,
- .bool_callback = redisProtocolToLuaType_Bool,
- .double_callback = redisProtocolToLuaType_Double,
- .null_callback = redisProtocolToLuaType_Null,
- .big_number_callback = redisProtocolToLuaType_BigNumber,
- .verbatim_string_callback = redisProtocolToLuaType_VerbatimString,
- .attribute_callback = redisProtocolToLuaType_Attribute,
- .error = NULL,
-};
-
-void redisProtocolToLuaType(lua_State *lua, char* reply) {
- ReplyParser parser = {.curr_location = reply, .callbacks = DefaultLuaTypeParserCallbacks};
-
- parseReply(&parser, lua);
-}
-
-static void redisProtocolToLuaType_Int(void *ctx, long long val, const char *proto, size_t proto_len) {
- UNUSED(proto);
- UNUSED(proto_len);
- if (!ctx) {
- return;
- }
-
- lua_State *lua = ctx;
- if (!lua_checkstack(lua, 1)) {
- /* Increase the Lua stack if needed, to make sure there is enough room
- * to push elements to the stack. On failure, exit with panic. */
- serverPanic("lua stack limit reach when parsing redis.call reply");
- }
- lua_pushnumber(lua,(lua_Number)val);
-}
-
-static void redisProtocolToLuaType_NullBulkString(void *ctx, const char *proto, size_t proto_len) {
- UNUSED(proto);
- UNUSED(proto_len);
- if (!ctx) {
- return;
- }
-
- lua_State *lua = ctx;
- if (!lua_checkstack(lua, 1)) {
- /* Increase the Lua stack if needed, to make sure there is enough room
- * to push elements to the stack. On failure, exit with panic. */
- serverPanic("lua stack limit reach when parsing redis.call reply");
- }
- lua_pushboolean(lua,0);
-}
-
-static void redisProtocolToLuaType_NullArray(void *ctx, const char *proto, size_t proto_len) {
- UNUSED(proto);
- UNUSED(proto_len);
- if (!ctx) {
- return;
- }
- lua_State *lua = ctx;
- if (!lua_checkstack(lua, 1)) {
- /* Increase the Lua stack if needed, to make sure there is enough room
- * to push elements to the stack. On failure, exit with panic. */
- serverPanic("lua stack limit reach when parsing redis.call reply");
- }
- lua_pushboolean(lua,0);
-}
-
-
-static void redisProtocolToLuaType_BulkString(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len) {
- UNUSED(proto);
- UNUSED(proto_len);
- if (!ctx) {
- return;
- }
-
- lua_State *lua = ctx;
- if (!lua_checkstack(lua, 1)) {
- /* Increase the Lua stack if needed, to make sure there is enough room
- * to push elements to the stack. On failure, exit with panic. */
- serverPanic("lua stack limit reach when parsing redis.call reply");
- }
- lua_pushlstring(lua,str,len);
-}
-
-static void redisProtocolToLuaType_Status(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len) {
- UNUSED(proto);
- UNUSED(proto_len);
- if (!ctx) {
- return;
- }
-
- lua_State *lua = ctx;
- if (!lua_checkstack(lua, 3)) {
- /* Increase the Lua stack if needed, to make sure there is enough room
- * to push elements to the stack. On failure, exit with panic. */
- serverPanic("lua stack limit reach when parsing redis.call reply");
- }
- lua_newtable(lua);
- lua_pushstring(lua,"ok");
- lua_pushlstring(lua,str,len);
- lua_settable(lua,-3);
-}
-
-static void redisProtocolToLuaType_Error(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len) {
- UNUSED(proto);
- UNUSED(proto_len);
- if (!ctx) {
- return;
- }
-
- lua_State *lua = ctx;
- if (!lua_checkstack(lua, 3)) {
- /* Increase the Lua stack if needed, to make sure there is enough room
- * to push elements to the stack. On failure, exit with panic. */
- serverPanic("lua stack limit reach when parsing redis.call reply");
- }
- lua_newtable(lua);
- lua_pushstring(lua,"err");
- lua_pushlstring(lua,str,len);
- lua_settable(lua,-3);
-}
-
-static void redisProtocolToLuaType_Map(struct ReplyParser *parser, void *ctx, size_t len, const char *proto) {
- UNUSED(proto);
- lua_State *lua = ctx;
- if (lua) {
- if (!lua_checkstack(lua, 3)) {
- /* Increase the Lua stack if needed, to make sure there is enough room
- * to push elements to the stack. On failure, exit with panic. */
- serverPanic("lua stack limit reach when parsing redis.call reply");
- }
- lua_newtable(lua);
- lua_pushstring(lua, "map");
- lua_newtable(lua);
- }
- for (size_t j = 0; j < len; j++) {
- parseReply(parser,lua);
- parseReply(parser,lua);
- if (lua) lua_settable(lua,-3);
- }
- if (lua) lua_settable(lua,-3);
-}
-
-static void redisProtocolToLuaType_Set(struct ReplyParser *parser, void *ctx, size_t len, const char *proto) {
- UNUSED(proto);
-
- lua_State *lua = ctx;
- if (lua) {
- if (!lua_checkstack(lua, 3)) {
- /* Increase the Lua stack if needed, to make sure there is enough room
- * to push elements to the stack. On failure, exit with panic. */
- serverPanic("lua stack limit reach when parsing redis.call reply");
- }
- lua_newtable(lua);
- lua_pushstring(lua, "set");
- lua_newtable(lua);
- }
- for (size_t j = 0; j < len; j++) {
- parseReply(parser,lua);
- if (lua) {
- if (!lua_checkstack(lua, 1)) {
- /* Increase the Lua stack if needed, to make sure there is enough room
- * to push elements to the stack. On failure, exit with panic.
- * Notice that here we need to check the stack again because the recursive
- * call to redisProtocolToLuaType might have use the room allocated in the stack*/
- serverPanic("lua stack limit reach when parsing redis.call reply");
- }
- lua_pushboolean(lua,1);
- lua_settable(lua,-3);
- }
- }
- if (lua) lua_settable(lua,-3);
-}
-
-static void redisProtocolToLuaType_Array(struct ReplyParser *parser, void *ctx, size_t len, const char *proto) {
- UNUSED(proto);
-
- lua_State *lua = ctx;
- if (lua){
- if (!lua_checkstack(lua, 2)) {
- /* Increase the Lua stack if needed, to make sure there is enough room
- * to push elements to the stack. On failure, exit with panic. */
- serverPanic("lua stack limit reach when parsing redis.call reply");
- }
- lua_newtable(lua);
- }
- for (size_t j = 0; j < len; j++) {
- if (lua) lua_pushnumber(lua,j+1);
- parseReply(parser,lua);
- if (lua) lua_settable(lua,-3);
- }
-}
-
-static void redisProtocolToLuaType_Attribute(struct ReplyParser *parser, void *ctx, size_t len, const char *proto) {
- UNUSED(proto);
-
- /* Parse the attribute reply.
- * Currently, we do not expose the attribute to the Lua script so
- * we just need to continue parsing and ignore it (the NULL ensures that the
- * reply will be ignored). */
- for (size_t j = 0; j < len; j++) {
- parseReply(parser,NULL);
- parseReply(parser,NULL);
- }
-
- /* Parse the reply itself. */
- parseReply(parser,ctx);
-}
-
-static void redisProtocolToLuaType_VerbatimString(void *ctx, const char *format, const char *str, size_t len, const char *proto, size_t proto_len) {
- UNUSED(proto);
- UNUSED(proto_len);
- if (!ctx) {
- return;
- }
-
- lua_State *lua = ctx;
- if (!lua_checkstack(lua, 5)) {
- /* Increase the Lua stack if needed, to make sure there is enough room
- * to push elements to the stack. On failure, exit with panic. */
- serverPanic("lua stack limit reach when parsing redis.call reply");
- }
- lua_newtable(lua);
- lua_pushstring(lua,"verbatim_string");
- lua_newtable(lua);
- lua_pushstring(lua,"string");
- lua_pushlstring(lua,str,len);
- lua_settable(lua,-3);
- lua_pushstring(lua,"format");
- lua_pushlstring(lua,format,3);
- lua_settable(lua,-3);
- lua_settable(lua,-3);
-}
-
-static void redisProtocolToLuaType_BigNumber(void *ctx, const char *str, size_t len, const char *proto, size_t proto_len) {
- UNUSED(proto);
- UNUSED(proto_len);
- if (!ctx) {
- return;
- }
-
- lua_State *lua = ctx;
- if (!lua_checkstack(lua, 3)) {
- /* Increase the Lua stack if needed, to make sure there is enough room
- * to push elements to the stack. On failure, exit with panic. */
- serverPanic("lua stack limit reach when parsing redis.call reply");
- }
- lua_newtable(lua);
- lua_pushstring(lua,"big_number");
- lua_pushlstring(lua,str,len);
- lua_settable(lua,-3);
-}
-
-static void redisProtocolToLuaType_Null(void *ctx, const char *proto, size_t proto_len) {
- UNUSED(proto);
- UNUSED(proto_len);
- if (!ctx) {
- return;
- }
-
- lua_State *lua = ctx;
- if (!lua_checkstack(lua, 1)) {
- /* Increase the Lua stack if needed, to make sure there is enough room
- * to push elements to the stack. On failure, exit with panic. */
- serverPanic("lua stack limit reach when parsing redis.call reply");
- }
- lua_pushnil(lua);
-}
-
-static void redisProtocolToLuaType_Bool(void *ctx, int val, const char *proto, size_t proto_len) {
- UNUSED(proto);
- UNUSED(proto_len);
- if (!ctx) {
- return;
- }
-
- lua_State *lua = ctx;
- if (!lua_checkstack(lua, 1)) {
- /* Increase the Lua stack if needed, to make sure there is enough room
- * to push elements to the stack. On failure, exit with panic. */
- serverPanic("lua stack limit reach when parsing redis.call reply");
- }
- lua_pushboolean(lua,val);
-}
-
-static void redisProtocolToLuaType_Double(void *ctx, double d, const char *proto, size_t proto_len) {
- UNUSED(proto);
- UNUSED(proto_len);
- if (!ctx) {
- return;
- }
-
- lua_State *lua = ctx;
- if (!lua_checkstack(lua, 3)) {
- /* Increase the Lua stack if needed, to make sure there is enough room
- * to push elements to the stack. On failure, exit with panic. */
- serverPanic("lua stack limit reach when parsing redis.call reply");
- }
- lua_newtable(lua);
- lua_pushstring(lua,"double");
- lua_pushnumber(lua,d);
- lua_settable(lua,-3);
-}
-
-/* This function is used in order to push an error on the Lua stack in the
- * format used by redis.pcall to return errors, which is a lua table
- * with a single "err" field set to the error string. Note that this
- * table is never a valid reply by proper commands, since the returned
- * tables are otherwise always indexed by integers, never by strings. */
-void luaPushError(lua_State *lua, char *error) {
- lua_Debug dbg;
-
- /* If debugging is active and in step mode, log errors resulting from
- * Redis commands. */
- if (ldb.active && ldb.step) {
- ldbLog(sdscatprintf(sdsempty(),"<error> %s",error));
- }
-
- lua_newtable(lua);
- lua_pushstring(lua,"err");
-
- /* Attempt to figure out where this function was called, if possible */
- if(lua_getstack(lua, 1, &dbg) && lua_getinfo(lua, "nSl", &dbg)) {
- sds msg = sdscatprintf(sdsempty(), "%s: %d: %s",
- dbg.source, dbg.currentline, error);
- lua_pushstring(lua, msg);
- sdsfree(msg);
- } else {
- lua_pushstring(lua, error);
- }
- lua_settable(lua,-3);
-}
-
-/* In case the error set into the Lua stack by luaPushError() was generated
- * by the non-error-trapping version of redis.pcall(), which is redis.call(),
- * this function will raise the Lua error so that the execution of the
- * script will be halted. */
-int luaRaiseError(lua_State *lua) {
- lua_pushstring(lua,"err");
- lua_gettable(lua,-2);
- return lua_error(lua);
-}
-
-/* Sort the array currently in the stack. We do this to make the output
- * of commands like KEYS or SMEMBERS something deterministic when called
- * from Lua (to play well with AOf/replication).
- *
- * The array is sorted using table.sort itself, and assuming all the
- * list elements are strings. */
-void luaSortArray(lua_State *lua) {
- /* Initial Stack: array */
- lua_getglobal(lua,"table");
- lua_pushstring(lua,"sort");
- lua_gettable(lua,-2); /* Stack: array, table, table.sort */
- lua_pushvalue(lua,-3); /* Stack: array, table, table.sort, array */
- if (lua_pcall(lua,1,0,0)) {
- /* Stack: array, table, error */
-
- /* We are not interested in the error, we assume that the problem is
- * that there are 'false' elements inside the array, so we try
- * again with a slower function but able to handle this case, that
- * is: table.sort(table, __redis__compare_helper) */
- lua_pop(lua,1); /* Stack: array, table */
- lua_pushstring(lua,"sort"); /* Stack: array, table, sort */
- lua_gettable(lua,-2); /* Stack: array, table, table.sort */
- lua_pushvalue(lua,-3); /* Stack: array, table, table.sort, array */
- lua_getglobal(lua,"__redis__compare_helper");
- /* Stack: array, table, table.sort, array, __redis__compare_helper */
- lua_call(lua,2,0);
- }
- /* Stack: array (sorted), table */
- lua_pop(lua,1); /* Stack: array (sorted) */
-}
-
-/* ---------------------------------------------------------------------------
- * Lua reply to Redis reply conversion functions.
- * ------------------------------------------------------------------------- */
-
-/* Reply to client 'c' converting the top element in the Lua stack to a
- * Redis reply. As a side effect the element is consumed from the stack. */
-void luaReplyToRedisReply(client *c, lua_State *lua) {
- int t = lua_type(lua,-1);
-
- if (!lua_checkstack(lua, 4)) {
- /* Increase the Lua stack if needed to make sure there is enough room
- * to push 4 elements to the stack. On failure, return error.
- * Notice that we need, in the worst case, 4 elements because returning a map might
- * require push 4 elements to the Lua stack.*/
- addReplyErrorFormat(c, "reached lua stack limit");
- lua_pop(lua,1); /* pop the element from the stack */
- return;
- }
-
- switch(t) {
- case LUA_TSTRING:
- addReplyBulkCBuffer(c,(char*)lua_tostring(lua,-1),lua_strlen(lua,-1));
- break;
- case LUA_TBOOLEAN:
- if (server.lua_client->resp == 2)
- addReply(c,lua_toboolean(lua,-1) ? shared.cone :
- shared.null[c->resp]);
- else
- addReplyBool(c,lua_toboolean(lua,-1));
- break;
- case LUA_TNUMBER:
- addReplyLongLong(c,(long long)lua_tonumber(lua,-1));
- break;
- case LUA_TTABLE:
- /* We need to check if it is an array, an error, or a status reply.
- * Error are returned as a single element table with 'err' field.
- * Status replies are returned as single element table with 'ok'
- * field. */
-
- /* Handle error reply. */
- /* we took care of the stack size on function start */
- lua_pushstring(lua,"err");
- lua_gettable(lua,-2);
- t = lua_type(lua,-1);
- if (t == LUA_TSTRING) {
- addReplyErrorFormat(c,"-%s",lua_tostring(lua,-1));
- lua_pop(lua,2);
- return;
- }
- lua_pop(lua,1); /* Discard field name pushed before. */
-
- /* Handle status reply. */
- lua_pushstring(lua,"ok");
- lua_gettable(lua,-2);
- t = lua_type(lua,-1);
- if (t == LUA_TSTRING) {
- sds ok = sdsnew(lua_tostring(lua,-1));
- sdsmapchars(ok,"\r\n"," ",2);
- addReplySds(c,sdscatprintf(sdsempty(),"+%s\r\n",ok));
- sdsfree(ok);
- lua_pop(lua,2);
- return;
- }
- lua_pop(lua,1); /* Discard field name pushed before. */
-
- /* Handle double reply. */
- lua_pushstring(lua,"double");
- lua_gettable(lua,-2);
- t = lua_type(lua,-1);
- if (t == LUA_TNUMBER) {
- addReplyDouble(c,lua_tonumber(lua,-1));
- lua_pop(lua,2);
- return;
- }
- lua_pop(lua,1); /* Discard field name pushed before. */
-
- /* Handle big number reply. */
- lua_pushstring(lua,"big_number");
- lua_gettable(lua,-2);
- t = lua_type(lua,-1);
- if (t == LUA_TSTRING) {
- sds big_num = sdsnewlen(lua_tostring(lua,-1), lua_strlen(lua,-1));
- sdsmapchars(big_num,"\r\n"," ",2);
- addReplyBigNum(c,big_num,sdslen(big_num));
- sdsfree(big_num);
- lua_pop(lua,2);
- return;
- }
- lua_pop(lua,1); /* Discard field name pushed before. */
-
- /* Handle verbatim reply. */
- lua_pushstring(lua,"verbatim_string");
- lua_gettable(lua,-2);
- t = lua_type(lua,-1);
- if (t == LUA_TTABLE) {
- lua_pushstring(lua,"format");
- lua_gettable(lua,-2);
- t = lua_type(lua,-1);
- if (t == LUA_TSTRING){
- char* format = (char*)lua_tostring(lua,-1);
- lua_pushstring(lua,"string");
- lua_gettable(lua,-3);
- t = lua_type(lua,-1);
- if (t == LUA_TSTRING){
- size_t len;
- char* str = (char*)lua_tolstring(lua,-1,&len);
- addReplyVerbatim(c, str, len, format);
- lua_pop(lua,4);
- return;
- }
- lua_pop(lua,1);
- }
- lua_pop(lua,1);
- }
- lua_pop(lua,1); /* Discard field name pushed before. */
-
- /* Handle map reply. */
- lua_pushstring(lua,"map");
- lua_gettable(lua,-2);
- t = lua_type(lua,-1);
- if (t == LUA_TTABLE) {
- int maplen = 0;
- void *replylen = addReplyDeferredLen(c);
- /* we took care of the stack size on function start */
- lua_pushnil(lua); /* Use nil to start iteration. */
- while (lua_next(lua,-2)) {
- /* Stack now: table, key, value */
- lua_pushvalue(lua,-2); /* Dup key before consuming. */
- luaReplyToRedisReply(c, lua); /* Return key. */
- luaReplyToRedisReply(c, lua); /* Return value. */
- /* Stack now: table, key. */
- maplen++;
- }
- setDeferredMapLen(c,replylen,maplen);
- lua_pop(lua,2);
- return;
- }
- lua_pop(lua,1); /* Discard field name pushed before. */
-
- /* Handle set reply. */
- lua_pushstring(lua,"set");
- lua_gettable(lua,-2);
- t = lua_type(lua,-1);
- if (t == LUA_TTABLE) {
- int setlen = 0;
- void *replylen = addReplyDeferredLen(c);
- /* we took care of the stack size on function start */
- lua_pushnil(lua); /* Use nil to start iteration. */
- while (lua_next(lua,-2)) {
- /* Stack now: table, key, true */
- lua_pop(lua,1); /* Discard the boolean value. */
- lua_pushvalue(lua,-1); /* Dup key before consuming. */
- luaReplyToRedisReply(c, lua); /* Return key. */
- /* Stack now: table, key. */
- setlen++;
- }
- setDeferredSetLen(c,replylen,setlen);
- lua_pop(lua,2);
- return;
- }
- lua_pop(lua,1); /* Discard field name pushed before. */
-
- /* Handle the array reply. */
- void *replylen = addReplyDeferredLen(c);
- int j = 1, mbulklen = 0;
- while(1) {
- /* we took care of the stack size on function start */
- lua_pushnumber(lua,j++);
- lua_gettable(lua,-2);
- t = lua_type(lua,-1);
- if (t == LUA_TNIL) {
- lua_pop(lua,1);
- break;
- }
- luaReplyToRedisReply(c, lua);
- mbulklen++;
- }
- setDeferredArrayLen(c,replylen,mbulklen);
- break;
- default:
- addReplyNull(c);
- }
- lua_pop(lua,1);
-}
-
-/* ---------------------------------------------------------------------------
- * Lua redis.* functions implementations.
- * ------------------------------------------------------------------------- */
-
-#define LUA_CMD_OBJCACHE_SIZE 32
-#define LUA_CMD_OBJCACHE_MAX_LEN 64
-int luaRedisGenericCommand(lua_State *lua, int raise_error) {
- int j, argc = lua_gettop(lua);
- struct redisCommand *cmd;
- client *c = server.lua_client;
- sds reply;
-
- /* Cached across calls. */
- static robj **argv = NULL;
- static int argv_size = 0;
- static robj *cached_objects[LUA_CMD_OBJCACHE_SIZE];
- static size_t cached_objects_len[LUA_CMD_OBJCACHE_SIZE];
- static int inuse = 0; /* Recursive calls detection. */
-
- /* By using Lua debug hooks it is possible to trigger a recursive call
- * to luaRedisGenericCommand(), which normally should never happen.
- * To make this function reentrant is futile and makes it slower, but
- * we should at least detect such a misuse, and abort. */
- if (inuse) {
- char *recursion_warning =
- "luaRedisGenericCommand() recursive call detected. "
- "Are you doing funny stuff with Lua debug hooks?";
- serverLog(LL_WARNING,"%s",recursion_warning);
- luaPushError(lua,recursion_warning);
- return 1;
- }
- inuse++;
-
- /* Require at least one argument */
- if (argc == 0) {
- luaPushError(lua,
- "Please specify at least one argument for redis.call()");
- inuse--;
- return raise_error ? luaRaiseError(lua) : 1;
- }
-
- /* Build the arguments vector */
- if (argv_size < argc) {
- argv = zrealloc(argv,sizeof(robj*)*argc);
- argv_size = argc;
- }
-
- for (j = 0; j < argc; j++) {
- char *obj_s;
- size_t obj_len;
- char dbuf[64];
-
- if (lua_type(lua,j+1) == LUA_TNUMBER) {
- /* We can't use lua_tolstring() for number -> string conversion
- * since Lua uses a format specifier that loses precision. */
- lua_Number num = lua_tonumber(lua,j+1);
-
- obj_len = snprintf(dbuf,sizeof(dbuf),"%.17g",(double)num);
- obj_s = dbuf;
- } else {
- obj_s = (char*)lua_tolstring(lua,j+1,&obj_len);
- if (obj_s == NULL) break; /* Not a string. */
- }
-
- /* Try to use a cached object. */
- if (j < LUA_CMD_OBJCACHE_SIZE && cached_objects[j] &&
- cached_objects_len[j] >= obj_len)
- {
- sds s = cached_objects[j]->ptr;
- argv[j] = cached_objects[j];
- cached_objects[j] = NULL;
- memcpy(s,obj_s,obj_len+1);
- sdssetlen(s, obj_len);
- } else {
- argv[j] = createStringObject(obj_s, obj_len);
- }
- }
-
- /* Check if one of the arguments passed by the Lua script
- * is not a string or an integer (lua_isstring() return true for
- * integers as well). */
- if (j != argc) {
- j--;
- while (j >= 0) {
- decrRefCount(argv[j]);
- j--;
- }
- luaPushError(lua,
- "Lua redis() command arguments must be strings or integers");
- inuse--;
- return raise_error ? luaRaiseError(lua) : 1;
- }
-
- /* Pop all arguments from the stack, we do not need them anymore
- * and this way we guaranty we will have room on the stack for the result. */
- lua_pop(lua, argc);
-
- /* Setup our fake client for command execution */
- c->argv = argv;
- c->argc = argc;
- c->user = server.lua_caller->user;
-
- /* Process module hooks */
- moduleCallCommandFilters(c);
- argv = c->argv;
- argc = c->argc;
-
- /* Log the command if debugging is active. */
- if (ldb.active && ldb.step) {
- sds cmdlog = sdsnew("<redis>");
- for (j = 0; j < c->argc; j++) {
- if (j == 10) {
- cmdlog = sdscatprintf(cmdlog," ... (%d more)",
- c->argc-j-1);
- break;
- } else {
- cmdlog = sdscatlen(cmdlog," ",1);
- cmdlog = sdscatsds(cmdlog,c->argv[j]->ptr);
- }
- }
- ldbLog(cmdlog);
- }
-
- /* Command lookup */
- cmd = lookupCommand(argv,argc);
- if (!cmd || ((cmd->arity > 0 && cmd->arity != argc) ||
- (argc < -cmd->arity)))
- {
- if (cmd)
- luaPushError(lua,
- "Wrong number of args calling Redis command From Lua script");
- else
- luaPushError(lua,"Unknown Redis command called from Lua script");
- goto cleanup;
- }
- c->cmd = c->lastcmd = cmd;
-
- /* There are commands that are not allowed inside scripts. */
- if (!server.lua_disable_deny_script && (cmd->flags & CMD_NOSCRIPT)) {
- luaPushError(lua, "This Redis command is not allowed from scripts");
- goto cleanup;
- }
-
- /* This check is for EVAL_RO, EVALSHA_RO. We want to allow only read only commands */
- if ((server.lua_caller->cmd->proc == evalRoCommand ||
- server.lua_caller->cmd->proc == evalShaRoCommand) &&
- (cmd->flags & CMD_WRITE))
- {
- luaPushError(lua, "Write commands are not allowed from read-only scripts");
- goto cleanup;
- }
-
- /* Check the ACLs. */
- int acl_errpos;
- int acl_retval = ACLCheckAllPerm(c,&acl_errpos);
- if (acl_retval != ACL_OK) {
- addACLLogEntry(c,acl_retval,ACL_LOG_CTX_LUA,acl_errpos,NULL,NULL);
- switch (acl_retval) {
- case ACL_DENIED_CMD:
- luaPushError(lua, "The user executing the script can't run this "
- "command or subcommand");
- break;
- case ACL_DENIED_KEY:
- luaPushError(lua, "The user executing the script can't access "
- "at least one of the keys mentioned in the "
- "command arguments");
- break;
- case ACL_DENIED_CHANNEL:
- luaPushError(lua, "The user executing the script can't publish "
- "to the channel mentioned in the command");
- break;
- default:
- luaPushError(lua, "The user executing the script is lacking the "
- "permissions for the command");
- break;
- }
- goto cleanup;
- }
-
- /* Write commands are forbidden against read-only slaves, or if a
- * command marked as non-deterministic was already called in the context
- * of this script. */
- if (cmd->flags & CMD_WRITE) {
- int deny_write_type = writeCommandsDeniedByDiskError();
- if (server.lua_random_dirty && !server.lua_replicate_commands) {
- luaPushError(lua,
- "Write commands not allowed after non deterministic commands. Call redis.replicate_commands() at the start of your script in order to switch to single commands replication mode.");
- goto cleanup;
- } else if (server.masterhost && server.repl_slave_ro &&
- server.lua_caller->id != CLIENT_ID_AOF &&
- !(server.lua_caller->flags & CLIENT_MASTER))
- {
- luaPushError(lua, shared.roslaveerr->ptr);
- goto cleanup;
- } else if (deny_write_type != DISK_ERROR_TYPE_NONE) {
- if (deny_write_type == DISK_ERROR_TYPE_RDB) {
- luaPushError(lua, shared.bgsaveerr->ptr);
- } else {
- sds aof_write_err = sdscatfmt(sdsempty(),
- "-MISCONF Errors writing to the AOF file: %s\r\n",
- strerror(server.aof_last_write_errno));
- luaPushError(lua, aof_write_err);
- sdsfree(aof_write_err);
- }
- goto cleanup;
- }
- }
-
- /* If we reached the memory limit configured via maxmemory, commands that
- * could enlarge the memory usage are not allowed, but only if this is the
- * first write in the context of this script, otherwise we can't stop
- * in the middle. */
- if (server.maxmemory && /* Maxmemory is actually enabled. */
- server.lua_caller->id != CLIENT_ID_AOF && /* Don't care about mem if loading from AOF. */
- !server.masterhost && /* Slave must execute the script. */
- server.lua_write_dirty == 0 && /* Script had no side effects so far. */
- server.lua_oom && /* Detected OOM when script start. */
- (cmd->flags & CMD_DENYOOM))
- {
- luaPushError(lua, shared.oomerr->ptr);
- goto cleanup;
- }
-
- if (cmd->flags & CMD_RANDOM) server.lua_random_dirty = 1;
- if (cmd->flags & CMD_WRITE) server.lua_write_dirty = 1;
-
- /* If this is a Redis Cluster node, we need to make sure Lua is not
- * trying to access non-local keys, with the exception of commands
- * received from our master or when loading the AOF back in memory. */
- if (server.cluster_enabled && server.lua_caller->id != CLIENT_ID_AOF &&
- !(server.lua_caller->flags & CLIENT_MASTER))
- {
- int error_code;
- /* Duplicate relevant flags in the lua client. */
- c->flags &= ~(CLIENT_READONLY|CLIENT_ASKING);
- c->flags |= server.lua_caller->flags & (CLIENT_READONLY|CLIENT_ASKING);
- if (getNodeByQuery(c,c->cmd,c->argv,c->argc,NULL,&error_code) !=
- server.cluster->myself)
- {
- if (error_code == CLUSTER_REDIR_DOWN_RO_STATE) {
- luaPushError(lua,
- "Lua script attempted to execute a write command while the "
- "cluster is down and readonly");
- } else if (error_code == CLUSTER_REDIR_DOWN_STATE) {
- luaPushError(lua,
- "Lua script attempted to execute a command while the "
- "cluster is down");
- } else {
- luaPushError(lua,
- "Lua script attempted to access a non local key in a "
- "cluster node");
- }
-
- goto cleanup;
- }
- }
-
- /* If we are using single commands replication, we need to wrap what
- * we propagate into a MULTI/EXEC block, so that it will be atomic like
- * a Lua script in the context of AOF and slaves. */
- if (server.lua_replicate_commands &&
- !server.lua_multi_emitted &&
- !(server.lua_caller->flags & CLIENT_MULTI) &&
- server.lua_write_dirty &&
- server.lua_repl != PROPAGATE_NONE)
- {
- execCommandPropagateMulti(server.lua_caller->db->id);
- server.lua_multi_emitted = 1;
- /* Now we are in the MULTI context, the lua_client should be
- * flag as CLIENT_MULTI. */
- c->flags |= CLIENT_MULTI;
- }
-
- /* Run the command */
- int call_flags = CMD_CALL_SLOWLOG | CMD_CALL_STATS;
- if (server.lua_replicate_commands) {
- /* Set flags according to redis.set_repl() settings. */
- if (server.lua_repl & PROPAGATE_AOF)
- call_flags |= CMD_CALL_PROPAGATE_AOF;
- if (server.lua_repl & PROPAGATE_REPL)
- call_flags |= CMD_CALL_PROPAGATE_REPL;
- }
- call(c,call_flags);
- serverAssert((c->flags & CLIENT_BLOCKED) == 0);
-
- /* Convert the result of the Redis command into a suitable Lua type.
- * The first thing we need is to create a single string from the client
- * output buffers. */
- if (listLength(c->reply) == 0 && (size_t)c->bufpos < c->buf_usable_size) {
- /* This is a fast path for the common case of a reply inside the
- * client static buffer. Don't create an SDS string but just use
- * the client buffer directly. */
- c->buf[c->bufpos] = '\0';
- reply = c->buf;
- c->bufpos = 0;
- } else {
- reply = sdsnewlen(c->buf,c->bufpos);
- c->bufpos = 0;
- while(listLength(c->reply)) {
- clientReplyBlock *o = listNodeValue(listFirst(c->reply));
-
- reply = sdscatlen(reply,o->buf,o->used);
- listDelNode(c->reply,listFirst(c->reply));
- }
- }
- if (raise_error && reply[0] != '-') raise_error = 0;
- redisProtocolToLuaType(lua,reply);
-
- /* If the debugger is active, log the reply from Redis. */
- if (ldb.active && ldb.step)
- ldbLogRedisReply(reply);
-
- /* Sort the output array if needed, assuming it is a non-null multi bulk
- * reply as expected. */
- if ((cmd->flags & CMD_SORT_FOR_SCRIPT) &&
- (server.lua_replicate_commands == 0) &&
- (reply[0] == '*' && reply[1] != '-')) {
- luaSortArray(lua);
- }
- if (reply != c->buf) sdsfree(reply);
- c->reply_bytes = 0;
-
-cleanup:
- /* Clean up. Command code may have changed argv/argc so we use the
- * argv/argc of the client instead of the local variables. */
- for (j = 0; j < c->argc; j++) {
- robj *o = c->argv[j];
-
- /* Try to cache the object in the cached_objects array.
- * The object must be small, SDS-encoded, and with refcount = 1
- * (we must be the only owner) for us to cache it. */
- if (j < LUA_CMD_OBJCACHE_SIZE &&
- o->refcount == 1 &&
- (o->encoding == OBJ_ENCODING_RAW ||
- o->encoding == OBJ_ENCODING_EMBSTR) &&
- sdslen(o->ptr) <= LUA_CMD_OBJCACHE_MAX_LEN)
- {
- sds s = o->ptr;
- if (cached_objects[j]) decrRefCount(cached_objects[j]);
- cached_objects[j] = o;
- cached_objects_len[j] = sdsalloc(s);
- } else {
- decrRefCount(o);
- }
- }
-
- if (c->argv != argv) {
- zfree(c->argv);
- argv = NULL;
- argv_size = 0;
- }
-
- c->user = NULL;
-
- if (raise_error) {
- /* If we are here we should have an error in the stack, in the
- * form of a table with an "err" field. Extract the string to
- * return the plain error. */
- inuse--;
- return luaRaiseError(lua);
- }
- inuse--;
- return 1;
-}
-
-/* redis.call() */
-int luaRedisCallCommand(lua_State *lua) {
- return luaRedisGenericCommand(lua,1);
-}
-
-/* redis.pcall() */
-int luaRedisPCallCommand(lua_State *lua) {
- return luaRedisGenericCommand(lua,0);
-}
-
-/* This adds redis.sha1hex(string) to Lua scripts using the same hashing
- * function used for sha1ing lua scripts. */
-int luaRedisSha1hexCommand(lua_State *lua) {
- int argc = lua_gettop(lua);
- char digest[41];
- size_t len;
- char *s;
-
- if (argc != 1) {
- lua_pushstring(lua, "wrong number of arguments");
- return lua_error(lua);
- }
-
- s = (char*)lua_tolstring(lua,1,&len);
- sha1hex(digest,s,len);
- lua_pushstring(lua,digest);
- return 1;
-}
-
-/* Returns a table with a single field 'field' set to the string value
- * passed as argument. This helper function is handy when returning
- * a Redis Protocol error or status reply from Lua:
- *
- * return redis.error_reply("ERR Some Error")
- * return redis.status_reply("ERR Some Error")
- */
-int luaRedisReturnSingleFieldTable(lua_State *lua, char *field) {
- if (lua_gettop(lua) != 1 || lua_type(lua,-1) != LUA_TSTRING) {
- luaPushError(lua, "wrong number or type of arguments");
- return 1;
- }
-
- lua_newtable(lua);
- lua_pushstring(lua, field);
- lua_pushvalue(lua, -3);
- lua_settable(lua, -3);
- return 1;
-}
-
-/* redis.error_reply() */
-int luaRedisErrorReplyCommand(lua_State *lua) {
- return luaRedisReturnSingleFieldTable(lua,"err");
-}
-
-/* redis.status_reply() */
-int luaRedisStatusReplyCommand(lua_State *lua) {
- return luaRedisReturnSingleFieldTable(lua,"ok");
-}
-
-/* redis.replicate_commands()
- *
- * Turn on single commands replication if the script never called
- * a write command so far, and returns true. Otherwise if the script
- * already started to write, returns false and stick to whole scripts
- * replication, which is our default. */
-int luaRedisReplicateCommandsCommand(lua_State *lua) {
- if (server.lua_write_dirty) {
- lua_pushboolean(lua,0);
- } else {
- server.lua_replicate_commands = 1;
- /* When we switch to single commands replication, we can provide
- * different math.random() sequences at every call, which is what
- * the user normally expects. */
- redisSrand48(rand());
- lua_pushboolean(lua,1);
- }
- return 1;
-}
-
-/* redis.breakpoint()
- *
- * Allows to stop execution during a debugging session from within
- * the Lua code implementation, like if a breakpoint was set in the code
- * immediately after the function. */
-int luaRedisBreakpointCommand(lua_State *lua) {
- if (ldb.active) {
- ldb.luabp = 1;
- lua_pushboolean(lua,1);
- } else {
- lua_pushboolean(lua,0);
- }
- return 1;
-}
-
-/* redis.debug()
- *
- * Log a string message into the output console.
- * Can take multiple arguments that will be separated by commas.
- * Nothing is returned to the caller. */
-int luaRedisDebugCommand(lua_State *lua) {
- if (!ldb.active) return 0;
- int argc = lua_gettop(lua);
- sds log = sdscatprintf(sdsempty(),"<debug> line %d: ", ldb.currentline);
- while(argc--) {
- log = ldbCatStackValue(log,lua,-1 - argc);
- if (argc != 0) log = sdscatlen(log,", ",2);
- }
- ldbLog(log);
- return 0;
-}
-
-/* redis.set_repl()
- *
- * Set the propagation of write commands executed in the context of the
- * script to on/off for AOF and slaves. */
-int luaRedisSetReplCommand(lua_State *lua) {
- int argc = lua_gettop(lua);
- int flags;
-
- if (server.lua_replicate_commands == 0) {
- lua_pushstring(lua, "You can set the replication behavior only after turning on single commands replication with redis.replicate_commands().");
- return lua_error(lua);
- } else if (argc != 1) {
- lua_pushstring(lua, "redis.set_repl() requires two arguments.");
- return lua_error(lua);
- }
-
- flags = lua_tonumber(lua,-1);
- if ((flags & ~(PROPAGATE_AOF|PROPAGATE_REPL)) != 0) {
- lua_pushstring(lua, "Invalid replication flags. Use REPL_AOF, REPL_REPLICA, REPL_ALL or REPL_NONE.");
- return lua_error(lua);
- }
- server.lua_repl = flags;
- return 0;
-}
-
-/* redis.log() */
-int luaLogCommand(lua_State *lua) {
- int j, argc = lua_gettop(lua);
- int level;
- sds log;
-
- if (argc < 2) {
- lua_pushstring(lua, "redis.log() requires two arguments or more.");
- return lua_error(lua);
- } else if (!lua_isnumber(lua,-argc)) {
- lua_pushstring(lua, "First argument must be a number (log level).");
- return lua_error(lua);
- }
- level = lua_tonumber(lua,-argc);
- if (level < LL_DEBUG || level > LL_WARNING) {
- lua_pushstring(lua, "Invalid debug level.");
- return lua_error(lua);
- }
- if (level < server.verbosity) return 0;
-
- /* Glue together all the arguments */
- log = sdsempty();
- for (j = 1; j < argc; j++) {
- size_t len;
- char *s;
-
- s = (char*)lua_tolstring(lua,(-argc)+j,&len);
- if (s) {
- if (j != 1) log = sdscatlen(log," ",1);
- log = sdscatlen(log,s,len);
- }
- }
- serverLogRaw(level,log);
- sdsfree(log);
- return 0;
-}
-
-/* redis.setresp() */
-int luaSetResp(lua_State *lua) {
- int argc = lua_gettop(lua);
-
- if (argc != 1) {
- lua_pushstring(lua, "redis.setresp() requires one argument.");
- return lua_error(lua);
- }
-
- int resp = lua_tonumber(lua,-argc);
- if (resp != 2 && resp != 3) {
- lua_pushstring(lua, "RESP version must be 2 or 3.");
- return lua_error(lua);
- }
-
- server.lua_client->resp = resp;
- return 0;
-}
-
-/* ---------------------------------------------------------------------------
- * Lua engine initialization and reset.
- * ------------------------------------------------------------------------- */
-
-void luaLoadLib(lua_State *lua, const char *libname, lua_CFunction luafunc) {
- lua_pushcfunction(lua, luafunc);
- lua_pushstring(lua, libname);
- lua_call(lua, 1, 0);
-}
-
-LUALIB_API int (luaopen_cjson) (lua_State *L);
-LUALIB_API int (luaopen_struct) (lua_State *L);
-LUALIB_API int (luaopen_cmsgpack) (lua_State *L);
-LUALIB_API int (luaopen_bit) (lua_State *L);
-
-void luaLoadLibraries(lua_State *lua) {
- luaLoadLib(lua, "", luaopen_base);
- luaLoadLib(lua, LUA_TABLIBNAME, luaopen_table);
- luaLoadLib(lua, LUA_STRLIBNAME, luaopen_string);
- luaLoadLib(lua, LUA_MATHLIBNAME, luaopen_math);
- luaLoadLib(lua, LUA_DBLIBNAME, luaopen_debug);
- luaLoadLib(lua, "cjson", luaopen_cjson);
- luaLoadLib(lua, "struct", luaopen_struct);
- luaLoadLib(lua, "cmsgpack", luaopen_cmsgpack);
- luaLoadLib(lua, "bit", luaopen_bit);
-
-#if 0 /* Stuff that we don't load currently, for sandboxing concerns. */
- luaLoadLib(lua, LUA_LOADLIBNAME, luaopen_package);
- luaLoadLib(lua, LUA_OSLIBNAME, luaopen_os);
-#endif
-}
-
-/* Remove a functions that we don't want to expose to the Redis scripting
- * environment. */
-void luaRemoveUnsupportedFunctions(lua_State *lua) {
- lua_pushnil(lua);
- lua_setglobal(lua,"loadfile");
- lua_pushnil(lua);
- lua_setglobal(lua,"dofile");
-}
-
-/* This function installs metamethods in the global table _G that prevent
- * the creation of globals accidentally.
- *
- * It should be the last to be called in the scripting engine initialization
- * sequence, because it may interact with creation of globals. */
-void scriptingEnableGlobalsProtection(lua_State *lua) {
- char *s[32];
- sds code = sdsempty();
- int j = 0;
-
- /* strict.lua from: http://metalua.luaforge.net/src/lib/strict.lua.html.
- * Modified to be adapted to Redis. */
- s[j++]="local dbg=debug\n";
- s[j++]="local mt = {}\n";
- s[j++]="setmetatable(_G, mt)\n";
- s[j++]="mt.__newindex = function (t, n, v)\n";
- s[j++]=" if dbg.getinfo(2) then\n";
- s[j++]=" local w = dbg.getinfo(2, \"S\").what\n";
- s[j++]=" if w ~= \"main\" and w ~= \"C\" then\n";
- s[j++]=" error(\"Script attempted to create global variable '\"..tostring(n)..\"'\", 2)\n";
- s[j++]=" end\n";
- s[j++]=" end\n";
- s[j++]=" rawset(t, n, v)\n";
- s[j++]="end\n";
- s[j++]="mt.__index = function (t, n)\n";
- s[j++]=" if dbg.getinfo(2) and dbg.getinfo(2, \"S\").what ~= \"C\" then\n";
- s[j++]=" error(\"Script attempted to access nonexistent global variable '\"..tostring(n)..\"'\", 2)\n";
- s[j++]=" end\n";
- s[j++]=" return rawget(t, n)\n";
- s[j++]="end\n";
- s[j++]="debug = nil\n";
- s[j++]=NULL;
-
- for (j = 0; s[j] != NULL; j++) code = sdscatlen(code,s[j],strlen(s[j]));
- luaL_loadbuffer(lua,code,sdslen(code),"@enable_strict_lua");
- lua_pcall(lua,0,0,0);
- sdsfree(code);
-}
-
-/* Initialize the scripting environment.
- *
- * This function is called the first time at server startup with
- * the 'setup' argument set to 1.
- *
- * It can be called again multiple times during the lifetime of the Redis
- * process, with 'setup' set to 0, and following a scriptingRelease() call,
- * in order to reset the Lua scripting environment.
- *
- * However it is simpler to just call scriptingReset() that does just that. */
-void scriptingInit(int setup) {
- lua_State *lua = lua_open();
-
- if (setup) {
- server.lua_client = NULL;
- server.lua_caller = NULL;
- server.lua_cur_script = NULL;
- server.lua_timedout = 0;
- server.lua_disable_deny_script = 0;
- ldbInit();
- }
-
- luaLoadLibraries(lua);
- luaRemoveUnsupportedFunctions(lua);
-
- /* Initialize a dictionary we use to map SHAs to scripts.
- * This is useful for replication, as we need to replicate EVALSHA
- * as EVAL, so we need to remember the associated script. */
- server.lua_scripts = dictCreate(&shaScriptObjectDictType);
- server.lua_scripts_mem = 0;
-
- /* Register the redis commands table and fields */
- lua_newtable(lua);
-
- /* redis.call */
- lua_pushstring(lua,"call");
- lua_pushcfunction(lua,luaRedisCallCommand);
- lua_settable(lua,-3);
-
- /* redis.pcall */
- lua_pushstring(lua,"pcall");
- lua_pushcfunction(lua,luaRedisPCallCommand);
- lua_settable(lua,-3);
-
- /* redis.log and log levels. */
- lua_pushstring(lua,"log");
- lua_pushcfunction(lua,luaLogCommand);
- lua_settable(lua,-3);
-
- /* redis.setresp */
- lua_pushstring(lua,"setresp");
- lua_pushcfunction(lua,luaSetResp);
- lua_settable(lua,-3);
-
- lua_pushstring(lua,"LOG_DEBUG");
- lua_pushnumber(lua,LL_DEBUG);
- lua_settable(lua,-3);
-
- lua_pushstring(lua,"LOG_VERBOSE");
- lua_pushnumber(lua,LL_VERBOSE);
- lua_settable(lua,-3);
-
- lua_pushstring(lua,"LOG_NOTICE");
- lua_pushnumber(lua,LL_NOTICE);
- lua_settable(lua,-3);
-
- lua_pushstring(lua,"LOG_WARNING");
- lua_pushnumber(lua,LL_WARNING);
- lua_settable(lua,-3);
-
- /* redis.sha1hex */
- lua_pushstring(lua, "sha1hex");
- lua_pushcfunction(lua, luaRedisSha1hexCommand);
- lua_settable(lua, -3);
-
- /* redis.error_reply and redis.status_reply */
- lua_pushstring(lua, "error_reply");
- lua_pushcfunction(lua, luaRedisErrorReplyCommand);
- lua_settable(lua, -3);
- lua_pushstring(lua, "status_reply");
- lua_pushcfunction(lua, luaRedisStatusReplyCommand);
- lua_settable(lua, -3);
-
- /* redis.replicate_commands */
- lua_pushstring(lua, "replicate_commands");
- lua_pushcfunction(lua, luaRedisReplicateCommandsCommand);
- lua_settable(lua, -3);
-
- /* redis.set_repl and associated flags. */
- lua_pushstring(lua,"set_repl");
- lua_pushcfunction(lua,luaRedisSetReplCommand);
- lua_settable(lua,-3);
-
- lua_pushstring(lua,"REPL_NONE");
- lua_pushnumber(lua,PROPAGATE_NONE);
- lua_settable(lua,-3);
-
- lua_pushstring(lua,"REPL_AOF");
- lua_pushnumber(lua,PROPAGATE_AOF);
- lua_settable(lua,-3);
-
- lua_pushstring(lua,"REPL_SLAVE");
- lua_pushnumber(lua,PROPAGATE_REPL);
- lua_settable(lua,-3);
-
- lua_pushstring(lua,"REPL_REPLICA");
- lua_pushnumber(lua,PROPAGATE_REPL);
- lua_settable(lua,-3);
-
- lua_pushstring(lua,"REPL_ALL");
- lua_pushnumber(lua,PROPAGATE_AOF|PROPAGATE_REPL);
- lua_settable(lua,-3);
-
- /* redis.breakpoint */
- lua_pushstring(lua,"breakpoint");
- lua_pushcfunction(lua,luaRedisBreakpointCommand);
- lua_settable(lua,-3);
-
- /* redis.debug */
- lua_pushstring(lua,"debug");
- lua_pushcfunction(lua,luaRedisDebugCommand);
- lua_settable(lua,-3);
-
- /* Finally set the table as 'redis' global var. */
- lua_setglobal(lua,"redis");
-
- /* Replace math.random and math.randomseed with our implementations. */
- lua_getglobal(lua,"math");
-
- lua_pushstring(lua,"random");
- lua_pushcfunction(lua,redis_math_random);
- lua_settable(lua,-3);
-
- lua_pushstring(lua,"randomseed");
- lua_pushcfunction(lua,redis_math_randomseed);
- lua_settable(lua,-3);
-
- lua_setglobal(lua,"math");
-
- /* Add a helper function that we use to sort the multi bulk output of non
- * deterministic commands, when containing 'false' elements. */
- {
- char *compare_func = "function __redis__compare_helper(a,b)\n"
- " if a == false then a = '' end\n"
- " if b == false then b = '' end\n"
- " return a<b\n"
- "end\n";
- luaL_loadbuffer(lua,compare_func,strlen(compare_func),"@cmp_func_def");
- lua_pcall(lua,0,0,0);
- }
-
- /* Add a helper function we use for pcall error reporting.
- * Note that when the error is in the C function we want to report the
- * information about the caller, that's what makes sense from the point
- * of view of the user debugging a script. */
- {
- char *errh_func = "local dbg = debug\n"
- "function __redis__err__handler(err)\n"
- " local i = dbg.getinfo(2,'nSl')\n"
- " if i and i.what == 'C' then\n"
- " i = dbg.getinfo(3,'nSl')\n"
- " end\n"
- " if i then\n"
- " return i.source .. ':' .. i.currentline .. ': ' .. err\n"
- " else\n"
- " return err\n"
- " end\n"
- "end\n";
- luaL_loadbuffer(lua,errh_func,strlen(errh_func),"@err_handler_def");
- lua_pcall(lua,0,0,0);
- }
-
- /* Create the (non connected) client that we use to execute Redis commands
- * inside the Lua interpreter.
- * Note: there is no need to create it again when this function is called
- * by scriptingReset(). */
- if (server.lua_client == NULL) {
- server.lua_client = createClient(NULL);
- server.lua_client->flags |= CLIENT_LUA;
-
- /* We do not want to allow blocking commands inside Lua */
- server.lua_client->flags |= CLIENT_DENY_BLOCKING;
- }
-
- /* Lua beginners often don't use "local", this is likely to introduce
- * subtle bugs in their code. To prevent problems we protect accesses
- * to global variables. */
- scriptingEnableGlobalsProtection(lua);
-
- server.lua = lua;
-}
-
-/* Release resources related to Lua scripting.
- * This function is used in order to reset the scripting environment. */
-void scriptingRelease(int async) {
- if (async)
- freeLuaScriptsAsync(server.lua_scripts);
- else
- dictRelease(server.lua_scripts);
- server.lua_scripts_mem = 0;
- lua_close(server.lua);
-}
-
-void scriptingReset(int async) {
- scriptingRelease(async);
- scriptingInit(0);
-}
-
-/* Set an array of Redis String Objects as a Lua array (table) stored into a
- * global variable. */
-void luaSetGlobalArray(lua_State *lua, char *var, robj **elev, int elec) {
- int j;
-
- lua_newtable(lua);
- for (j = 0; j < elec; j++) {
- lua_pushlstring(lua,(char*)elev[j]->ptr,sdslen(elev[j]->ptr));
- lua_rawseti(lua,-2,j+1);
- }
- lua_setglobal(lua,var);
-}
-
-/* ---------------------------------------------------------------------------
- * Redis provided math.random
- * ------------------------------------------------------------------------- */
-
-/* We replace math.random() with our implementation that is not affected
- * by specific libc random() implementations and will output the same sequence
- * (for the same seed) in every arch. */
-
-/* The following implementation is the one shipped with Lua itself but with
- * rand() replaced by redisLrand48(). */
-int redis_math_random (lua_State *L) {
- /* the `%' avoids the (rare) case of r==1, and is needed also because on
- some systems (SunOS!) `rand()' may return a value larger than RAND_MAX */
- lua_Number r = (lua_Number)(redisLrand48()%REDIS_LRAND48_MAX) /
- (lua_Number)REDIS_LRAND48_MAX;
- switch (lua_gettop(L)) { /* check number of arguments */
- case 0: { /* no arguments */
- lua_pushnumber(L, r); /* Number between 0 and 1 */
- break;
- }
- case 1: { /* only upper limit */
- int u = luaL_checkint(L, 1);
- luaL_argcheck(L, 1<=u, 1, "interval is empty");
- lua_pushnumber(L, floor(r*u)+1); /* int between 1 and `u' */
- break;
- }
- case 2: { /* lower and upper limits */
- int l = luaL_checkint(L, 1);
- int u = luaL_checkint(L, 2);
- luaL_argcheck(L, l<=u, 2, "interval is empty");
- lua_pushnumber(L, floor(r*(u-l+1))+l); /* int between `l' and `u' */
- break;
- }
- default: return luaL_error(L, "wrong number of arguments");
- }
- return 1;
-}
-
-int redis_math_randomseed (lua_State *L) {
- redisSrand48(luaL_checkint(L, 1));
- return 0;
-}
-
-/* ---------------------------------------------------------------------------
- * EVAL and SCRIPT commands implementation
- * ------------------------------------------------------------------------- */
-
-/* Define a Lua function with the specified body.
- * The function name will be generated in the following form:
- *
- * f_<hex sha1 sum>
- *
- * The function increments the reference count of the 'body' object as a
- * side effect of a successful call.
- *
- * On success a pointer to an SDS string representing the function SHA1 of the
- * just added function is returned (and will be valid until the next call
- * to scriptingReset() function), otherwise NULL is returned.
- *
- * The function handles the fact of being called with a script that already
- * exists, and in such a case, it behaves like in the success case.
- *
- * If 'c' is not NULL, on error the client is informed with an appropriate
- * error describing the nature of the problem and the Lua interpreter error. */
-sds luaCreateFunction(client *c, lua_State *lua, robj *body) {
- char funcname[43];
- dictEntry *de;
-
- funcname[0] = 'f';
- funcname[1] = '_';
- sha1hex(funcname+2,body->ptr,sdslen(body->ptr));
-
- sds sha = sdsnewlen(funcname+2,40);
- if ((de = dictFind(server.lua_scripts,sha)) != NULL) {
- sdsfree(sha);
- return dictGetKey(de);
- }
-
- sds funcdef = sdsempty();
- funcdef = sdscat(funcdef,"function ");
- funcdef = sdscatlen(funcdef,funcname,42);
- funcdef = sdscatlen(funcdef,"() ",3);
- funcdef = sdscatlen(funcdef,body->ptr,sdslen(body->ptr));
- funcdef = sdscatlen(funcdef,"\nend",4);
-
- if (luaL_loadbuffer(lua,funcdef,sdslen(funcdef),"@user_script")) {
- if (c != NULL) {
- addReplyErrorFormat(c,
- "Error compiling script (new function): %s\n",
- lua_tostring(lua,-1));
- }
- lua_pop(lua,1);
- sdsfree(sha);
- sdsfree(funcdef);
- return NULL;
- }
- sdsfree(funcdef);
-
- if (lua_pcall(lua,0,0,0)) {
- if (c != NULL) {
- addReplyErrorFormat(c,"Error running script (new function): %s\n",
- lua_tostring(lua,-1));
- }
- lua_pop(lua,1);
- sdsfree(sha);
- return NULL;
- }
-
- /* We also save a SHA1 -> Original script map in a dictionary
- * so that we can replicate / write in the AOF all the
- * EVALSHA commands as EVAL using the original script. */
- int retval = dictAdd(server.lua_scripts,sha,body);
- serverAssertWithInfo(c ? c : server.lua_client,NULL,retval == DICT_OK);
- server.lua_scripts_mem += sdsZmallocSize(sha) + getStringObjectSdsUsedMemory(body);
- incrRefCount(body);
- return sha;
-}
-
-/* This is the Lua script "count" hook that we use to detect scripts timeout. */
-void luaMaskCountHook(lua_State *lua, lua_Debug *ar) {
- long long elapsed = elapsedMs(server.lua_time_start);
- UNUSED(ar);
- UNUSED(lua);
-
- /* Set the timeout condition if not already set and the maximum
- * execution time was reached. */
- if (elapsed >= server.lua_time_limit && server.lua_timedout == 0) {
- serverLog(LL_WARNING,
- "Lua slow script detected: still in execution after %lld milliseconds. "
- "You can try killing the script using the SCRIPT KILL command. "
- "Script SHA1 is: %s",
- elapsed, server.lua_cur_script);
- server.lua_timedout = 1;
- blockingOperationStarts();
- /* Once the script timeouts we reenter the event loop to permit others
- * to call SCRIPT KILL or SHUTDOWN NOSAVE if needed. For this reason
- * we need to mask the client executing the script from the event loop.
- * If we don't do that the client may disconnect and could no longer be
- * here when the EVAL command will return. */
- protectClient(server.lua_caller);
- }
- if (server.lua_timedout) processEventsWhileBlocked();
- if (server.lua_kill) {
- serverLog(LL_WARNING,"Lua script killed by user with SCRIPT KILL.");
-
- /*
- * Set the hook to invoke all the time so the user
-         * will not be able to catch the error with pcall and invoke
-         * pcall again which will prevent the script from ever been killed
- */
- lua_sethook(lua, luaMaskCountHook, LUA_MASKLINE, 0);
-
- lua_pushstring(lua,"Script killed by user with SCRIPT KILL...");
- lua_error(lua);
- }
-}
-
-void prepareLuaClient(void) {
- /* Select the right DB in the context of the Lua client */
- selectDb(server.lua_client,server.lua_caller->db->id);
- server.lua_client->resp = 2; /* Default is RESP2, scripts can change it. */
-
- /* If we are in MULTI context, flag Lua client as CLIENT_MULTI. */
- if (server.lua_caller->flags & CLIENT_MULTI) {
- server.lua_client->flags |= CLIENT_MULTI;
- }
-}
-
-void resetLuaClient(void) {
- /* After the script done, remove the MULTI state. */
- server.lua_client->flags &= ~CLIENT_MULTI;
-}
-
-void evalGenericCommand(client *c, int evalsha) {
- lua_State *lua = server.lua;
- char funcname[43];
- long long numkeys;
- long long initial_server_dirty = server.dirty;
- int delhook = 0, err;
-
- /* When we replicate whole scripts, we want the same PRNG sequence at
- * every call so that our PRNG is not affected by external state. */
- redisSrand48(0);
-
- /* We set this flag to zero to remember that so far no random command
- * was called. This way we can allow the user to call commands like
- * SRANDMEMBER or RANDOMKEY from Lua scripts as far as no write command
- * is called (otherwise the replication and AOF would end with non
- * deterministic sequences).
- *
- * Thanks to this flag we'll raise an error every time a write command
- * is called after a random command was used. */
- server.lua_random_dirty = 0;
- server.lua_write_dirty = 0;
- server.lua_replicate_commands = server.lua_always_replicate_commands;
- server.lua_multi_emitted = 0;
- server.lua_repl = PROPAGATE_AOF|PROPAGATE_REPL;
-
- /* Get the number of arguments that are keys */
- if (getLongLongFromObjectOrReply(c,c->argv[2],&numkeys,NULL) != C_OK)
- return;
- if (numkeys > (c->argc - 3)) {
- addReplyError(c,"Number of keys can't be greater than number of args");
- return;
- } else if (numkeys < 0) {
- addReplyError(c,"Number of keys can't be negative");
- return;
- }
-
- /* We obtain the script SHA1, then check if this function is already
- * defined into the Lua state */
- funcname[0] = 'f';
- funcname[1] = '_';
- if (!evalsha) {
- /* Hash the code if this is an EVAL call */
- sha1hex(funcname+2,c->argv[1]->ptr,sdslen(c->argv[1]->ptr));
- } else {
- /* We already have the SHA if it is an EVALSHA */
- int j;
- char *sha = c->argv[1]->ptr;
-
- /* Convert to lowercase. We don't use tolower since the function
- * managed to always show up in the profiler output consuming
- * a non trivial amount of time. */
- for (j = 0; j < 40; j++)
- funcname[j+2] = (sha[j] >= 'A' && sha[j] <= 'Z') ?
- sha[j]+('a'-'A') : sha[j];
- funcname[42] = '\0';
- }
-
- /* Push the pcall error handler function on the stack. */
- lua_getglobal(lua, "__redis__err__handler");
-
- /* Try to lookup the Lua function */
- lua_getglobal(lua, funcname);
- if (lua_isnil(lua,-1)) {
- lua_pop(lua,1); /* remove the nil from the stack */
- /* Function not defined... let's define it if we have the
- * body of the function. If this is an EVALSHA call we can just
- * return an error. */
- if (evalsha) {
- lua_pop(lua,1); /* remove the error handler from the stack. */
- addReplyErrorObject(c, shared.noscripterr);
- return;
- }
- if (luaCreateFunction(c,lua,c->argv[1]) == NULL) {
- lua_pop(lua,1); /* remove the error handler from the stack. */
- /* The error is sent to the client by luaCreateFunction()
- * itself when it returns NULL. */
- return;
- }
- /* Now the following is guaranteed to return non nil */
- lua_getglobal(lua, funcname);
- serverAssert(!lua_isnil(lua,-1));
- }
-
- /* Populate the argv and keys table accordingly to the arguments that
- * EVAL received. */
- luaSetGlobalArray(lua,"KEYS",c->argv+3,numkeys);
- luaSetGlobalArray(lua,"ARGV",c->argv+3+numkeys,c->argc-3-numkeys);
-
- /* Set a hook in order to be able to stop the script execution if it
- * is running for too much time.
- * We set the hook only if the time limit is enabled as the hook will
- * make the Lua script execution slower.
- *
- * If we are debugging, we set instead a "line" hook so that the
- * debugger is call-back at every line executed by the script. */
- server.in_eval = 1;
- server.lua_caller = c;
- server.lua_cur_script = funcname + 2;
- server.lua_time_start = getMonotonicUs();
- server.lua_time_snapshot = mstime();
- server.lua_kill = 0;
- if (server.lua_time_limit > 0 && ldb.active == 0) {
- lua_sethook(lua,luaMaskCountHook,LUA_MASKCOUNT,100000);
- delhook = 1;
- } else if (ldb.active) {
- lua_sethook(server.lua,luaLdbLineHook,LUA_MASKLINE|LUA_MASKCOUNT,100000);
- delhook = 1;
- }
-
- prepareLuaClient();
-
- /* At this point whether this script was never seen before or if it was
- * already defined, we can call it. We have zero arguments and expect
- * a single return value. */
- err = lua_pcall(lua,0,1,-2);
-
- resetLuaClient();
-
- /* Perform some cleanup that we need to do both on error and success. */
- if (delhook) lua_sethook(lua,NULL,0,0); /* Disable hook */
- if (server.lua_timedout) {
- server.lua_timedout = 0;
- blockingOperationEnds();
- /* Restore the client that was protected when the script timeout
- * was detected. */
- unprotectClient(c);
- if (server.masterhost && server.master)
- queueClientForReprocessing(server.master);
- }
- server.in_eval = 0;
- server.lua_caller = NULL;
- server.lua_cur_script = NULL;
-
- /* Call the Lua garbage collector from time to time to avoid a
- * full cycle performed by Lua, which adds too latency.
- *
- * The call is performed every LUA_GC_CYCLE_PERIOD executed commands
- * (and for LUA_GC_CYCLE_PERIOD collection steps) because calling it
- * for every command uses too much CPU. */
- #define LUA_GC_CYCLE_PERIOD 50
- {
- static long gc_count = 0;
-
- gc_count++;
- if (gc_count == LUA_GC_CYCLE_PERIOD) {
- lua_gc(lua,LUA_GCSTEP,LUA_GC_CYCLE_PERIOD);
- gc_count = 0;
- }
- }
-
- if (err) {
- addReplyErrorFormat(c,"Error running script (call to %s): %s\n",
- funcname, lua_tostring(lua,-1));
- lua_pop(lua,2); /* Consume the Lua reply and remove error handler. */
- } else {
- /* On success convert the Lua return value into Redis protocol, and
- * send it to * the client. */
- luaReplyToRedisReply(c,lua); /* Convert and consume the reply. */
- lua_pop(lua,1); /* Remove the error handler. */
- }
-
- /* If we are using single commands replication, emit EXEC if there
- * was at least a write. */
- if (server.lua_replicate_commands) {
- preventCommandPropagation(c);
- if (server.lua_multi_emitted) {
- execCommandPropagateExec(c->db->id);
- }
- }
-
- /* EVALSHA should be propagated to Slave and AOF file as full EVAL, unless
- * we are sure that the script was already in the context of all the
- * attached slaves *and* the current AOF file if enabled.
- *
- * To do so we use a cache of SHA1s of scripts that we already propagated
- * as full EVAL, that's called the Replication Script Cache.
- *
- * For replication, every time a new slave attaches to the master, we need to
- * flush our cache of scripts that can be replicated as EVALSHA, while
- * for AOF we need to do so every time we rewrite the AOF file. */
- if (evalsha && !server.lua_replicate_commands) {
- if (!replicationScriptCacheExists(c->argv[1]->ptr)) {
- /* This script is not in our script cache, replicate it as
- * EVAL, then add it into the script cache, as from now on
- * slaves and AOF know about it. */
- robj *script = dictFetchValue(server.lua_scripts,c->argv[1]->ptr);
-
- replicationScriptCacheAdd(c->argv[1]->ptr);
- serverAssertWithInfo(c,NULL,script != NULL);
-
- /* If the script did not produce any changes in the dataset we want
- * just to replicate it as SCRIPT LOAD, otherwise we risk running
- * an aborted script on slaves (that may then produce results there)
- * or just running a CPU costly read-only script on the slaves. */
- if (server.dirty == initial_server_dirty) {
- rewriteClientCommandVector(c,3,
- shared.script,
- shared.load,
- script);
- } else {
- rewriteClientCommandArgument(c,0,shared.eval);
- rewriteClientCommandArgument(c,1,script);
- }
- forceCommandPropagation(c,PROPAGATE_REPL|PROPAGATE_AOF);
- }
- }
-}
-
-void evalCommand(client *c) {
- /* Explicitly feed monitor here so that lua commands appear after their
- * script command. */
- replicationFeedMonitors(c,server.monitors,c->db->id,c->argv,c->argc);
- if (!(c->flags & CLIENT_LUA_DEBUG))
- evalGenericCommand(c,0);
- else
- evalGenericCommandWithDebugging(c,0);
-}
-
-void evalRoCommand(client *c) {
- evalCommand(c);
-}
-
-void evalShaCommand(client *c) {
- /* Explicitly feed monitor here so that lua commands appear after their
- * script command. */
- replicationFeedMonitors(c,server.monitors,c->db->id,c->argv,c->argc);
- if (sdslen(c->argv[1]->ptr) != 40) {
- /* We know that a match is not possible if the provided SHA is
- * not the right length. So we return an error ASAP, this way
- * evalGenericCommand() can be implemented without string length
- * sanity check */
- addReplyErrorObject(c, shared.noscripterr);
- return;
- }
- if (!(c->flags & CLIENT_LUA_DEBUG))
- evalGenericCommand(c,1);
- else {
- addReplyError(c,"Please use EVAL instead of EVALSHA for debugging");
- return;
- }
-}
-
-void evalShaRoCommand(client *c) {
- evalShaCommand(c);
-}
-
-void scriptCommand(client *c) {
- if (c->argc == 2 && !strcasecmp(c->argv[1]->ptr,"help")) {
- const char *help[] = {
-"DEBUG (YES|SYNC|NO)",
-" Set the debug mode for subsequent scripts executed.",
-"EXISTS <sha1> [<sha1> ...]",
-" Return information about the existence of the scripts in the script cache.",
-"FLUSH [ASYNC|SYNC]",
-" Flush the Lua scripts cache. Very dangerous on replicas.",
-" When called without the optional mode argument, the behavior is determined by the",
-" lazyfree-lazy-user-flush configuration directive. Valid modes are:",
-" * ASYNC: Asynchronously flush the scripts cache.",
-" * SYNC: Synchronously flush the scripts cache.",
-"KILL",
-" Kill the currently executing Lua script.",
-"LOAD <script>",
-" Load a script into the scripts cache without executing it.",
-NULL
- };
- addReplyHelp(c, help);
- } else if (c->argc >= 2 && !strcasecmp(c->argv[1]->ptr,"flush")) {
- int async = 0;
- if (c->argc == 3 && !strcasecmp(c->argv[2]->ptr,"sync")) {
- async = 0;
- } else if (c->argc == 3 && !strcasecmp(c->argv[2]->ptr,"async")) {
- async = 1;
- } else if (c->argc == 2) {
- async = server.lazyfree_lazy_user_flush ? 1 : 0;
- } else {
- addReplyError(c,"SCRIPT FLUSH only support SYNC|ASYNC option");
- return;
- }
- scriptingReset(async);
- addReply(c,shared.ok);
- replicationScriptCacheFlush();
- server.dirty++; /* Propagating this command is a good idea. */
- } else if (c->argc >= 2 && !strcasecmp(c->argv[1]->ptr,"exists")) {
- int j;
-
- addReplyArrayLen(c, c->argc-2);
- for (j = 2; j < c->argc; j++) {
- if (dictFind(server.lua_scripts,c->argv[j]->ptr))
- addReply(c,shared.cone);
- else
- addReply(c,shared.czero);
- }
- } else if (c->argc == 3 && !strcasecmp(c->argv[1]->ptr,"load")) {
- sds sha = luaCreateFunction(c,server.lua,c->argv[2]);
- if (sha == NULL) return; /* The error was sent by luaCreateFunction(). */
- addReplyBulkCBuffer(c,sha,40);
- forceCommandPropagation(c,PROPAGATE_REPL|PROPAGATE_AOF);
- } else if (c->argc == 2 && !strcasecmp(c->argv[1]->ptr,"kill")) {
- if (server.lua_caller == NULL) {
- addReplyError(c,"-NOTBUSY No scripts in execution right now.");
- } else if (server.lua_caller->flags & CLIENT_MASTER) {
- addReplyError(c,"-UNKILLABLE The busy script was sent by a master instance in the context of replication and cannot be killed.");
- } else if (server.lua_write_dirty) {
- addReplyError(c,"-UNKILLABLE Sorry the script already executed write commands against the dataset. You can either wait the script termination or kill the server in a hard way using the SHUTDOWN NOSAVE command.");
- } else {
- server.lua_kill = 1;
- addReply(c,shared.ok);
- }
- } else if (c->argc == 3 && !strcasecmp(c->argv[1]->ptr,"debug")) {
- if (clientHasPendingReplies(c)) {
- addReplyError(c,"SCRIPT DEBUG must be called outside a pipeline");
- return;
- }
- if (!strcasecmp(c->argv[2]->ptr,"no")) {
- ldbDisable(c);
- addReply(c,shared.ok);
- } else if (!strcasecmp(c->argv[2]->ptr,"yes")) {
- ldbEnable(c);
- addReply(c,shared.ok);
- } else if (!strcasecmp(c->argv[2]->ptr,"sync")) {
- ldbEnable(c);
- addReply(c,shared.ok);
- c->flags |= CLIENT_LUA_DEBUG_SYNC;
- } else {
- addReplyError(c,"Use SCRIPT DEBUG YES/SYNC/NO");
- return;
- }
- } else {
- addReplySubcommandSyntaxError(c);
- }
-}
-
-/* ---------------------------------------------------------------------------
- * LDB: Redis Lua debugging facilities
- * ------------------------------------------------------------------------- */
-
-/* Initialize Lua debugger data structures. */
-void ldbInit(void) {
- ldb.conn = NULL;
- ldb.active = 0;
- ldb.logs = listCreate();
- listSetFreeMethod(ldb.logs,(void (*)(void*))sdsfree);
- ldb.children = listCreate();
- ldb.src = NULL;
- ldb.lines = 0;
- ldb.cbuf = sdsempty();
-}
-
-/* Remove all the pending messages in the specified list. */
-void ldbFlushLog(list *log) {
- listNode *ln;
-
- while((ln = listFirst(log)) != NULL)
- listDelNode(log,ln);
-}
-
-/* Enable debug mode of Lua scripts for this client. */
-void ldbEnable(client *c) {
- c->flags |= CLIENT_LUA_DEBUG;
- ldbFlushLog(ldb.logs);
- ldb.conn = c->conn;
- ldb.step = 1;
- ldb.bpcount = 0;
- ldb.luabp = 0;
- sdsfree(ldb.cbuf);
- ldb.cbuf = sdsempty();
- ldb.maxlen = LDB_MAX_LEN_DEFAULT;
- ldb.maxlen_hint_sent = 0;
-}
-
-/* Exit debugging mode from the POV of client. This function is not enough
- * to properly shut down a client debugging session, see ldbEndSession()
- * for more information. */
-void ldbDisable(client *c) {
- c->flags &= ~(CLIENT_LUA_DEBUG|CLIENT_LUA_DEBUG_SYNC);
-}
-
-/* Append a log entry to the specified LDB log. */
-void ldbLog(sds entry) {
- listAddNodeTail(ldb.logs,entry);
-}
-
-/* A version of ldbLog() which prevents producing logs greater than
- * ldb.maxlen. The first time the limit is reached a hint is generated
- * to inform the user that reply trimming can be disabled using the
- * debugger "maxlen" command. */
-void ldbLogWithMaxLen(sds entry) {
- int trimmed = 0;
- if (ldb.maxlen && sdslen(entry) > ldb.maxlen) {
- sdsrange(entry,0,ldb.maxlen-1);
- entry = sdscatlen(entry," ...",4);
- trimmed = 1;
- }
- ldbLog(entry);
- if (trimmed && ldb.maxlen_hint_sent == 0) {
- ldb.maxlen_hint_sent = 1;
- ldbLog(sdsnew(
- "<hint> The above reply was trimmed. Use 'maxlen 0' to disable trimming."));
- }
-}
-
-/* Send ldb.logs to the debugging client as a multi-bulk reply
- * consisting of simple strings. Log entries which include newlines have them
- * replaced with spaces. The entries sent are also consumed. */
-void ldbSendLogs(void) {
- sds proto = sdsempty();
- proto = sdscatfmt(proto,"*%i\r\n", (int)listLength(ldb.logs));
- while(listLength(ldb.logs)) {
- listNode *ln = listFirst(ldb.logs);
- proto = sdscatlen(proto,"+",1);
- sdsmapchars(ln->value,"\r\n"," ",2);
- proto = sdscatsds(proto,ln->value);
- proto = sdscatlen(proto,"\r\n",2);
- listDelNode(ldb.logs,ln);
- }
- if (connWrite(ldb.conn,proto,sdslen(proto)) == -1) {
- /* Avoid warning. We don't check the return value of write()
- * since the next read() will catch the I/O error and will
- * close the debugging session. */
- }
- sdsfree(proto);
-}
-
-/* Start a debugging session before calling EVAL implementation.
- * The technique we use is to capture the client socket file descriptor,
- * in order to perform direct I/O with it from within Lua hooks. This
- * way we don't have to re-enter Redis in order to handle I/O.
- *
- * The function returns 1 if the caller should proceed to call EVAL,
- * and 0 if instead the caller should abort the operation (this happens
- * for the parent in a forked session, since it's up to the children
- * to continue, or when fork returned an error).
- *
- * The caller should call ldbEndSession() only if ldbStartSession()
- * returned 1. */
-int ldbStartSession(client *c) {
- ldb.forked = (c->flags & CLIENT_LUA_DEBUG_SYNC) == 0;
- if (ldb.forked) {
- pid_t cp = redisFork(CHILD_TYPE_LDB);
- if (cp == -1) {
- addReplyErrorFormat(c,"Fork() failed: can't run EVAL in debugging mode: %s", strerror(errno));
- return 0;
- } else if (cp == 0) {
- /* Child. Let's ignore important signals handled by the parent. */
- struct sigaction act;
- sigemptyset(&act.sa_mask);
- act.sa_flags = 0;
- act.sa_handler = SIG_IGN;
- sigaction(SIGTERM, &act, NULL);
- sigaction(SIGINT, &act, NULL);
-
- /* Log the creation of the child and close the listening
- * socket to make sure if the parent crashes a reset is sent
- * to the clients. */
- serverLog(LL_WARNING,"Redis forked for debugging eval");
- } else {
- /* Parent */
- listAddNodeTail(ldb.children,(void*)(unsigned long)cp);
- freeClientAsync(c); /* Close the client in the parent side. */
- return 0;
- }
- } else {
- serverLog(LL_WARNING,
- "Redis synchronous debugging eval session started");
- }
-
- /* Setup our debugging session. */
- connBlock(ldb.conn);
- connSendTimeout(ldb.conn,5000);
- ldb.active = 1;
-
- /* First argument of EVAL is the script itself. We split it into different
- * lines since this is the way the debugger accesses the source code. */
- sds srcstring = sdsdup(c->argv[1]->ptr);
- size_t srclen = sdslen(srcstring);
- while(srclen && (srcstring[srclen-1] == '\n' ||
- srcstring[srclen-1] == '\r'))
- {
- srcstring[--srclen] = '\0';
- }
- sdssetlen(srcstring,srclen);
- ldb.src = sdssplitlen(srcstring,sdslen(srcstring),"\n",1,&ldb.lines);
- sdsfree(srcstring);
- return 1;
-}
-
-/* End a debugging session after the EVAL call with debugging enabled
- * returned. */
-void ldbEndSession(client *c) {
- /* Emit the remaining logs and an <endsession> mark. */
- ldbLog(sdsnew("<endsession>"));
- ldbSendLogs();
-
- /* If it's a fork()ed session, we just exit. */
- if (ldb.forked) {
- writeToClient(c,0);
- serverLog(LL_WARNING,"Lua debugging session child exiting");
- exitFromChild(0);
- } else {
- serverLog(LL_WARNING,
- "Redis synchronous debugging eval session ended");
- }
-
- /* Otherwise let's restore client's state. */
- connNonBlock(ldb.conn);
- connSendTimeout(ldb.conn,0);
-
- /* Close the client connection after sending the final EVAL reply
- * in order to signal the end of the debugging session. */
- c->flags |= CLIENT_CLOSE_AFTER_REPLY;
-
- /* Cleanup. */
- sdsfreesplitres(ldb.src,ldb.lines);
- ldb.lines = 0;
- ldb.active = 0;
-}
-
-/* If the specified pid is among the list of children spawned for
- * forked debugging sessions, it is removed from the children list.
- * If the pid was found non-zero is returned. */
-int ldbRemoveChild(pid_t pid) {
- listNode *ln = listSearchKey(ldb.children,(void*)(unsigned long)pid);
- if (ln) {
- listDelNode(ldb.children,ln);
- return 1;
- }
- return 0;
-}
-
-/* Return the number of children we still did not receive termination
- * acknowledge via wait() in the parent process. */
-int ldbPendingChildren(void) {
- return listLength(ldb.children);
-}
-
-/* Kill all the forked sessions. */
-void ldbKillForkedSessions(void) {
- listIter li;
- listNode *ln;
-
- listRewind(ldb.children,&li);
- while((ln = listNext(&li))) {
- pid_t pid = (unsigned long) ln->value;
- serverLog(LL_WARNING,"Killing debugging session %ld",(long)pid);
- kill(pid,SIGKILL);
- }
- listRelease(ldb.children);
- ldb.children = listCreate();
-}
-
-/* Wrapper for EVAL / EVALSHA that enables debugging, and makes sure
- * that when EVAL returns, whatever happened, the session is ended. */
-void evalGenericCommandWithDebugging(client *c, int evalsha) {
- if (ldbStartSession(c)) {
- evalGenericCommand(c,evalsha);
- ldbEndSession(c);
- } else {
- ldbDisable(c);
- }
-}
-
-/* Return a pointer to ldb.src source code line, considering line to be
- * one-based, and returning a special string for out of range lines. */
-char *ldbGetSourceLine(int line) {
- int idx = line-1;
- if (idx < 0 || idx >= ldb.lines) return "<out of range source code line>";
- return ldb.src[idx];
-}
-
-/* Return true if there is a breakpoint in the specified line. */
-int ldbIsBreakpoint(int line) {
- int j;
-
- for (j = 0; j < ldb.bpcount; j++)
- if (ldb.bp[j] == line) return 1;
- return 0;
-}
-
-/* Add the specified breakpoint. Ignore it if we already reached the max.
- * Returns 1 if the breakpoint was added (or was already set). 0 if there is
- * no space for the breakpoint or if the line is invalid. */
-int ldbAddBreakpoint(int line) {
- if (line <= 0 || line > ldb.lines) return 0;
- if (!ldbIsBreakpoint(line) && ldb.bpcount != LDB_BREAKPOINTS_MAX) {
- ldb.bp[ldb.bpcount++] = line;
- return 1;
- }
- return 0;
-}
-
-/* Remove the specified breakpoint, returning 1 if the operation was
- * performed or 0 if there was no such breakpoint. */
-int ldbDelBreakpoint(int line) {
- int j;
-
- for (j = 0; j < ldb.bpcount; j++) {
- if (ldb.bp[j] == line) {
- ldb.bpcount--;
- memmove(ldb.bp+j,ldb.bp+j+1,ldb.bpcount-j);
- return 1;
- }
- }
- return 0;
-}
-
-/* Expect a valid multi-bulk command in the debugging client query buffer.
- * On success the command is parsed and returned as an array of SDS strings,
- * otherwise NULL is returned and there is to read more buffer. */
-sds *ldbReplParseCommand(int *argcp, char** err) {
- static char* protocol_error = "protocol error";
- sds *argv = NULL;
- int argc = 0;
- if (sdslen(ldb.cbuf) == 0) return NULL;
-
- /* Working on a copy is simpler in this case. We can modify it freely
- * for the sake of simpler parsing. */
- sds copy = sdsdup(ldb.cbuf);
- char *p = copy;
-
- /* This Redis protocol parser is a joke... just the simplest thing that
- * works in this context. It is also very forgiving regarding broken
- * protocol. */
-
- /* Seek and parse *<count>\r\n. */
- p = strchr(p,'*'); if (!p) goto protoerr;
- char *plen = p+1; /* Multi bulk len pointer. */
- p = strstr(p,"\r\n"); if (!p) goto keep_reading;
- *p = '\0'; p += 2;
- *argcp = atoi(plen);
- if (*argcp <= 0 || *argcp > 1024) goto protoerr;
-
- /* Parse each argument. */
- argv = zmalloc(sizeof(sds)*(*argcp));
- argc = 0;
- while(argc < *argcp) {
- /* reached the end but there should be more data to read */
- if (*p == '\0') goto keep_reading;
-
- if (*p != '$') goto protoerr;
- plen = p+1; /* Bulk string len pointer. */
- p = strstr(p,"\r\n"); if (!p) goto keep_reading;
- *p = '\0'; p += 2;
- int slen = atoi(plen); /* Length of this arg. */
- if (slen <= 0 || slen > 1024) goto protoerr;
- if ((size_t)(p + slen + 2 - copy) > sdslen(copy) ) goto keep_reading;
- argv[argc++] = sdsnewlen(p,slen);
- p += slen; /* Skip the already parsed argument. */
- if (p[0] != '\r' || p[1] != '\n') goto protoerr;
- p += 2; /* Skip \r\n. */
- }
- sdsfree(copy);
- return argv;
-
-protoerr:
- *err = protocol_error;
-keep_reading:
- sdsfreesplitres(argv,argc);
- sdsfree(copy);
- return NULL;
-}
-
-/* Log the specified line in the Lua debugger output. */
-void ldbLogSourceLine(int lnum) {
- char *line = ldbGetSourceLine(lnum);
- char *prefix;
- int bp = ldbIsBreakpoint(lnum);
- int current = ldb.currentline == lnum;
-
- if (current && bp)
- prefix = "->#";
- else if (current)
- prefix = "-> ";
- else if (bp)
- prefix = " #";
- else
- prefix = " ";
- sds thisline = sdscatprintf(sdsempty(),"%s%-3d %s", prefix, lnum, line);
- ldbLog(thisline);
-}
-
-/* Implement the "list" command of the Lua debugger. If around is 0
- * the whole file is listed, otherwise only a small portion of the file
- * around the specified line is shown. When a line number is specified
- * the amount of context (lines before/after) is specified via the
- * 'context' argument. */
-void ldbList(int around, int context) {
- int j;
-
- for (j = 1; j <= ldb.lines; j++) {
- if (around != 0 && abs(around-j) > context) continue;
- ldbLogSourceLine(j);
- }
-}
-
-/* Append a human readable representation of the Lua value at position 'idx'
- * on the stack of the 'lua' state, to the SDS string passed as argument.
- * The new SDS string with the represented value attached is returned.
- * Used in order to implement ldbLogStackValue().
- *
- * The element is not automatically removed from the stack, nor it is
- * converted to a different type. */
-#define LDB_MAX_VALUES_DEPTH (LUA_MINSTACK/2)
-sds ldbCatStackValueRec(sds s, lua_State *lua, int idx, int level) {
- int t = lua_type(lua,idx);
-
- if (level++ == LDB_MAX_VALUES_DEPTH)
- return sdscat(s,"<max recursion level reached! Nested table?>");
-
- switch(t) {
- case LUA_TSTRING:
- {
- size_t strl;
- char *strp = (char*)lua_tolstring(lua,idx,&strl);
- s = sdscatrepr(s,strp,strl);
- }
- break;
- case LUA_TBOOLEAN:
- s = sdscat(s,lua_toboolean(lua,idx) ? "true" : "false");
- break;
- case LUA_TNUMBER:
- s = sdscatprintf(s,"%g",(double)lua_tonumber(lua,idx));
- break;
- case LUA_TNIL:
- s = sdscatlen(s,"nil",3);
- break;
- case LUA_TTABLE:
- {
- int expected_index = 1; /* First index we expect in an array. */
- int is_array = 1; /* Will be set to null if check fails. */
- /* Note: we create two representations at the same time, one
- * assuming the table is an array, one assuming it is not. At the
- * end we know what is true and select the right one. */
- sds repr1 = sdsempty();
- sds repr2 = sdsempty();
- lua_pushnil(lua); /* The first key to start the iteration is nil. */
- while (lua_next(lua,idx-1)) {
- /* Test if so far the table looks like an array. */
- if (is_array &&
- (lua_type(lua,-2) != LUA_TNUMBER ||
- lua_tonumber(lua,-2) != expected_index)) is_array = 0;
- /* Stack now: table, key, value */
- /* Array repr. */
- repr1 = ldbCatStackValueRec(repr1,lua,-1,level);
- repr1 = sdscatlen(repr1,"; ",2);
- /* Full repr. */
- repr2 = sdscatlen(repr2,"[",1);
- repr2 = ldbCatStackValueRec(repr2,lua,-2,level);
- repr2 = sdscatlen(repr2,"]=",2);
- repr2 = ldbCatStackValueRec(repr2,lua,-1,level);
- repr2 = sdscatlen(repr2,"; ",2);
- lua_pop(lua,1); /* Stack: table, key. Ready for next iteration. */
- expected_index++;
- }
- /* Strip the last " ;" from both the representations. */
- if (sdslen(repr1)) sdsrange(repr1,0,-3);
- if (sdslen(repr2)) sdsrange(repr2,0,-3);
- /* Select the right one and discard the other. */
- s = sdscatlen(s,"{",1);
- s = sdscatsds(s,is_array ? repr1 : repr2);
- s = sdscatlen(s,"}",1);
- sdsfree(repr1);
- sdsfree(repr2);
- }
- break;
- case LUA_TFUNCTION:
- case LUA_TUSERDATA:
- case LUA_TTHREAD:
- case LUA_TLIGHTUSERDATA:
- {
- const void *p = lua_topointer(lua,idx);
- char *typename = "unknown";
- if (t == LUA_TFUNCTION) typename = "function";
- else if (t == LUA_TUSERDATA) typename = "userdata";
- else if (t == LUA_TTHREAD) typename = "thread";
- else if (t == LUA_TLIGHTUSERDATA) typename = "light-userdata";
- s = sdscatprintf(s,"\"%s@%p\"",typename,p);
- }
- break;
- default:
- s = sdscat(s,"\"<unknown-lua-type>\"");
- break;
- }
- return s;
-}
-
-/* Higher level wrapper for ldbCatStackValueRec() that just uses an initial
- * recursion level of '0'. */
-sds ldbCatStackValue(sds s, lua_State *lua, int idx) {
- return ldbCatStackValueRec(s,lua,idx,0);
-}
-
-/* Produce a debugger log entry representing the value of the Lua object
- * currently on the top of the stack. The element is not popped nor modified.
- * Check ldbCatStackValue() for the actual implementation. */
-void ldbLogStackValue(lua_State *lua, char *prefix) {
- sds s = sdsnew(prefix);
- s = ldbCatStackValue(s,lua,-1);
- ldbLogWithMaxLen(s);
-}
-
-char *ldbRedisProtocolToHuman_Int(sds *o, char *reply);
-char *ldbRedisProtocolToHuman_Bulk(sds *o, char *reply);
-char *ldbRedisProtocolToHuman_Status(sds *o, char *reply);
-char *ldbRedisProtocolToHuman_MultiBulk(sds *o, char *reply);
-char *ldbRedisProtocolToHuman_Set(sds *o, char *reply);
-char *ldbRedisProtocolToHuman_Map(sds *o, char *reply);
-char *ldbRedisProtocolToHuman_Null(sds *o, char *reply);
-char *ldbRedisProtocolToHuman_Bool(sds *o, char *reply);
-char *ldbRedisProtocolToHuman_Double(sds *o, char *reply);
-
-/* Get Redis protocol from 'reply' and appends it in human readable form to
- * the passed SDS string 'o'.
- *
- * Note that the SDS string is passed by reference (pointer of pointer to
- * char*) so that we can return a modified pointer, as for SDS semantics. */
-char *ldbRedisProtocolToHuman(sds *o, char *reply) {
- char *p = reply;
- switch(*p) {
- case ':': p = ldbRedisProtocolToHuman_Int(o,reply); break;
- case '$': p = ldbRedisProtocolToHuman_Bulk(o,reply); break;
- case '+': p = ldbRedisProtocolToHuman_Status(o,reply); break;
- case '-': p = ldbRedisProtocolToHuman_Status(o,reply); break;
- case '*': p = ldbRedisProtocolToHuman_MultiBulk(o,reply); break;
- case '~': p = ldbRedisProtocolToHuman_Set(o,reply); break;
- case '%': p = ldbRedisProtocolToHuman_Map(o,reply); break;
- case '_': p = ldbRedisProtocolToHuman_Null(o,reply); break;
- case '#': p = ldbRedisProtocolToHuman_Bool(o,reply); break;
- case ',': p = ldbRedisProtocolToHuman_Double(o,reply); break;
- }
- return p;
-}
-
-/* The following functions are helpers for ldbRedisProtocolToHuman(), each
- * take care of a given Redis return type. */
-
-char *ldbRedisProtocolToHuman_Int(sds *o, char *reply) {
- char *p = strchr(reply+1,'\r');
- *o = sdscatlen(*o,reply+1,p-reply-1);
- return p+2;
-}
-
-char *ldbRedisProtocolToHuman_Bulk(sds *o, char *reply) {
- char *p = strchr(reply+1,'\r');
- long long bulklen;
-
- string2ll(reply+1,p-reply-1,&bulklen);
- if (bulklen == -1) {
- *o = sdscatlen(*o,"NULL",4);
- return p+2;
- } else {
- *o = sdscatrepr(*o,p+2,bulklen);
- return p+2+bulklen+2;
- }
-}
-
-char *ldbRedisProtocolToHuman_Status(sds *o, char *reply) {
- char *p = strchr(reply+1,'\r');
-
- *o = sdscatrepr(*o,reply,p-reply);
- return p+2;
-}
-
-char *ldbRedisProtocolToHuman_MultiBulk(sds *o, char *reply) {
- char *p = strchr(reply+1,'\r');
- long long mbulklen;
- int j = 0;
-
- string2ll(reply+1,p-reply-1,&mbulklen);
- p += 2;
- if (mbulklen == -1) {
- *o = sdscatlen(*o,"NULL",4);
- return p;
- }
- *o = sdscatlen(*o,"[",1);
- for (j = 0; j < mbulklen; j++) {
- p = ldbRedisProtocolToHuman(o,p);
- if (j != mbulklen-1) *o = sdscatlen(*o,",",1);
- }
- *o = sdscatlen(*o,"]",1);
- return p;
-}
-
-char *ldbRedisProtocolToHuman_Set(sds *o, char *reply) {
- char *p = strchr(reply+1,'\r');
- long long mbulklen;
- int j = 0;
-
- string2ll(reply+1,p-reply-1,&mbulklen);
- p += 2;
- *o = sdscatlen(*o,"~(",2);
- for (j = 0; j < mbulklen; j++) {
- p = ldbRedisProtocolToHuman(o,p);
- if (j != mbulklen-1) *o = sdscatlen(*o,",",1);
- }
- *o = sdscatlen(*o,")",1);
- return p;
-}
-
-char *ldbRedisProtocolToHuman_Map(sds *o, char *reply) {
- char *p = strchr(reply+1,'\r');
- long long mbulklen;
- int j = 0;
-
- string2ll(reply+1,p-reply-1,&mbulklen);
- p += 2;
- *o = sdscatlen(*o,"{",1);
- for (j = 0; j < mbulklen; j++) {
- p = ldbRedisProtocolToHuman(o,p);
- *o = sdscatlen(*o," => ",4);
- p = ldbRedisProtocolToHuman(o,p);
- if (j != mbulklen-1) *o = sdscatlen(*o,",",1);
- }
- *o = sdscatlen(*o,"}",1);
- return p;
-}
-
-char *ldbRedisProtocolToHuman_Null(sds *o, char *reply) {
- char *p = strchr(reply+1,'\r');
- *o = sdscatlen(*o,"(null)",6);
- return p+2;
-}
-
-char *ldbRedisProtocolToHuman_Bool(sds *o, char *reply) {
- char *p = strchr(reply+1,'\r');
- if (reply[1] == 't')
- *o = sdscatlen(*o,"#true",5);
- else
- *o = sdscatlen(*o,"#false",6);
- return p+2;
-}
-
-char *ldbRedisProtocolToHuman_Double(sds *o, char *reply) {
- char *p = strchr(reply+1,'\r');
- *o = sdscatlen(*o,"(double) ",9);
- *o = sdscatlen(*o,reply+1,p-reply-1);
- return p+2;
-}
-
-/* Log a Redis reply as debugger output, in a human readable format.
- * If the resulting string is longer than 'len' plus a few more chars
- * used as prefix, it gets truncated. */
-void ldbLogRedisReply(char *reply) {
- sds log = sdsnew("<reply> ");
- ldbRedisProtocolToHuman(&log,reply);
- ldbLogWithMaxLen(log);
-}
-
-/* Implements the "print <var>" command of the Lua debugger. It scans for Lua
- * var "varname" starting from the current stack frame up to the top stack
- * frame. The first matching variable is printed. */
-void ldbPrint(lua_State *lua, char *varname) {
- lua_Debug ar;
-
- int l = 0; /* Stack level. */
- while (lua_getstack(lua,l,&ar) != 0) {
- l++;
- const char *name;
- int i = 1; /* Variable index. */
- while((name = lua_getlocal(lua,&ar,i)) != NULL) {
- i++;
- if (strcmp(varname,name) == 0) {
- ldbLogStackValue(lua,"<value> ");
- lua_pop(lua,1);
- return;
- } else {
- lua_pop(lua,1); /* Discard the var name on the stack. */
- }
- }
- }
-
- /* Let's try with global vars in two selected cases */
- if (!strcmp(varname,"ARGV") || !strcmp(varname,"KEYS")) {
- lua_getglobal(lua, varname);
- ldbLogStackValue(lua,"<value> ");
- lua_pop(lua,1);
- } else {
- ldbLog(sdsnew("No such variable."));
- }
-}
-
-/* Implements the "print" command (without arguments) of the Lua debugger.
- * Prints all the variables in the current stack frame. */
-void ldbPrintAll(lua_State *lua) {
- lua_Debug ar;
- int vars = 0;
-
- if (lua_getstack(lua,0,&ar) != 0) {
- const char *name;
- int i = 1; /* Variable index. */
- while((name = lua_getlocal(lua,&ar,i)) != NULL) {
- i++;
- if (!strstr(name,"(*temporary)")) {
- sds prefix = sdscatprintf(sdsempty(),"<value> %s = ",name);
- ldbLogStackValue(lua,prefix);
- sdsfree(prefix);
- vars++;
- }
- lua_pop(lua,1);
- }
- }
-
- if (vars == 0) {
- ldbLog(sdsnew("No local variables in the current context."));
- }
-}
-
-/* Implements the break command to list, add and remove breakpoints. */
-void ldbBreak(sds *argv, int argc) {
- if (argc == 1) {
- if (ldb.bpcount == 0) {
- ldbLog(sdsnew("No breakpoints set. Use 'b <line>' to add one."));
- return;
- } else {
- ldbLog(sdscatfmt(sdsempty(),"%i breakpoints set:",ldb.bpcount));
- int j;
- for (j = 0; j < ldb.bpcount; j++)
- ldbLogSourceLine(ldb.bp[j]);
- }
- } else {
- int j;
- for (j = 1; j < argc; j++) {
- char *arg = argv[j];
- long line;
- if (!string2l(arg,sdslen(arg),&line)) {
- ldbLog(sdscatfmt(sdsempty(),"Invalid argument:'%s'",arg));
- } else {
- if (line == 0) {
- ldb.bpcount = 0;
- ldbLog(sdsnew("All breakpoints removed."));
- } else if (line > 0) {
- if (ldb.bpcount == LDB_BREAKPOINTS_MAX) {
- ldbLog(sdsnew("Too many breakpoints set."));
- } else if (ldbAddBreakpoint(line)) {
- ldbList(line,1);
- } else {
- ldbLog(sdsnew("Wrong line number."));
- }
- } else if (line < 0) {
- if (ldbDelBreakpoint(-line))
- ldbLog(sdsnew("Breakpoint removed."));
- else
- ldbLog(sdsnew("No breakpoint in the specified line."));
- }
- }
- }
- }
-}
-
-/* Implements the Lua debugger "eval" command. It just compiles the user
- * passed fragment of code and executes it, showing the result left on
- * the stack. */
-void ldbEval(lua_State *lua, sds *argv, int argc) {
- /* Glue the script together if it is composed of multiple arguments. */
- sds code = sdsjoinsds(argv+1,argc-1," ",1);
- sds expr = sdscatsds(sdsnew("return "),code);
-
- /* Try to compile it as an expression, prepending "return ". */
- if (luaL_loadbuffer(lua,expr,sdslen(expr),"@ldb_eval")) {
- lua_pop(lua,1);
- /* Failed? Try as a statement. */
- if (luaL_loadbuffer(lua,code,sdslen(code),"@ldb_eval")) {
- ldbLog(sdscatfmt(sdsempty(),"<error> %s",lua_tostring(lua,-1)));
- lua_pop(lua,1);
- sdsfree(code);
- sdsfree(expr);
- return;
- }
- }
-
- /* Call it. */
- sdsfree(code);
- sdsfree(expr);
- if (lua_pcall(lua,0,1,0)) {
- ldbLog(sdscatfmt(sdsempty(),"<error> %s",lua_tostring(lua,-1)));
- lua_pop(lua,1);
- return;
- }
- ldbLogStackValue(lua,"<retval> ");
- lua_pop(lua,1);
-}
-
-/* Implement the debugger "redis" command. We use a trick in order to make
- * the implementation very simple: we just call the Lua redis.call() command
- * implementation, with ldb.step enabled, so as a side effect the Redis command
- * and its reply are logged. */
-void ldbRedis(lua_State *lua, sds *argv, int argc) {
- int j, saved_rc = server.lua_replicate_commands;
-
- if (!lua_checkstack(lua, argc + 1)) {
- /* Increase the Lua stack if needed to make sure there is enough room
- * to push 'argc + 1' elements to the stack. On failure, return error.
- * Notice that we need, in worst case, 'argc + 1' elements because we push all the arguments
- * given by the user (without the first argument) and we also push the 'redis' global table and
- * 'redis.call' function so:
- * (1 (redis table)) + (1 (redis.call function)) + (argc - 1 (all arguments without the first)) = argc + 1*/
- ldbLogRedisReply("max lua stack reached");
- return;
- }
-
- lua_getglobal(lua,"redis");
- lua_pushstring(lua,"call");
- lua_gettable(lua,-2); /* Stack: redis, redis.call */
- for (j = 1; j < argc; j++)
- lua_pushlstring(lua,argv[j],sdslen(argv[j]));
- ldb.step = 1; /* Force redis.call() to log. */
- server.lua_replicate_commands = 1;
- lua_pcall(lua,argc-1,1,0); /* Stack: redis, result */
- ldb.step = 0; /* Disable logging. */
- server.lua_replicate_commands = saved_rc;
- lua_pop(lua,2); /* Discard the result and clean the stack. */
-}
-
-/* Implements "trace" command of the Lua debugger. It just prints a backtrace
- * querying Lua starting from the current callframe back to the outer one. */
-void ldbTrace(lua_State *lua) {
- lua_Debug ar;
- int level = 0;
-
- while(lua_getstack(lua,level,&ar)) {
- lua_getinfo(lua,"Snl",&ar);
- if(strstr(ar.short_src,"user_script") != NULL) {
- ldbLog(sdscatprintf(sdsempty(),"%s %s:",
- (level == 0) ? "In" : "From",
- ar.name ? ar.name : "top level"));
- ldbLogSourceLine(ar.currentline);
- }
- level++;
- }
- if (level == 0) {
- ldbLog(sdsnew("<error> Can't retrieve Lua stack."));
- }
-}
-
-/* Implements the debugger "maxlen" command. It just queries or sets the
- * ldb.maxlen variable. */
-void ldbMaxlen(sds *argv, int argc) {
- if (argc == 2) {
- int newval = atoi(argv[1]);
- ldb.maxlen_hint_sent = 1; /* User knows about this command. */
- if (newval != 0 && newval <= 60) newval = 60;
- ldb.maxlen = newval;
- }
- if (ldb.maxlen) {
- ldbLog(sdscatprintf(sdsempty(),"<value> replies are truncated at %d bytes.",(int)ldb.maxlen));
- } else {
- ldbLog(sdscatprintf(sdsempty(),"<value> replies are unlimited."));
- }
-}
-
-/* Read debugging commands from client.
- * Return C_OK if the debugging session is continuing, otherwise
- * C_ERR if the client closed the connection or is timing out. */
-int ldbRepl(lua_State *lua) {
- sds *argv;
- int argc;
- char* err = NULL;
-
- /* We continue processing commands until a command that should return
- * to the Lua interpreter is found. */
- while(1) {
- while((argv = ldbReplParseCommand(&argc, &err)) == NULL) {
- char buf[1024];
- if (err) {
- lua_pushstring(lua, err);
- lua_error(lua);
- }
- int nread = connRead(ldb.conn,buf,sizeof(buf));
- if (nread <= 0) {
- /* Make sure the script runs without user input since the
- * client is no longer connected. */
- ldb.step = 0;
- ldb.bpcount = 0;
- return C_ERR;
- }
- ldb.cbuf = sdscatlen(ldb.cbuf,buf,nread);
- /* after 1M we will exit with an error
- * so that the client will not blow the memory
- */
- if (sdslen(ldb.cbuf) > 1<<20) {
- sdsfree(ldb.cbuf);
- ldb.cbuf = sdsempty();
- lua_pushstring(lua, "max client buffer reached");
- lua_error(lua);
- }
- }
-
- /* Flush the old buffer. */
- sdsfree(ldb.cbuf);
- ldb.cbuf = sdsempty();
-
- /* Execute the command. */
- if (!strcasecmp(argv[0],"h") || !strcasecmp(argv[0],"help")) {
-ldbLog(sdsnew("Redis Lua debugger help:"));
-ldbLog(sdsnew("[h]elp Show this help."));
-ldbLog(sdsnew("[s]tep Run current line and stop again."));
-ldbLog(sdsnew("[n]ext Alias for step."));
-ldbLog(sdsnew("[c]ontinue Run till next breakpoint."));
-ldbLog(sdsnew("[l]ist List source code around current line."));
-ldbLog(sdsnew("[l]ist [line] List source code around [line]."));
-ldbLog(sdsnew(" line = 0 means: current position."));
-ldbLog(sdsnew("[l]ist [line] [ctx] In this form [ctx] specifies how many lines"));
-ldbLog(sdsnew(" to show before/after [line]."));
-ldbLog(sdsnew("[w]hole List all source code. Alias for 'list 1 1000000'."));
-ldbLog(sdsnew("[p]rint Show all the local variables."));
-ldbLog(sdsnew("[p]rint <var> Show the value of the specified variable."));
-ldbLog(sdsnew(" Can also show global vars KEYS and ARGV."));
-ldbLog(sdsnew("[b]reak Show all breakpoints."));
-ldbLog(sdsnew("[b]reak <line> Add a breakpoint to the specified line."));
-ldbLog(sdsnew("[b]reak -<line> Remove breakpoint from the specified line."));
-ldbLog(sdsnew("[b]reak 0 Remove all breakpoints."));
-ldbLog(sdsnew("[t]race Show a backtrace."));
-ldbLog(sdsnew("[e]val <code> Execute some Lua code (in a different callframe)."));
-ldbLog(sdsnew("[r]edis <cmd> Execute a Redis command."));
-ldbLog(sdsnew("[m]axlen [len] Trim logged Redis replies and Lua var dumps to len."));
-ldbLog(sdsnew(" Specifying zero as <len> means unlimited."));
-ldbLog(sdsnew("[a]bort Stop the execution of the script. In sync"));
-ldbLog(sdsnew(" mode dataset changes will be retained."));
-ldbLog(sdsnew(""));
-ldbLog(sdsnew("Debugger functions you can call from Lua scripts:"));
-ldbLog(sdsnew("redis.debug() Produce logs in the debugger console."));
-ldbLog(sdsnew("redis.breakpoint() Stop execution like if there was a breakpoint in the"));
-ldbLog(sdsnew(" next line of code."));
- ldbSendLogs();
- } else if (!strcasecmp(argv[0],"s") || !strcasecmp(argv[0],"step") ||
- !strcasecmp(argv[0],"n") || !strcasecmp(argv[0],"next")) {
- ldb.step = 1;
- break;
- } else if (!strcasecmp(argv[0],"c") || !strcasecmp(argv[0],"continue")){
- break;
- } else if (!strcasecmp(argv[0],"t") || !strcasecmp(argv[0],"trace")) {
- ldbTrace(lua);
- ldbSendLogs();
- } else if (!strcasecmp(argv[0],"m") || !strcasecmp(argv[0],"maxlen")) {
- ldbMaxlen(argv,argc);
- ldbSendLogs();
- } else if (!strcasecmp(argv[0],"b") || !strcasecmp(argv[0],"break")) {
- ldbBreak(argv,argc);
- ldbSendLogs();
- } else if (!strcasecmp(argv[0],"e") || !strcasecmp(argv[0],"eval")) {
- ldbEval(lua,argv,argc);
- ldbSendLogs();
- } else if (!strcasecmp(argv[0],"a") || !strcasecmp(argv[0],"abort")) {
- lua_pushstring(lua, "script aborted for user request");
- lua_error(lua);
- } else if (argc > 1 &&
- (!strcasecmp(argv[0],"r") || !strcasecmp(argv[0],"redis"))) {
- ldbRedis(lua,argv,argc);
- ldbSendLogs();
- } else if ((!strcasecmp(argv[0],"p") || !strcasecmp(argv[0],"print"))) {
- if (argc == 2)
- ldbPrint(lua,argv[1]);
- else
- ldbPrintAll(lua);
- ldbSendLogs();
- } else if (!strcasecmp(argv[0],"l") || !strcasecmp(argv[0],"list")){
- int around = ldb.currentline, ctx = 5;
- if (argc > 1) {
- int num = atoi(argv[1]);
- if (num > 0) around = num;
- }
- if (argc > 2) ctx = atoi(argv[2]);
- ldbList(around,ctx);
- ldbSendLogs();
- } else if (!strcasecmp(argv[0],"w") || !strcasecmp(argv[0],"whole")){
- ldbList(1,1000000);
- ldbSendLogs();
- } else {
- ldbLog(sdsnew("<error> Unknown Redis Lua debugger command or "
- "wrong number of arguments."));
- ldbSendLogs();
- }
-
- /* Free the command vector. */
- sdsfreesplitres(argv,argc);
- }
-
- /* Free the current command argv if we break inside the while loop. */
- sdsfreesplitres(argv,argc);
- return C_OK;
-}
-
-/* This is the core of our Lua debugger, called each time Lua is about
- * to start executing a new line. */
-void luaLdbLineHook(lua_State *lua, lua_Debug *ar) {
- lua_getstack(lua,0,ar);
- lua_getinfo(lua,"Sl",ar);
- ldb.currentline = ar->currentline;
-
- int bp = ldbIsBreakpoint(ldb.currentline) || ldb.luabp;
- int timeout = 0;
-
- /* Events outside our script are not interesting. */
- if(strstr(ar->short_src,"user_script") == NULL) return;
-
- /* Check if a timeout occurred. */
- if (ar->event == LUA_HOOKCOUNT && ldb.step == 0 && bp == 0) {
- mstime_t elapsed = elapsedMs(server.lua_time_start);
- mstime_t timelimit = server.lua_time_limit ?
- server.lua_time_limit : 5000;
- if (elapsed >= timelimit) {
- timeout = 1;
- ldb.step = 1;
- } else {
- return; /* No timeout, ignore the COUNT event. */
- }
- }
-
- if (ldb.step || bp) {
- char *reason = "step over";
- if (bp) reason = ldb.luabp ? "redis.breakpoint() called" :
- "break point";
- else if (timeout) reason = "timeout reached, infinite loop?";
- ldb.step = 0;
- ldb.luabp = 0;
- ldbLog(sdscatprintf(sdsempty(),
- "* Stopped at %d, stop reason = %s",
- ldb.currentline, reason));
- ldbLogSourceLine(ldb.currentline);
- ldbSendLogs();
- if (ldbRepl(lua) == C_ERR && timeout) {
- /* If the client closed the connection and we have a timeout
- * connection, let's kill the script otherwise the process
- * will remain blocked indefinitely. */
- lua_pushstring(lua, "timeout during Lua debugging with client closing connection");
- lua_error(lua);
- }
- server.lua_time_start = getMonotonicUs();
- server.lua_time_snapshot = mstime();
- }
-}
diff --git a/src/server.c b/src/server.c
index 05ab6b93a..571eb1b06 100644
--- a/src/server.c
+++ b/src/server.c
@@ -35,6 +35,7 @@
#include "latency.h"
#include "atomicvar.h"
#include "mt19937-64.h"
+#include "functions.h"
#include <time.h>
#include <signal.h>
@@ -471,6 +472,31 @@ struct redisCommand scriptSubcommands[] = {
{NULL},
};
+struct redisCommand functionSubcommands[] = {
+ {"create",functionsCreateCommand,-5,
+ "may-replicate no-script @scripting"},
+
+ {"delete",functionsDeleteCommand,3,
+ "may-replicate no-script @scripting"},
+
+ {"kill",functionsKillCommand,2,
+ "no-script @scripting"},
+
+ {"info",functionsInfoCommand,-3,
+ "no-script @scripting"},
+
+ {"list",functionsListCommand,2,
+ "no-script @scripting"},
+
+ {"stats",functionsStatsCommand,2,
+ "no-script @scripting"},
+
+ {"help",functionsHelpCommand,2,
+ "ok-loading ok-stale @scripting"},
+
+ {NULL},
+};
+
struct redisCommand clientSubcommands[] = {
{"caching",clientCommand,3,
"no-script ok-loading ok-stale @connection"},
@@ -2032,7 +2058,25 @@ struct redisCommand redisCommandTable[] = {
"no-auth no-script ok-stale ok-loading fast @connection"},
{"failover",failoverCommand,-1,
- "admin no-script ok-stale"}
+ "admin no-script ok-stale"},
+
+ {"function",NULL,-2,
+ "",
+ .subcommands=functionSubcommands},
+
+ {"fcall",fcallCommand,-3,
+ "no-script no-monitor may-replicate no-mandatory-keys @scripting",
+ {{"read write", /* We pass both read and write because these flag are worst-case-scenario */
+ KSPEC_BS_INDEX,.bs.index={2},
+ KSPEC_FK_KEYNUM,.fk.keynum={0,1,1}}},
+ functionGetKeys},
+
+ {"fcall_ro",fcallCommandReadOnly,-3,
+ "no-script no-monitor no-mandatory-keys @scripting",
+ {{"read",
+ KSPEC_BS_INDEX,.bs.index={2},
+ KSPEC_FK_KEYNUM,.fk.keynum={0,1,1}}},
+ functionGetKeys},
};
/*============================ Utility functions ============================ */
@@ -2208,6 +2252,11 @@ void dictSdsDestructor(dict *d, void *val)
sdsfree(val);
}
+void *dictSdsDup(dict *d, const void *key) {
+ UNUSED(d);
+ return sdsdup((const sds) key);
+}
+
int dictObjKeyCompare(dict *d, const void *key1,
const void *key2)
{
@@ -2994,7 +3043,7 @@ void cronUpdateMemoryStats() {
/* LUA memory isn't part of zmalloc_used, but it is part of the process RSS,
* so we must deduct it in order to be able to calculate correct
* "allocator fragmentation" ratio */
- size_t lua_memory = lua_gc(server.lua,LUA_GCCOUNT,0)*1024LL;
+ size_t lua_memory = evalMemory();
server.cron_malloc_stats.allocator_resident = server.cron_malloc_stats.process_rss - lua_memory;
}
if (!server.cron_malloc_stats.allocator_active)
@@ -3516,8 +3565,10 @@ void createSharedObjects(void) {
"-NOSCRIPT No matching script. Please use EVAL.\r\n"));
shared.loadingerr = createObject(OBJ_STRING,sdsnew(
"-LOADING Redis is loading the dataset in memory\r\n"));
- shared.slowscripterr = createObject(OBJ_STRING,sdsnew(
+ shared.slowevalerr = createObject(OBJ_STRING,sdsnew(
"-BUSY Redis is busy running a script. You can only call SCRIPT KILL or SHUTDOWN NOSAVE.\r\n"));
+ shared.slowscripterr = createObject(OBJ_STRING,sdsnew(
+ "-BUSY Redis is busy running a script. You can only call FUNCTION KILL or SHUTDOWN NOSAVE.\r\n"));
shared.masterdownerr = createObject(OBJ_STRING,sdsnew(
"-MASTERDOWN Link with MASTER is down and replica-serve-stale-data is set to 'no'.\r\n"));
shared.bgsaveerr = createObject(OBJ_STRING,sdsnew(
@@ -4247,7 +4298,7 @@ void initServer(void) {
server.pubsub_channels = dictCreate(&keylistDictType);
server.pubsub_patterns = dictCreate(&keylistDictType);
server.cronloops = 0;
- server.in_eval = 0;
+ server.in_script = 0;
server.in_exec = 0;
server.propagate_in_transaction = 0;
server.client_pause_in_transaction = 0;
@@ -4344,6 +4395,7 @@ void initServer(void) {
if (server.cluster_enabled) clusterInit();
replicationScriptCacheInit();
scriptingInit(1);
+ functionsInit();
slowlogInit();
latencyMonitorInit();
@@ -4931,17 +4983,17 @@ void call(client *c, int flags) {
/* When EVAL is called loading the AOF we don't want commands called
* from Lua to go into the slowlog or to populate statistics. */
- if (server.loading && c->flags & CLIENT_LUA)
+ if (server.loading && c->flags & CLIENT_SCRIPT)
flags &= ~(CMD_CALL_SLOWLOG | CMD_CALL_STATS);
/* If the caller is Lua, we want to force the EVAL caller to propagate
* the script if the command flag or client flag are forcing the
* propagation. */
- if (c->flags & CLIENT_LUA && server.lua_caller) {
+ if (c->flags & CLIENT_SCRIPT && server.script_caller) {
if (c->flags & CLIENT_FORCE_REPL)
- server.lua_caller->flags |= CLIENT_FORCE_REPL;
+ server.script_caller->flags |= CLIENT_FORCE_REPL;
if (c->flags & CLIENT_FORCE_AOF)
- server.lua_caller->flags |= CLIENT_FORCE_AOF;
+ server.script_caller->flags |= CLIENT_FORCE_AOF;
}
/* Note: the code below uses the real command that was executed
@@ -5070,8 +5122,8 @@ void call(client *c, int flags) {
/* If the client has keys tracking enabled for client side caching,
* make sure to remember the keys it fetched via this command. */
if (c->cmd->flags & CMD_READONLY) {
- client *caller = (c->flags & CLIENT_LUA && server.lua_caller) ?
- server.lua_caller : c;
+ client *caller = (c->flags & CLIENT_SCRIPT && server.script_caller) ?
+ server.script_caller : c;
if (caller->flags & CLIENT_TRACKING &&
!(caller->flags & CLIENT_TRACKING_BCAST))
{
@@ -5172,14 +5224,14 @@ void populateCommandMovableKeys(struct redisCommand *cmd) {
* other operations can be performed by the caller. Otherwise
* if C_ERR is returned the client was destroyed (i.e. after QUIT). */
int processCommand(client *c) {
- if (!server.lua_timedout) {
+ if (!scriptIsTimedout()) {
/* Both EXEC and EVAL call call() directly so there should be
* no way in_exec or in_eval or propagate_in_transaction is 1.
* That is unless lua_timedout, in which case client may run
* some commands. */
serverAssert(!server.propagate_in_transaction);
serverAssert(!server.in_exec);
- serverAssert(!server.in_eval);
+ serverAssert(!server.in_script);
}
moduleCallCommandFilters(c);
@@ -5273,8 +5325,8 @@ int processCommand(client *c) {
* 2) The command has no key arguments. */
if (server.cluster_enabled &&
!(c->flags & CLIENT_MASTER) &&
- !(c->flags & CLIENT_LUA &&
- server.lua_caller->flags & CLIENT_MASTER) &&
+ !(c->flags & CLIENT_SCRIPT &&
+ server.script_caller->flags & CLIENT_MASTER) &&
!(!c->cmd->movablekeys && c->cmd->key_specs_num == 0 &&
c->cmd->proc != execCommand))
{
@@ -5309,7 +5361,7 @@ int processCommand(client *c) {
* the event loop since there is a busy Lua script running in timeout
* condition, to avoid mixing the propagation of scripts with the
* propagation of DELs due to eviction. */
- if (server.maxmemory && !server.lua_timedout) {
+ if (server.maxmemory && !scriptIsTimedout()) {
int out_of_memory = (performEvictions() == EVICT_FAIL);
/* performEvictions may evict keys, so we need flush pending tracking
@@ -5345,7 +5397,7 @@ int processCommand(client *c) {
* until first write within script, memory used by lua stack and
* arguments might interfere. */
if (c->cmd->proc == evalCommand || c->cmd->proc == evalShaCommand) {
- server.lua_oom = out_of_memory;
+ server.script_oom = out_of_memory;
}
}
@@ -5432,7 +5484,7 @@ int processCommand(client *c) {
* the MULTI plus a few initial commands refused, then the timeout
* condition resolves, and the bottom-half of the transaction gets
* executed, see Github PR #7022. */
- if (server.lua_timedout &&
+ if (scriptIsTimedout() &&
c->cmd->proc != authCommand &&
c->cmd->proc != helloCommand &&
c->cmd->proc != replconfCommand &&
@@ -5447,9 +5499,15 @@ int processCommand(client *c) {
tolower(((char*)c->argv[1]->ptr)[0]) == 'n') &&
!(c->cmd->proc == scriptCommand &&
c->argc == 2 &&
- tolower(((char*)c->argv[1]->ptr)[0]) == 'k'))
+ tolower(((char*)c->argv[1]->ptr)[0]) == 'k') &&
+ !(c->cmd->proc == functionsKillCommand) &&
+ !(c->cmd->proc == functionsStatsCommand))
{
- rejectCommand(c, shared.slowscripterr);
+ if (scriptIsEval()) {
+ rejectCommand(c, shared.slowevalerr);
+ } else {
+ rejectCommand(c, shared.slowscripterr);
+ }
return C_OK;
}
@@ -6280,13 +6338,15 @@ sds genRedisInfoString(const char *section) {
char peak_hmem[64];
char total_system_hmem[64];
char used_memory_lua_hmem[64];
+ char used_memory_vm_total_hmem[64];
char used_memory_scripts_hmem[64];
char used_memory_rss_hmem[64];
char maxmemory_hmem[64];
size_t zmalloc_used = zmalloc_used_memory();
size_t total_system_mem = server.system_memory_size;
const char *evict_policy = evictPolicyToString();
- long long memory_lua = server.lua ? (long long)lua_gc(server.lua,LUA_GCCOUNT,0)*1024 : 0;
+ long long memory_lua = evalMemory();
+ long long memory_functions = functionsMemory();
struct redisMemOverhead *mh = getMemoryOverheadData();
/* Peak memory is updated from time to time by serverCron() so it
@@ -6300,7 +6360,8 @@ sds genRedisInfoString(const char *section) {
bytesToHuman(peak_hmem,server.stat_peak_memory);
bytesToHuman(total_system_hmem,total_system_mem);
bytesToHuman(used_memory_lua_hmem,memory_lua);
- bytesToHuman(used_memory_scripts_hmem,mh->lua_caches);
+ bytesToHuman(used_memory_vm_total_hmem,memory_functions + memory_lua);
+ bytesToHuman(used_memory_scripts_hmem,mh->lua_caches + mh->functions_caches);
bytesToHuman(used_memory_rss_hmem,server.cron_malloc_stats.process_rss);
bytesToHuman(maxmemory_hmem,server.maxmemory);
@@ -6323,11 +6384,18 @@ sds genRedisInfoString(const char *section) {
"allocator_resident:%zu\r\n"
"total_system_memory:%lu\r\n"
"total_system_memory_human:%s\r\n"
- "used_memory_lua:%lld\r\n"
- "used_memory_lua_human:%s\r\n"
+ "used_memory_lua:%lld\r\n" /* deprecated, renamed to used_memory_vm_eval */
+ "used_memory_vm_eval:%lld\r\n"
+ "used_memory_lua_human:%s\r\n" /* deprecated */
+ "used_memory_scripts_eval:%lld\r\n"
+ "number_of_cached_scripts:%lu\r\n"
+ "number_of_functions:%lu\r\n"
+ "used_memory_vm_functions:%lld\r\n"
+ "used_memory_vm_total:%lld\r\n"
+ "used_memory_vm_total_human:%s\r\n"
+ "used_memory_functions:%lld\r\n"
"used_memory_scripts:%lld\r\n"
"used_memory_scripts_human:%s\r\n"
- "number_of_cached_scripts:%lu\r\n"
"maxmemory:%lld\r\n"
"maxmemory_human:%s\r\n"
"maxmemory_policy:%s\r\n"
@@ -6366,10 +6434,17 @@ sds genRedisInfoString(const char *section) {
(unsigned long)total_system_mem,
total_system_hmem,
memory_lua,
+ memory_lua,
used_memory_lua_hmem,
(long long) mh->lua_caches,
+ dictSize(evalScriptsDict()),
+ functionsNum(),
+ memory_functions,
+ memory_functions + memory_lua,
+ used_memory_vm_total_hmem,
+ (long long) mh->functions_caches,
+ (long long) mh->lua_caches + (long long) mh->functions_caches,
used_memory_scripts_hmem,
- dictSize(server.lua_scripts),
server.maxmemory,
maxmemory_hmem,
evict_policy,
diff --git a/src/server.h b/src/server.h
index c06b23631..535d21dd9 100644
--- a/src/server.h
+++ b/src/server.h
@@ -260,7 +260,7 @@ extern int configOOMScoreAdjValuesDefaults[CONFIG_OOM_COUNT];
#define CLIENT_CLOSE_AFTER_REPLY (1<<6) /* Close after writing entire reply. */
#define CLIENT_UNBLOCKED (1<<7) /* This client was unblocked and is stored in
server.unblocked_clients */
-#define CLIENT_LUA (1<<8) /* This is a non connected client used by Lua */
+#define CLIENT_SCRIPT (1<<8) /* This is a non connected client used by Lua */
#define CLIENT_ASKING (1<<9) /* Client issued the ASKING command */
#define CLIENT_CLOSE_ASAP (1<<10)/* Close this client ASAP */
#define CLIENT_UNIX_SOCKET (1<<11) /* Client connected via Unix domain socket */
@@ -822,6 +822,19 @@ typedef struct redisDb {
clusterSlotToKeyMapping *slots_to_keys; /* Array of slots to keys. Only used in cluster mode (db 0). */
} redisDb;
+/* forward declaration for functions ctx */
+typedef struct functionsCtx functionsCtx;
+
+/* Holding object that need to be populated during
+ * rdb loading. On loading end it is possible to decide
+ * whether not to set those objects on their rightful place.
+ * For example: dbarray need to be set as main database on
+ * successful loading and dropped on failure. */
+typedef struct rdbLoadingCtx {
+ redisDb* dbarray;
+ functionsCtx* functions_ctx;
+}rdbLoadingCtx;
+
/* Client MULTI/EXEC state */
typedef struct multiCmd {
robj **argv;
@@ -1122,7 +1135,7 @@ struct sharedObjectsStruct {
robj *crlf, *ok, *err, *emptybulk, *czero, *cone, *pong, *space,
*queued, *null[4], *nullarray[4], *emptymap[4], *emptyset[4],
*emptyarray, *wrongtypeerr, *nokeyerr, *syntaxerr, *sameobjecterr,
- *outofrangeerr, *noscripterr, *loadingerr, *slowscripterr, *bgsaveerr,
+ *outofrangeerr, *noscripterr, *loadingerr, *slowevalerr, *slowscripterr, *bgsaveerr,
*masterdownerr, *roslaveerr, *execaborterr, *noautherr, *noreplicaserr,
*busykeyerr, *oomerr, *plus, *messagebulk, *pmessagebulk, *subscribebulk,
*unsubscribebulk, *psubscribebulk, *punsubscribebulk, *del, *unlink,
@@ -1203,6 +1216,7 @@ struct redisMemOverhead {
size_t clients_normal;
size_t aof_buffer;
size_t lua_caches;
+ size_t functions_caches;
size_t overhead_total;
size_t dataset;
size_t total_keys;
@@ -1334,7 +1348,7 @@ struct redisServer {
int sentinel_mode; /* True if this instance is a Sentinel. */
size_t initial_memory_usage; /* Bytes used after initialization. */
int always_show_logo; /* Show logo even for non-stdout logging. */
- int in_eval; /* Are we inside EVAL? */
+ int in_script; /* Are we inside EVAL? */
int in_exec; /* Are we inside EXEC? */
int propagate_in_transaction; /* Make sure we don't propagate nested MULTI/EXEC */
char *ignore_warnings; /* Config: warnings that should be ignored. */
@@ -1719,28 +1733,11 @@ struct redisServer {
is down? */
int cluster_config_file_lock_fd; /* cluster config fd, will be flock */
/* Scripting */
- lua_State *lua; /* The Lua interpreter. We use just one for all clients */
- client *lua_client; /* The "fake client" to query Redis from Lua */
- client *lua_caller; /* The client running EVAL right now, or NULL */
- char* lua_cur_script; /* SHA1 of the script currently running, or NULL */
- dict *lua_scripts; /* A dictionary of SHA1 -> Lua scripts */
- unsigned long long lua_scripts_mem; /* Cached scripts' memory + oh */
- mstime_t lua_time_limit; /* Script timeout in milliseconds */
- monotime lua_time_start; /* monotonic timer to detect timed-out script */
- mstime_t lua_time_snapshot; /* Snapshot of mstime when script is started */
- int lua_write_dirty; /* True if a write command was called during the
- execution of the current script. */
- int lua_random_dirty; /* True if a random command was called during the
- execution of the current script. */
- int lua_replicate_commands; /* True if we are doing single commands repl. */
- int lua_multi_emitted;/* True if we already propagated MULTI. */
- int lua_repl; /* Script replication flags for redis.set_repl(). */
- int lua_timedout; /* True if we reached the time limit for script
- execution. */
- int lua_kill; /* Kill the script if true. */
+ client *script_caller; /* The client running script right now, or NULL */
+ mstime_t script_time_limit; /* Script timeout in milliseconds */
int lua_always_replicate_commands; /* Default replication type. */
- int lua_oom; /* OOM detected when script start? */
- int lua_disable_deny_script; /* Allow running commands marked "no-script" inside a script. */
+ int script_oom; /* OOM detected when script start */
+ int script_disable_deny_script; /* Allow running commands marked "no-script" inside a script. */
/* Lazy free */
int lazyfree_lazy_eviction;
int lazyfree_lazy_expire;
@@ -2687,6 +2684,7 @@ int sintercardGetKeys(struct redisCommand *cmd,robj **argv, int argc, getKeysRes
int zunionInterDiffGetKeys(struct redisCommand *cmd,robj **argv, int argc, getKeysResult *result);
int zunionInterDiffStoreGetKeys(struct redisCommand *cmd,robj **argv, int argc, getKeysResult *result);
int evalGetKeys(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result);
+int functionGetKeys(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result);
int sortGetKeys(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result);
int migrateGetKeys(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result);
int georadiusGetKeys(struct redisCommand *cmd, robj **argv, int argc, getKeysResult *result);
@@ -2722,8 +2720,17 @@ void scriptingInit(int setup);
int ldbRemoveChild(pid_t pid);
void ldbKillForkedSessions(void);
int ldbPendingChildren(void);
-sds luaCreateFunction(client *c, lua_State *lua, robj *body);
+sds luaCreateFunction(client *c, robj *body);
+void luaLdbLineHook(lua_State *lua, lua_Debug *ar);
void freeLuaScriptsAsync(dict *lua_scripts);
+int ldbIsEnabled();
+void ldbLog(sds entry);
+void ldbLogRedisReply(char *reply);
+void sha1hex(char *digest, char *script, size_t len);
+unsigned long evalMemory();
+dict* evalScriptsDict();
+unsigned long evalScriptsMemory();
+mstime_t evalTimeSnapshot();
/* Blocked clients */
void processUnblockedClients(void);
@@ -2769,6 +2776,7 @@ uint64_t dictSdsCaseHash(const void *key);
int dictSdsKeyCompare(dict *d, const void *key1, const void *key2);
int dictSdsKeyCaseCompare(dict *d, const void *key1, const void *key2);
void dictSdsDestructor(dict *d, void *val);
+void *dictSdsDup(dict *d, const void *key);
/* Git SHA1 */
char *redisGitSHA1(void);
diff --git a/src/sort.c b/src/sort.c
index fe0d55fea..153d6ba79 100644
--- a/src/sort.c
+++ b/src/sort.c
@@ -294,7 +294,7 @@ void sortCommandGeneric(client *c, int readonly) {
* scripting and replication. */
if (dontsort &&
sortval->type == OBJ_SET &&
- (storekey || c->flags & CLIENT_LUA))
+ (storekey || c->flags & CLIENT_SCRIPT))
{
/* Force ALPHA sorting */
dontsort = 0;
diff --git a/src/t_stream.c b/src/t_stream.c
index af32bc18f..14dd2bd02 100644
--- a/src/t_stream.c
+++ b/src/t_stream.c
@@ -1998,7 +1998,7 @@ void xreadCommand(client *c) {
int moreargs = c->argc-i-1;
char *o = c->argv[i]->ptr;
if (!strcasecmp(o,"BLOCK") && moreargs) {
- if (c->flags & CLIENT_LUA) {
+ if (c->flags & CLIENT_SCRIPT) {
/*
* Although the CLIENT_DENY_BLOCKING flag should protect from blocking the client
* on Lua/MULTI/RM_Call we want special treatment for Lua to keep backward compatibility.