summaryrefslogtreecommitdiff
path: root/src/networking.c
Commit message (Collapse)AuthorAgeFilesLines
* Slave removal: networking.c logs fixed.antirez2018-09-111-5/+5
|
* Slave removal: slave -> replica in redis.conf and output buffer option.antirez2018-09-111-0/+1
|
* Clarify why remaining may be zero in readQueryFromClient().antirez2018-09-041-0/+2
| | | | See #5304.
* Merge pull request #5304 from soloestoy/fix-unexpected-readlenSalvatore Sanfilippo2018-09-041-1/+1
|\ | | | | networking: fix unexpected negative or zero readlen
| * networking: fix unexpected negative or zero readlenzhaozhao.zz2018-08-311-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To avoid copying buffers to create a large Redis Object which exceeding PROTO_IOBUF_LEN 32KB, we just read the remaining data we need, which may less than PROTO_IOBUF_LEN. But the remaining len may be zero, if the bulklen+2 equals sdslen(c->querybuf), in client pause context. For example: Time1: python >>> import os, socket >>> server="127.0.0.1" >>> port=6379 >>> data1="*3\r\n$3\r\nset\r\n$1\r\na\r\n$33000\r\n" >>> data2="".join("x" for _ in range(33000)) + "\r\n" >>> data3="\n\n" >>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) >>> s.settimeout(10) >>> s.connect((server, port)) >>> s.send(data1) 28 Time2: redis-cli client pause 10000 Time3: >>> s.send(data2) 33002 >>> s.send(data3) 2 >>> s.send(data3) Traceback (most recent call last): File "<stdin>", line 1, in <module> socket.error: [Errno 104] Connection reset by peer To fix that, we should check if remaining is greater than zero.
* | Merge pull request #5315 from soloestoy/optimize-parsing-large-bulkSalvatore Sanfilippo2018-09-041-10/+13
|\ \ | | | | | | networking: optimize parsing large bulk greater than 32k
| * | networking: optimize parsing large bulk greater than 32kzhaozhao.zz2018-09-041-10/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If we are going to read a large object from network try to make it likely that it will start at c->querybuf boundary so that we can optimize object creation avoiding a large copy of data. But only when the data we have not parsed is less than or equal to ll+2. If the data length is greater than ll+2, trimming querybuf is just a waste of time, because at this time the querybuf contains not only our bulk. It's easy to reproduce the that: Time1: call `client pause 10000` on slave. Time2: redis-benchmark -t set -r 10000 -d 33000 -n 10000. Then slave hung after 10 seconds.
* | | Unblocked clients API refactoring. See #4418.antirez2018-09-031-5/+4
| | |
* | | Merge pull request #4418 from soloestoy/fix-multiple-unblockSalvatore Sanfilippo2018-09-031-3/+3
|\ \ \ | | | | | | | | fix multiple unblock for clientsArePaused()
| * | | fix multiple unblock for clientsArePaused()zhaozhao.zz2018-09-031-3/+3
| |/ /
* | | Make pending buffer processing safe for CLIENT_MASTER client.antirez2018-09-031-12/+20
|/ / | | | | | | Related to #5305.
* | After slave Lua script leaves busy state, re-process the master buffer.antirez2018-08-311-2/+1
| | | | | | | | | | | | | | Technically speaking we don't really need to put the master client in the clients that need to be processed, since in practice the PING commands from the master will take care, however it is conceptually more sane to do so.
* | While the slave is busy, just accumulate master input.antirez2018-08-311-0/+6
|/ | | | | | | | | | Processing command from the master while the slave is in busy state is not correct, however we cannot, also, just reply -BUSY to the replication stream commands from the master. The correct solution is to stop processing data from the master, but just accumulate the stream into the buffers and resume the processing later. Related to #5297.
* networking: make setProtocolError simple and clearzhaozhao.zz2018-08-231-13/+11
| | | | | | | Function setProtocolError just records proctocol error details in server log, set client as CLIENT_CLOSE_AFTER_REPLY. It doesn't care about querybuf sdsrange, because we will do it after procotol parsing.
* networking: just move qb_pos instead of sdsrange in processInlineBufferzhaozhao.zz2018-08-141-3/+2
|
* networking: just return C_OK if multibulk processing saw a <= 0 length.zhaozhao.zz2018-08-141-5/+2
|
* pipeline: do not sdsrange querybuf unless all commands processedzhaozhao.zz2018-08-141-40/+47
| | | | | | | | | This is an optimization for processing pipeline, we discussed a problem in issue #5229: clients may be paused if we apply `CLIENT PAUSE` command, and then querybuf may grow too large, the cost of memmove in sdsrange after parsing a completed command will be horrible. The optimization is that parsing all commands in queyrbuf , after that we can just call sdsrange only once.
* In addReplyErrorLength() only panic when replying to slave.antirez2018-07-181-3/+4
| | | | See #5135 for more context.
* Refine comment in addReplyErrorLength() about replying to masters/slaves.antirez2018-07-181-0/+11
| | | | See #5135 for some context.
* Panic when we are sending an error to our master/slave.antirez2018-07-171-0/+5
| | | | Related to #5135, see discussion there.
* fix rare replication stream corruption with disk-based replicationOran Agra2018-07-171-7/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The slave sends \n keepalive messages to the master while parsing the rdb, and later sends REPLCONF ACK once a second. rarely, the master recives both a linefeed char and a REPLCONF in the same read, \n*3\r\n$8\r\nREPLCONF\r\n... and it tries to trim two chars (\r\n) from the query buffer, trimming the '*' from *3\r\n$8\r\nREPLCONF\r\n... then the master tries to process a command starting with '3' and replies to the slave a bunch of -ERR and one +OK. although the slave silently ignores these (prints a log message), this corrupts the replication offset at the slave since the slave increases the replication offset, and the master did not. other than the fix in processInlineBuffer, i did several other improvments while hunting this very rare bug. - when redis replies with "unknown command" it includes a portion of the arguments, not just the command name. so it would be easier to understand what was recived, in my case, on the slave side, it was -ERR, but the "arguments" were the interesting part (containing info on the error). - about a year ago i added code in addReplyErrorLength to print the error to the log in case of a reply to master (since this string isn't actually trasmitted to the master), now changed that block to print a similar log message to indicate an error being sent from the master to the slave. note that the slave is marked as CLIENT_SLAVE only after PSYNC was received, so this will not cause any harm for REPLCONF, and will only indicate problems that are gonna corrupt the replication stream anyway. - two places were c->reply was emptied, and i wanted to reset sentlen this is a precaution (i did not actually see such a problem), since a non-zero sentlen will cause corruption to be transmitted on the socket.
* Hopefully improve commenting of #5126.antirez2018-07-161-10/+20
| | | | | | | | Reading the PR gave me the opportunity to better specify what the code was doing in places where I was not immediately sure about what was going on. Moreover I documented the structure in server.h so that people reading the header file will immediately understand what the structure is useful for.
* slave buffers were wasteful and incorrectly counted causing evictionOran Agra2018-07-161-45/+61
| | | | | | | | | | | | | | | | | | | | | | | | | A) slave buffers didn't count internal fragmentation and sds unused space, this caused them to induce eviction although we didn't mean for it. B) slave buffers were consuming about twice the memory of what they actually needed. - this was mainly due to sdsMakeRoomFor growing to twice as much as needed each time but networking.c not storing more than 16k (partially fixed recently in 237a38737). - besides it wasn't able to store half of the new string into one buffer and the other half into the next (so the above mentioned fix helped mainly for small items). - lastly, the sds buffers had up to 30% internal fragmentation that was wasted, consumed but not used. C) inefficient performance due to starting from a small string and reallocing many times. what i changed: - creating dedicated buffers for reply list, counting their size with zmalloc_size - when creating a new reply node from, preallocate it to at least 16k. - when appending a new reply to the buffer, first fill all the unused space of the previous node before starting a new one. other changes: - expose mem_not_counted_for_evict info field for the benefit of the test suite - add a test to make sure slave buffers are counted correctly and that they don't cause eviction
* Bugfix: PEL is incorrect when consumer is blocked using xreadgroup with ↵dejun.xdj2018-07-091-0/+1
| | | | | | NOACK option. Save NOACK option into client.blockingState structure.
* CLIENT UNBLOCK: fix client unblock help message.dejun.xdj2018-07-091-1/+1
|
* fix compile warning in addReplySubcommandSyntaxErrorWuYunlong2018-07-091-1/+1
|
* addReplySubSyntaxError() renamed to addReplySubcommandSyntaxError().antirez2018-07-021-2/+2
|
* Merge pull request #4998 from itamarhaber/module_command_helpSalvatore Sanfilippo2018-07-021-0/+12
|\ | | | | Module command help
| * Adds MODULE HELP and implements addReplySubSyntaxErrorItamar Haber2018-06-071-0/+12
| |
* | Change CLIENT LIST TYPE help string.antirez2018-06-291-2/+2
| | | | | | | | Making it more similar to KILL.
* | clients: add type option for client listzhaozhao.zz2018-06-281-3/+17
| |
* | clients: show pubsub flag in client listzhaozhao.zz2018-06-281-0/+1
| |
* | Make CLIENT HELP output nicer to the eyes.antirez2018-06-281-11/+11
| |
* | Add unblock in CLIENT HELP.antirez2018-06-281-0/+1
| |
* | CLIENT UNBLOCK: support unblocking by error.antirez2018-06-271-3/+22
| |
* | CLIENT UNBLOCK implemented.antirez2018-06-271-0/+22
| |
* | Take clients in a ID -> Client handle dictionary.antirez2018-06-271-0/+4
| |
* | CLIENT ID implemented.antirez2018-06-271-0/+4
| |
* | optimize reply list memory usagezhaozhao.zz2018-06-131-1/+1
|/
* Add top comments in two addReply*() functions.antirez2018-03-221-0/+3
|
* Massivily simplify addReply*() functions in networking.cantirez2018-03-221-87/+11
|
* Make addReplyError...() family functions able to get error codes.antirez2018-03-151-1/+11
| | | | | | | | | Now you can use: addReplyError("-MYERRORCODE some message"); If the error code is omitted, the behavior is like in the past, the generic -ERR will be used.
* CG: add & populate group+consumer in the blocking state.antirez2018-03-151-0/+1
|
* Actually use ae_flags to add AE_BARRIER if needed.antirez2018-02-281-1/+1
| | | | Many thanks to @Plasma that spotted this problem reviewing the code.
* AOF: fix a bug that may prevent proper fsyncing when fsync=always.antirez2018-02-271-6/+18
| | | | | | | | | | | In case the write handler is already installed, it could happen that we serve the reply of a query in the same event loop cycle we received it, preventing beforeSleep() from guaranteeing that we do the AOF fsync before sending the reply to the client. The AE_BARRIER mechanism, introduced in a previous commit, prevents this problem. This commit makes actual use of this new feature to fix the bug.
* Merge pull request #3828 from oranagra/sdsnewlen_prSalvatore Sanfilippo2018-02-271-2/+2
|\ | | | | add SDS_NOINIT option to sdsnewlen to avoid unnecessary memsets.
| * add SDS_NOINIT option to sdsnewlen to avoid unnecessary memsets.oranagra2017-02-231-2/+2
| | | | | | | | | | this commit also contains small bugfix in rdbLoadLzfStringObject a bug that currently has no implications.
* | More verbose logging when slave sends errors to master.antirez2018-02-131-2/+6
| | | | | | | | See #3832.
* | Merge pull request #3832 from oranagra/slave_reply_to_master_prSalvatore Sanfilippo2018-02-131-0/+2
|\ \ | | | | | | when a slave responds with an error on commands that come from master, log it
| * | when a slave experiances an error on commands that come from master, print ↵oranagra2017-02-231-0/+2
| |/ | | | | | | | | | | | | | | to the log since slave isn't replying to it's master, these errors go unnoticed. since we don't expect the master to send garbadge to the slave, this should be safe. (as long as we don't log OOM errors there)
View Lorry Controller Status