summaryrefslogtreecommitdiff
path: root/doc
Commit message (Collapse)AuthorAgeFilesLines
* log: Add a new watcher to watch for deletions.Hemal Shah2023-03-151-1/+4
| | | | | | | `watch deletions`: would log all keys which are deleted using either `delete` or `md` command. The log line would contain the command used, the key, the clsid and size of the deleted item. Items which result in delete miss or are marked as stale wouldn't show up in the logs
* meta: N flag changes append/prepend. ms s flag.dormando2023-03-111-0/+10
| | | | | | | | | Sending 's' flag to metaset now returns the size of the item stored. Useful if you want to know how large an append/prepended item now is. If the 'N' flag is supplied while in append/prepend mode, allows autovivifying (with exptime supplied from N) for append/prepend style keys that don't need headers created first.
* Document missing flags of Meta ArithmeticMate Borcsok2023-03-081-0/+2
|
* crawler: mgdump documentationdormando2023-02-271-0/+25
|
* proxy: add proxy_await_active statdormando2022-12-011-0/+1
| | | | | | | proxy_req_active shows the number of active proxy requests, but if those proxy requests make sub-requests via mcp.await() they are not accounted for. This gives the number of await's active, but not the total number of in-flight async requests.
* docs: don't rebuild binprot XML anymoredormando2022-08-254-15/+3529
| | | | | | | | | xml2rfc's toolchain has bitrotted and I don't intend on changing the binary protocol documentation anymore. Instead I've included the .txt files from 1.6.16 and commented out the build docs. Even if I adjust the call args for xml2rfc it complains about the copyright being old and I don't want to think about that :(
* tls: Add switch to opt-in to kernel TLS on OpenSSL 3.0.0+Kevin Lin2022-07-031-0/+6
|
* proxy: replace proxycmds stream with proxyreqsdormando2022-04-081-1/+1
| | | | | | | | | delete the magic logging and require mcp.log_req* be used if you want those types of entries to appear. keeps a separate data stream from "proxyuser" just in case that's useful. proxycmds wasn't able to get enough context to autogenerate useful log lines, so I'd rather not have it in there at all.
* proxy: documentation updatesdormando2022-02-181-0/+18
|
* meta: add "proxy tokens" P, L which are ignoreddormando2022-02-041-0/+7
| | | | see doc/protocol.txt
* meta: protocol.txt updates for CAS returndormando2022-01-121-1/+6
|
* Track store errors in thread statsKevin Lin2021-11-231-0/+6
| | | | | | Add two new stat keys, `store_too_large` and `store_no_memory`, to track occurrences of storage request rejections due to writing too large of a value and writing beyond available provisioned memory, respectively.
* Expose number of currently active watchers in stats1.6.11Kevin Lin2021-09-271-0/+1
| | | | | The stat key `log_watchers` indicates the number of active connected `watch` clients.
* Configurable minimum supported TLS protocol versionKevin Lin2021-09-271-0/+2
| | | | | | | `-o ssl_min_version` can be used to configure the server to only accept handshakes from clients with a minimum TLS protocol version. Currently supported options are TLS v1.0, TLS v1.1, TLS v1.2, and TLS v1.3 (OpenSSL 1.1.1+ only).
* Implement LOG_CONNEVENTS watcher flag for connection state transitionsKevin Lin2021-08-071-1/+6
| | | | | | | | Add support for `watch connevents` to report opened (`conn_new`) and closed (`conn_close`) client connections. Event log lines indicate the connection's remote IP, remote port, and transport type. `conn_close` events additionally supply a reason for the closing the connection.
* Fix typos in doc/code comments (tem->item, etc)Tyson Andre2021-08-051-2/+2
| | | | | Note: Do not fix typos in crc32.c because it's copied from an upstream source
* Add settings stat for shutdown_command enabledKevin Lin2021-07-251-0/+1
|
* meta: remove EXPERIMENTAL mark + doc fixesdormando2021-07-251-12/+12
|
* meta: response code OK -> HDdormando2021-06-101-6/+6
| | | | | | | | | | | | | | I had the response code as "HD" in the past, but standardized on OK while merging a number of "OK-like" rescodes together. This was a mistake; as many "generic" memcached response codes use "OK". Most of these are management or specialized uncommon commands. With this, a client response parser can know for sure if a response is to a meta command, or some other command. `-o meta_response_old` starttime option has been added, valid for the next 3 months, which switches the response code back from HD to OK. In case any existing users depended on this and need time to migrate.
* meta: fix metaset syntaxdormando2021-06-101-4/+8
| | | | | | | payload length is now part of the main command rather than a flag/token pair. This allows byte swallow to be more reliable in the case of a flag parsing error, and makes sets easier to parse and buffer for servers implementing the protocol.
* meta: protect cachedump from bin keys and add docsdormando2021-06-071-1/+16
| | | | | cachedump was the only place in the codebase I can find which copied the key verbatim. wonder when I can finally remove the command :)
* 'shutdown graceful' command for raising SIGUSR1Kevin Lin2020-11-201-0/+6
|
* Expose memory_file path in stats settingsKevin Lin2020-11-201-0/+1
|
* item crawler hash table walk modedormando2020-11-201-1/+6
| | | | | | | specifying 'hash' instead of 'all' will make the LRU crawler iterate over ever bucket in the hash table once, instead of what's in the LRU. This also doesn't suffer from missing items because of LRU reordering or high lock contention.
* Introduce NAPI ID based worker thread selectionSridhar Samudrala2020-11-022-0/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | By default memcached assigns connections to worker threads in a round-robin manner. This patch introduces an option to select a worker thread based on the incoming connection's NAPI ID if SO_INCOMING_NAPI_ID socket option is supported by the OS. This allows a memcached worker thread to be associated with a NIC HW receive queue and service all the connection requests received on a specific RX queue. This mapping between a memcached thread and a HW NIC queue streamlines the flow of data from the NIC to the application. In addition, an optimal path with reduced context switches is possible, if epoll based busy polling (sysctl -w net.core.busy_poll = <non-zero value>) is also enabled. This feature is enabled via a new command line parameter -N <num> or "--napi_ids=<num>", where <num> is the number of available/assigned NIC hardware RX queues through which the connections can be received. The number of napi_ids specified cannot be greater than the number of worker threads specified using -t/--threads option. If the option is not specified, or the conditions not met, the code defaults to round robin thread selection. Signed-off-by: Kiran Patil <kiran.patil@intel.com> Signed-off-by: Sridhar Samudrala <sridhar.samudrala@intel.com>
* fix leak in merged resp/read buffersdormando2020-07-021-0/+3
| | | | | | | | | | The list grows toward next, not prev. Also wasn't zeroing out next ptr. Also didn't unmark free for first resp object. (thanks Prudhviraj!) Adds a couple counters so users can tell if something is wrong as well. response_obj_count is solely for response objects in-flight, they should not be held when idle (except for one bundle per worker thread).
* net: remove most response obj cache related codedormando2020-06-091-7/+4
| | | | | | accept but warn if the commandline is used. also keeps the oom counter distinct so users have a chance of telling what type of memory pressure they're under.
* Add Meta no-op command request to protocol.txtDmitry Volodin2020-04-141-0/+2
|
* Fix misprint in protocol.txtDmitry Volodin2020-04-101-1/+1
|
* Add: `-o ssl_session_cache`, disabled by defaultKevin Lin2020-03-272-1/+6
| | | | Enables server-side TLS session caching.
* ssl_errors statKevin Lin2020-03-161-0/+17
|
* extstore: enable by default.1.6.0dormando2020-03-081-18/+0
| | | | links to a forwarder for the wiki page.
* meta: documentation fixesdormando2020-03-071-2/+4
| | | | Thanks to @shivnagarajan
* meta: arithmetic command for incr/decrdormando2020-03-061-0/+82
| | | | | see doc/protocol.txt. needed slightly different code as we have to generate the response line after the main operation completes.
* meta: refactor do_store_item and add 'M' for msetdormando2020-03-061-5/+17
| | | | | | | | do_store_item's logic was really hard to follow. This is nearly the same logic but split out to be more clear. Also adds the mode switch for mset, so we get add/append/prepend/replace without separate top level commands.
* add separate limits for connection buffersdormando2020-02-261-0/+4
| | | | | | | | | | | allows specifying a megabyte limit for either response objects or read buffers. this is split among all of the worker threads. useful if connection limit is extremely high and you want to aggressively close connections if something happens and all connections become active at the same time. missing runtime tuning.
* meta: make return codes more genericdormando2020-02-141-6/+6
| | | | | | | | | | ST/DE/HD all mean "okay". EX/NF/etc are already reused. VA has different response parsing. EN/MN are specific. future commands should be able to reuse most of these. In the odd case new ones are added, clients would have to be updated. Generally they shouldn't have to be.
* meta: minor protocol adjustmentsdormando2020-02-011-65/+72
| | | | | | | | | | | | | remove EN\r\n trailer from valid mg responses. VA is the status code, EN is redundant. quiet mode elides the EN on miss. meta-noop now has a dedicated MN status code for clarity. fix bug in return data for "mg key\r\n" updates tests for new changes. also updates documentation.
* network: transient static read buffer for connsdormando2020-02-011-0/+2
| | | | | | | | | | | | instead of 2k and then realloc all over every time you set a large item, or do large pipelined fetches, use a static slightly larger buffer. Idle connections no longer hold a buffer, freeing up a ton of memory. To maintain compatibility with unbound ASCII multigets, those fall back to the old malloc/realloc/free routine which it's done since the dark ages.
* network: response stacking for all commandsdormando2020-02-011-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | This change refactors most of memcached's networking frontend code. The goals with this change: - Decouple buffer memory from client connections where plausible, reducing memory usage for currently inactive TCP sessions. - Allow easy response stacking for all protocols. Previously every binary command generates a syscall during response. This is no longer true when multiple commands are read off the wire at once. The new meta protocol and most text protocol commands are similar. - Reduce code complexity for network handling. Remove automatic buffer adjustments, error checking, and so on. This is accomplished by removing the iovec, msg, and "write buffer" structures from connection objects. A `mc_resp` object is now used to represent an individual response. These objects contain small iovec's and enough detail to be late-combined on transmit. As a side effect, UDP mode now works with extstore :) Adding to the iovec always had a remote chance of memory failure, so every call had to be checked for an error. Now once a response object is allocated, most manipulations can happen without any checking. This is both a boost to code robustness and performance for some hot paths. This patch is the first in a series, focusing on the client response.
* Update documentation for --max-item-sizeiqr4m2020-01-131-1/+1
| | | | | | This limit was added in this patch [1], while this patch [2] fixed it in -h output. [1] https://github.com/memcached/memcached/commit/b5ad069ed887b8086756def7fb2efa65f5c1e462#diff-3970a0b3806605c2c138450fdae4e80e [2] https://github.com/memcached/memcached/commit/50bdc9f3e134122e280031683f2b502f84e96624#diff-3970a0b3806605c2c138450fdae4e80e
* doc: Use wildcard instead of explicit file listOla Jeppsson2019-10-221-6/+2
| | | | | | | | | As per request here: https://github.com/memcached/memcached/pull/554#issuecomment-543362675 NB: Using wildcards for dependencies with Automake is not recommended and should be considered fragile. However, for this particular case it seems to work.
* doc: Update rfc2629.dtd, use local copy, fix error, and fix warningOla Jeppsson2019-10-222-16/+27
| | | | | | | | | | | | | | | | | | | | | | | - I get this warning when building the docs: WARNING: No DTD given, defaulting to /usr/lib/python3.7/site-packages/xml2rfc/templates/rfc2629.dtd - Patching Makefile.am to use the local DTD gives the below error: ERROR: Unable to validate the XML document: protocol-binary.full protocol-binary.full: Line 2: Value "trust200902" for attribute ipr of rfc is not among the enumerated set - This commit suggests that you prefer to use a local copy of the DTD. commit fe2629fd3fcdbde49f4016635866bd3488be88b0 Author: Steve Wills <steve@mouf.net> Date: Wed Jul 23 19:21:40 2014 +0000 keep local copies of dtd and use it This commit ensures we don't rely on and external site for the docs to build. If that site were to go away, we wouldn't be able to build docs. - Hence, update the DTD too.
* doc: Fix out-of-tree buildOla Jeppsson2019-10-221-14/+27
| | | | | | | | | | | | | Out-of-tree build didn't work for doc/ since $(srcdir) was not prepended to the source files. This solution uses pattern substitutions and make automatic variables. Here 'make' will use VPATH (set by Automake) so $(srcdir) isn't needed. Remove use of wildcards, Automake does not support them. https://www.gnu.org/software/automake/manual/html_node/Wildcards.html Always include built specifications when make dist.
* meta text protocol commandsdormando2019-09-301-1/+405
| | | | | | | | | | | | | | | | | | | | | | | | | - we get asked a lot to provide a "metaget" command, for various uses (debugging, etc) - we also get asked for random one-off commands for various use cases. - I really hate both of these situations and have been wanting to experiment with a slight tuning of how get commands work for a long time. Assuming that if I offer a metaget command which gives people the information they're curious about in an inefficient format, plus data they don't need, we'll just end up with a slow command with compatibility issues. No matter how you wrap warnings around a command, people will put it into production under high load. Then I'm stuck with it forever. Behold, the meta commands! See doc/protocol.txt and the wiki for a full explanation and examples. The intent of the meta commands is to support any features the binary protocol had over the text protocol. Though this is missing some commands still, it is close and surpasses the binary protocol in many ways.
* add server address to the "stats conns" outputTharanga Gamaethige2019-08-291-0/+2
| | | | | | this is helpful when it's required to identify which clients are connected to which server address when memcached listens on multiple addresses
* move mem_requested from slabs.c to items.cdormando2019-07-261-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | mem_requested is an oddball counter: it's the total number of bytes "actually requested" from the slab's caller. It's mainly used for a stats counter, alerting the user that the slab factor may not be efficient if the gap between total_chunks * chunk_size - mem_requested is large. However, since chunked items were added it's _also_ used to help the LRU balance itself. The total number of bytes used in the class vs the total number of bytes in a sub-LRU is used to judge whether to move items between sub-LRU's. This is a layer violation; forcing slabs.c to know more about how items work, as well as EXTSTORE for calculating item sizes from headers. Further, it turns out it wasn't necessary for item allocation: if we need to evict an item we _always_ pull from COLD_LRU or force a move from HOT_LRU. So the total doesn't matter. The total does matter in the LRU maintainer background thread. However, this thread caches mem_requested to avoid hitting the slab lock too frequently. Since sizes_bytes[] within items.c is generally redundant with mem_requested, we now total sizes_bytes[] from each sub-LRU before starting a batch of LRU juggles. This simplifies the code a bit, reduces the layer violations in slabs.c slightly, and actually speeds up some hot paths as a number of branches and operations are removed completely. This also fixes an issue I was having with the restartable memory branch :) recalculating p->requested and keeping a clean API is painful and slow. NOTE: This will vary a bit compared to what mem_requested originally did, mostly for large chunked items. For items which fit inside a single slab chunk, the stat is identical. However, items constructed by chaining chunks will have a single large "nbytes" value and end up in the highest slab class. Chunked items can be capped with chunks from smaller slab classes; you will see utilization of chunks but not an increase in mem_requested for them. I'm still thinking this through but this is probably acceptable. Large chunked items should be accounted for separately, perhaps with some new counters so they can be discounted from normal calculations.
* change some links from http to httpskun2019-05-202-3/+3
|
* remove inline_ascii_response optiondormando2019-05-201-3/+1
| | | | | | Has defaulted to false since 1.5.0, and with -o modern for a few years before that. Performance is fine, no reported bugs. Always was the intention. Code is simpler without the options.
* -Y [filename] for ascii authentication modedormando2019-05-201-0/+17
| | | | | | | | | | | | | | | | | | | | | | | | | Loads "username:password\n" tokens (up to 8) out of a supplied authfile. If enabled, disables binary protocol (though may be able to enable both if sasl is also used?). authentication is done via the "set" command. A separate handler is used to avoid some hot path conditionals and narrow the code executed in an unauthenticated state. ie: set foo 0 0 7\r\n foo bar\r\n returns "STORED" on success. Else returns CLIENT_ERROR with some information. Any key is accepted: if using a client that doesn't try to authenticate when connecting to a pool of servers, the authentication set can be tried with the same key as one that failed to coerce the client to routing to the correct server. Else an "auth" or similar key would always go to the same server.
View Lorry Controller Status