summaryrefslogtreecommitdiff
path: root/t
Commit message (Collapse)AuthorAgeFilesLines
* core: fix another dtrace compilation issue1.6.19dormando2023-03-081-4/+4
|
* proxy: mcp.internal fixes and testsdormando2023-03-062-0/+239
| | | | | | | | | | | - Refcount leak on sets - Move the response elapsed timer back closer to when the response was processed as to not clobber the wrong IO object data - Restores error messages from set/ms - Adds start of unit tests Requests will look like they run a tiiiiny bit faster than they do, but I need to get the elapsed time there for a later change.
* proxy: more await unit testsdormando2023-03-012-1/+113
| | | | test FASTGOOD and some set scenarios
* proxy: fix trailingdata error with ascii multigetdormando2023-02-281-0/+15
| | | | | | | | One of the side effects of pre-warming all of the tests I did with multiget, and not having done a second round on the unit tests, is that we somehow never tried an ascii multiget against a damn miss. Easy to test, easy to fix.
* proxy: allow workers to run IO optionallydormando2023-02-242-0/+114
| | | | | | | | | | | | | | | | | `mcp.pool(p, { dist = etc, iothread = true }` By default the IO thread is not used; instead a backend connection is created for each worker thread. This can be overridden by setting `iothread = true` when creating a pool. `mcp.pool(p, { dist = etc, beprefix = "etc" }` If a `beprefix` is added to pool arguments, it will create unique backend connections for this pool. This allows you to create multiple sockets per backend by making multiple pools with unique prefixes. There are legitimate use cases for sharing backend connections across different pools, which is why that is the default behavior.
* proxy: redo libevent handling codedormando2023-02-221-4/+6
| | | | | | | | | | | | | | | | | | The event handling code was unoptimized and temporary; it was slated for a rewrite for performance and non-critical bugs alone. However the old code may be causing critical bugs so it's being rewritten now. Fixes: - backend disconnects are detected immediately instead of on the next time they are used. - backend reconnects happen _after_ the retry timeout, not before - use a persistent read handler and a temporary write handler to avoid constantly calling epoll_ctl syscalls for potential performance boost. Updated some tests for proxyconfig.t as it was picking up the disconnects immediately. Unrelated to a timing issue I resolved to the benchmark.
* proxy: fix "missingend" error on reading responsesdormando2023-02-171-0/+55
| | | | | | | | | | | | | | If the backend handler reads an incomplete response from the network, it changes state to wait for more data. The want_read state was considering the data completed if "data read" was bigger than "value length", but it should have been "value + result line". This means if the response buffer landed in a bullseye where it has read more than the size of the value but less than the total size of the request (typically a span of 200 bytes or less), it would consider the request complete and look for the END\r\n marker. This change has been... here forever.
* proxy: add read buffer data to backend errorsdormando2023-02-151-0/+26
| | | | | | Errors like "trailing data" or "missingend" or etc are only useful if you're in a debugger and can break and inspect. This adds detail in uriencoding into the log message when applicable.
* proxy: fix partial responses on backend timeoutsdormando2023-02-142-6/+23
| | | | | | | Response object error conditions were not being checked before looking at the response buffer. If a response was partially filled then the backend timed out, a partial response could be sent intead of the proper backend error.
* proxy: add mcp.backend(t) for more overridesdormando2023-02-012-2/+53
| | | | | | | | | | ie: local b1 = mcp.backend({ label = "b1", host = "127.0.0.1", port = 11511, connecttimeout = 1, retrytimeout = 0.5, readtimeout = 0.1, failurelimit = 11 }) ... to allow for overriding connect/retry/etc tunables on a per-backend basis. If not passed in the global settings are used.
* proxy: add mcp.await_logerrors()dormando2023-01-272-0/+39
| | | | | | | Logs any backgrounded requests that resulted in an error. Note that this may be a temporary interface, and could be deprecated in the future.
* proxy: new integration tests.dormando2023-01-255-0/+933
| | | | | | | | uses mocked backend servers so we can test: - end to end client to backend proxying - lua API functions - configuration reload - various error conditions
* core: remove *conn object from cache commandsdormando2023-01-112-2/+6
| | | | | | | | | We want to start using cache commands in contexts without a client connection, but the client object has always been passed to all functions. In most cases we only need the worker thread (LIBEVENT_THREAD *t), so this change adjusts the arguments passed in.
* proxy: add mcp.AWAIT_BACKGROUNDdormando2022-12-011-4/+5
| | | | | | mcp.await(request, pools, 0, mcp.AWAIT_BACKGROUND) will, instead of waiting on any request to return, simply return an empty table as soon as the background requests are dispatched.
* meta: allow mg without flags + reflect O/k on ENdormando2022-10-201-1/+1
| | | | | | | | "mg" required at least one flag. now "mg key" returns a bare HD on hit if you don't care about the value. HD modes would reflect O and k flags in the response, but EN didn't. This is now fixed for full coverage.
* meta: meta arithmetic command had excess spacesdormando2022-10-201-5/+5
| | | | for HD/NF/etc responses but not VA.
* meta: remove excess spaces from meta responsesdormando2022-10-201-1/+9
| | | | was returning "HD \r\n" and "HD Oetc\r\n" - not to protocol spec.
* Find perl via /usr/bin/env instead of directlyDavid CARLIER2022-08-2583-83/+83
| | | | | At least FreeBSD has perl in /usr/local/bin/perl and no symlink by default.
* proxy: mcp.attach(CMD, r, "tag")dormando2022-08-241-0/+3
| | | | | | | | allows using tagged listeners (ex; `-l tag[test]:127.0.0.1:11212`) to select a top level route for a function. expects there to not be dozens of listeners, but for a handful will be faster than a hash table lookup.
* proxy: req:flag_token("F", "Freplacement")dormando2022-08-091-0/+2
| | | | | | returns (exists, previous_token) optional second argument will replace the flag/token with supplied flag/token, or nothing if "" is passed.
* proxy: add req:flag_token("F")dormando2022-08-031-0/+4
| | | | | | function accepts a flag, returns (bool, token|nil). bool indicates if the flag exists, and if the flag has a token it is returned instead of nil as the second value.
* proxy: mcp.response code and rline APIdormando2022-08-031-0/+15
| | | | | | | | | Adds resp:code(), which returns code you can compare with mcp.MCMC_CODE_* Adds resp:line(), which returns the raw response line after the command. This can be used in lua while the flag/token API is missing for the response object.
* proxy: add r:has_flag(), fix r:token() lengthdormando2022-08-031-0/+14
| | | | | | | | | Add mcp.request function for quick checking if a flag exists in a request string. Also updates internal code for checking the length of a token to use the endcap properly, and uses that for the r:token(n) requets as well, which fixes a subtle bug of the token length being too long.
* proxy: mcp.request improvementsdormando2022-08-031-0/+7
| | | | | | | | | | - errors if a string value is missing the "\r\n" terminator - properly uses a value from a response object - allows passing in a request object for the value as well - also adds r:vlen() for recovering the value length of a response think this still needs r:flags() or similar?
* tls: Add switch to opt-in to kernel TLS on OpenSSL 3.0.0+Kevin Lin2022-07-031-0/+1
|
* proxy: mcp.log_req* API interfacedormando2022-04-081-5/+14
| | | | | | Lua level API for logging full context of a request/response. Provides log_req() for simple logging and log_reqsample() for conditional logging.
* proxy: allow await() to be called recursivelydormando2022-03-021-3/+7
| | | | | | | | previously mcp.await() only worked if it was called before any other dispatches. also fixes a bug if the supplied pool table was key=value instead of an array-type table.
* proxy: add ring_hash builtindormando2022-02-242-31/+28
| | | | | | | | | | | | | this is ketama-based, with options for minor compat changes with major libraries. does _not_ support weights. The weights bits in the original ketama broke the algorithm, as changing the number of points would shift unrelated servers when the list changes. this also changes backends to take a "name" specifically, instead of an "ip address". Though note if supplying a hostname instead of an IP there might be inline DNS lookups on reconnects.
* storage: parameterize the compaction thread sleepdormando2022-02-215-5/+5
| | | | | | allows tests to run faster, let users make it sleep longer/less time. Also cuts the sleep time down when actively compacting and coming from high idle.
* proxy: trivial support for SO_KEEPALIVE on backenddormando2022-02-161-0/+1
| | | | | global only setting, not able to be changed for existing backends on config reload. a few fixes would improve but not can do later.
* proxy: logging improvements + lua mcp.log()dormando2022-02-111-0/+2
| | | | | | | adds watch commands for: proxycmds - internal raw timing log (tbd?) proxyevents - config updates, internal errors, etc proxyuser - logs generated by mcp.log()
* tests: workaround for t/watcher.t race1.6.14dormando2022-02-091-0/+1
| | | | need better option for this.. seems to fix this one thing.
* tests: repair race in maxconns.t testdormando2022-02-091-2/+11
| | | | | lsitener thread may not have actually accepted everything by the time we start closing them, so it might not show rejected connections.
* proxy: accept fractional time for timeoutsdormando2022-02-081-1/+1
| | | | | the uring bits make abstraction this to the obvious common function a little harder, so I skipped for now.
* proxy: add filter and hash options to mcp.pool()dormando2022-02-071-2/+4
| | | | | | | two builtin filter options (hash stop and tag), because why not :) hash defaults caused some code reorganization. default hash dist is now jump, because I can't think of why you'd use modulus over that.
* proxy: improve mcp_config_* VM table copyingdormando2022-02-041-4/+3
| | | | | | | The configuration reloader copies data between the "config" global VM and individual worker VM's. This fixes some crashes/limitations and improves error handling. It still has a sharp edge as the actual table copy is unhandled.
* proxy: comment out token test in startfile.luadormando2022-01-261-3/+3
| | | | didn't mean to leave that in there by default.
* meta: test coverage for cas fixesdormando2022-01-121-0/+35
|
* Fix full unit test suite under test_tlsKevin Lin2021-11-234-15/+36
| | | | | This allows `make test_tls` to reliably pass when running without parallelism.
* Track store errors in thread statsKevin Lin2021-11-232-6/+21
| | | | | | Add two new stat keys, `store_too_large` and `store_no_memory`, to track occurrences of storage request rejections due to writing too large of a value and writing beyond available provisioned memory, respectively.
* tests: maxconns test when extstore enableddormando2021-11-031-24/+42
| | | | seems to pass...
* proxy: initial commit.dormando2021-10-053-1/+550
| | | | | | | | | | | | | | | See BUILD for compilation details. See t/startfile.lua for configuration examples. (see also https://github.com/memcached/memcached-proxylibs for extensions, config libraries, more examples) NOTE: io_uring mode is _not stable_, will crash. As of this commit it is not recommended to run the proxy in production. If you are interested please let us know, as we are actively stabilizing for production use.
* Expose number of currently active watchers in stats1.6.11Kevin Lin2021-09-272-3/+32
| | | | | The stat key `log_watchers` indicates the number of active connected `watch` clients.
* Configurable minimum supported TLS protocol versionKevin Lin2021-09-272-1/+47
| | | | | | | `-o ssl_min_version` can be used to configure the server to only accept handshakes from clients with a minimum TLS protocol version. Currently supported options are TLS v1.0, TLS v1.1, TLS v1.2, and TLS v1.3 (OpenSSL 1.1.1+ only).
* core: fix hang bug in extstoredormando2021-08-091-0/+8
| | | | | | | | | | The previous commit (IO queue submit if queue is not NULL, instead of queue count) exposed an apparent long standing bug in extstore: do an ascii-multiget request with multiple keys, but one too-long or otherwise garbage key. This will reset the queue count to 0 but leave a stack pointer to now freed memory. The next request could infinite loop, as this test was causing.
* core: add queue.h to replace handrolled queues.dormando2021-08-091-0/+1
| | | | | FreeBSD version as there seems to be portability issues if you just use sys/queue, plus some extra features I like in this one.
* Implement LOG_CONNEVENTS watcher flag for connection state transitionsKevin Lin2021-08-071-1/+35
| | | | | | | | Add support for `watch connevents` to report opened (`conn_new`) and closed (`conn_close`) client connections. Event log lines indicate the connection's remote IP, remote port, and transport type. `conn_close` events additionally supply a reason for the closing the connection.
* Report item sizes for fetch, mutation, and eviction watchersKevin Lin2021-08-061-1/+24
| | | | | | This adds a new field `size` to logger entry lines for item_get, item_store, and eviction events indicating the size of the associated item in bytes.
* Add settings stat for shutdown_command enabledKevin Lin2021-07-251-1/+2
|
* meta: response code OK -> HDdormando2021-06-101-32/+32
| | | | | | | | | | | | | | I had the response code as "HD" in the past, but standardized on OK while merging a number of "OK-like" rescodes together. This was a mistake; as many "generic" memcached response codes use "OK". Most of these are management or specialized uncommon commands. With this, a client response parser can know for sure if a response is to a meta command, or some other command. `-o meta_response_old` starttime option has been added, valid for the next 3 months, which switches the response code back from HD to OK. In case any existing users depended on this and need time to migrate.
View Lorry Controller Status