summaryrefslogtreecommitdiff
path: root/logger.c
Commit message (Collapse)AuthorAgeFilesLines
* log: Add a new watcher to watch for deletions.Hemal Shah2023-03-151-0/+33
| | | | | | | `watch deletions`: would log all keys which are deleted using either `delete` or `md` command. The log line would contain the command used, the key, the clsid and size of the deleted item. Items which result in delete miss or are marked as stale wouldn't show up in the logs
* meta: N flag changes append/prepend. ms s flag.dormando2023-03-111-2/+2
| | | | | | | | | Sending 's' flag to metaset now returns the size of the item stored. Useful if you want to know how large an append/prepended item now is. If the 'N' flag is supplied while in append/prepend mode, allows autovivifying (with exptime supplied from N) for append/prepend style keys that don't need headers created first.
* log: fix race condition while incrementing log entries droppedRamasai2023-03-081-1/+1
|
* fix few unitialized data.David Carlier2023-03-081-1/+1
|
* proxy: add read buffer data to backend errorsdormando2023-02-151-2/+60
| | | | | | Errors like "trailing data" or "missingend" or etc are only useful if you're in a debugger and can break and inspect. This adds detail in uriencoding into the log message when applicable.
* proxy: fix mcp.log_req crash on nil resdormando2022-11-241-2/+7
| | | | | | If for some reason a res object is not passed into mcp.log_req or mcp.log_reqsample it would still strlen() the backend name/port's, which could crash.
* core: give threads unique namesdormando2022-11-011-0/+1
| | | | | allow users to differentiate thread functions externally to memcached. Useful for setting priorities or pinning threads to CPU's.
* Fix log timestamps after 2038Bernhard M. Wiedemann2022-10-131-16/+16
| | | | | | | | | | without this patch, tests failed with # Failed test 'saw a real log line after a skip' # at t/watcher.t line 54. # 'ts=-2126224733.765946 gid=80003 type=item_get key=foo status=not_found clsid=0 cfd=23 size=0 # ' # doesn't match '(?^:ts=\d+\.\d+\ gid=\d+ type=item_get)'
* log: fix obscure crashes due to size_t promotiondormando2022-08-121-3/+3
| | | | | | | | | It dawns on me now that I should've been hinting or forcing the type at all of the LOGGER_LOG endpoints to ensure they match the va_arg side on logger.c. This patch initially swaps size_t's for int's so at least it's using less complicated promotion rules. Fixes a crash on centos 7 when logging sets via `watch mutations`
* Race leads to deadlock during shutdown (sigterm/sigusr1)Sailesh Mukil2022-08-101-0/+4
| | | | | | | | | | | | | We've noticed in some rare cases, that sending a sigterm to a memcached process could lead to a race condition which leads to a deadlock of the process. The worker threads are stuck on 'worker_hang_lock' because the shutdown process hasn't completed yet. The shudown process hasn't completed since the logger thread signals 'logger_stack_cond' without locking its corresponding 'logger_stack_lock'. This could cause the signal to get lost, causing the logger thread to wait indefinitely on 'logger_stack_cond' in logger_thread().
* proxy: replace proxycmds stream with proxyreqsdormando2022-04-081-30/+1
| | | | | | | | | delete the magic logging and require mcp.log_req* be used if you want those types of entries to appear. keeps a separate data stream from "proxyuser" just in case that's useful. proxycmds wasn't able to get enough context to autogenerate useful log lines, so I'd rather not have it in there at all.
* proxy: mcp.log_req* API interfacedormando2022-04-081-0/+65
| | | | | | Lua level API for logging full context of a request/response. Provides log_req() for simple logging and log_reqsample() for conditional logging.
* proxy: add ring_hash builtindormando2022-02-241-1/+1
| | | | | | | | | | | | | this is ketama-based, with options for minor compat changes with major libraries. does _not_ support weights. The weights bits in the original ketama broke the algorithm, as changing the number of points would shift unrelated servers when the list changes. this also changes backends to take a "name" specifically, instead of an "ip address". Though note if supplying a hostname instead of an IP there might be inline DNS lookups on reconnects.
* proxy: fix logger entry memory corruptiondormando2022-02-241-0/+1
| | | | | | wasn't setting e->size of the log entry, so the data would corrupt. think I need an assert on e->size not being set from the function?
* proxy: add some basic logging for backend errorsdormando2022-02-111-0/+3
|
* proxy: logging improvements + lua mcp.log()dormando2022-02-111-4/+8
| | | | | | | adds watch commands for: proxycmds - internal raw timing log (tbd?) proxyevents - config updates, internal errors, etc proxyuser - logs generated by mcp.log()
* proxy: initial commit.dormando2021-10-051-0/+37
| | | | | | | | | | | | | | | See BUILD for compilation details. See t/startfile.lua for configuration examples. (see also https://github.com/memcached/memcached-proxylibs for extensions, config libraries, more examples) NOTE: io_uring mode is _not stable_, will crash. As of this commit it is not recommended to run the proxy in production. If you are interested please let us know, as we are actively stabilizing for production use.
* Expose number of currently active watchers in stats1.6.11Kevin Lin2021-09-271-2/+7
| | | | | The stat key `log_watchers` indicates the number of active connected `watch` clients.
* Properly accommodate IPv6 addresses in LOG_CONNEVENTS logsKevin Lin2021-09-271-4/+5
|
* logger: simplify logging codedormando2021-08-091-293/+243
| | | | | | | | | | Adding a new type of log entry required updating code in a bunch of places. Now we use callbacks in the entry structure instead, cutting down the code and some awkwardness. Should also make it easier for plugins/extra systems to add log entries. Moving the writer/parsers to their own section makes the actual logging code easier to follow, as an added bonus.
* logger: avoid polling without watchersdormando2021-08-091-1/+10
| | | | | | | | | Inspired by #768 - logger thread would wake up once per second looking for watchers or log entries, but without any watchers. Now sleeps on a condition. Note that if I ever actually get logger to syslog/stdout/stderr/etc working properly this will still always poll.
* Implement LOG_CONNEVENTS watcher flag for connection state transitionsKevin Lin2021-08-071-0/+100
| | | | | | | | Add support for `watch connevents` to report opened (`conn_new`) and closed (`conn_close`) client connections. Event log lines indicate the connection's remote IP, remote port, and transport type. `conn_close` events additionally supply a reason for the closing the connection.
* Report item sizes for fetch, mutation, and eviction watchersKevin Lin2021-08-061-10/+19
| | | | | | This adds a new field `size` to logger entry lines for item_get, item_store, and eviction events indicating the size of the associated item in bytes.
* logger: fix spurious watcher hangupsdormando2020-10-291-1/+1
| | | | | | | | | of course the previous commit didn't actually fix the bug: during poll() call we weren't filtering for the right watcher when we were polling a specific one. This could lead to reading a POLLHUP on the wrong socket and other issues. hopefully that solves some mysteries with the watcher system.
* fix potential deadlock bug in log watcherneal-zhu2019-10-171-0/+1
| | | | If too many watchers exist, fails to release logger stack lock.
* restartable cachedormando2019-09-171-12/+22
| | | | | | | | | | | | | | | "-e /path/to/tmpfsmnt/file" SIGUSR1 for graceful stop restart requires the same memory limit, slab sizes, and some other infrequently changed details. Most other options and features can change between restarts. Binary can be upgraded between restarts. Restart does some fixup work on start for every item in cache. Can take over a minute with more than a few hundred million items in cache. Keep in mind when a cache is down it may be missing invalidations, updates, and so on.
* log client connection id with fetchers and mutationsTharanga Gamaethige2019-08-271-9/+12
| | | | you can now monitor fetch and mutations of a given client
* Fix off-by-one in logger to allow CAS commands to be logged.Andrew Drake2019-08-271-1/+1
|
* Use correct buffer size for internal URI encoding.Tharanga Gamaethige2019-05-201-8/+8
| | | | | Modified Logger and Crawler to use the correct buffer length when they are printing URI encoded keys. Fixes #471
* fix compile error on centos7dormando2019-04-271-0/+1
| | | | I have no idea why this worked without the include? fixes #478
* Basic implementation of TLS for memcached.1.5.13Tharanga Gamaethige2019-04-151-2/+2
| | | | | | | | | | | | | Most of the work done by Tharanga. Some commits squashed in by dormando. Also reviewed by dormando. Tested, working, but experimental implementation of TLS for memcached. Enable with ./configure --enable-tls Requires OpenSSL 1.1.0 or better. See `memcached -h` output for usage.
* add missing va_end() call to logger_log()Miroslav Lichvar2018-07-111-0/+1
|
* Fix SIGBUS from alignment issues on 64bit ARMdormando2018-03-151-3/+11
| | | | | | | | | | | ARMv8 (and in general aarch64) has flipped some strictness requirements. However, at some point in history the NEED_ALIGN configure check code was optimized away by GCC. This fixes detection of alignment, as well as fixes an unaligned access that snuck in via the logging code. Also fixes a 64bit GCC atomics test that possibly never worked before.
* extstore: add evictions-related write logsdormando2017-12-151-1/+45
| | | | | | "watch evictions" will show a stream of evictions + writes to extstore. useful for analyzing the remaining ttl or key pattern of stuff being flushed.
* extstore: skip unhit objects if full in compactiondormando2017-11-281-1/+1
| | | | | | if < 2 free pages left, "evict" objects which haven't been hit at all. should be better than evicting everything if we can continue compacting.
* external storage base commitdormando2017-11-281-1/+21
| | | | | been squashing reorganizing, and pulling code off to go upstream ahead of merging the whole branch.
* make for loop more clear in logger watcherdormando2017-10-291-1/+1
| | | | | | | This wasn't a bug, but add -1 here makes it more clear. The function will early exit in any cases where it would've overflowed here. thanks to shqking on github for the report
* add a real slab automover algorithmdormando2017-06-231-0/+3
| | | | converts the python script to C, more or less.
* LRU crawler scheduling improvementsdormando2017-05-291-1/+1
| | | | | | | | | | | | | | | | | | | when trying to manually run a crawl, the internal autocrawler is now blocked from restarting for 60 seconds. the internal autocrawl now independently schedules LRU's, and can re-schedule sub-LRU's while others are still running. should allow much better memory control when some sub-lru's (such as TEMP or WARM) are small, or slab classes are differently sized. this also makes the crawler drop its lock frequently.. this fixes an issue where a long crawl happening at the same time as a hash table expansion could hang the server until the crawl finished. to improve still: - elapsed time can be wrong in the logger entry - need to cap number of entries scanned. enough set pressure and a crawl may never finish.
* Sleep more aggressively in some threadsGrant Mathews2017-05-221-5/+9
| | | | | | | | | | | The logger and lru maintainer threads both adjust how long they sleep based on how busy they are. This adjustment should be exponential, to more quickly adjust to workloads. The logger thread in particular would only adjust 50 microseconds at a time, and was capped at 100 milliseconds of sleep, causing many unnecessary wakeups on an otherwise idle dev machine. Adjust this cap to 1 second.
* log class id with fetchers and mutationsdormando2017-01-041-11/+17
| | | | you can now cross-reference slab classes of evicted prefixes with misses
* fix store log and add ttldormando2017-01-041-6/+12
| | | | | the TTL is relative so you won't see the exact TTL on entry. Also fixes two off-by-one's for the status values.
* tune crawler logger, fix crawler watermarkdormando2016-12-301-1/+1
| | | | | should be looking for 1% of potential reclaims, not 1% of all the items it saw. Otherwise could be wedged into doing nothing.
* add logging endpoint for LRU crawlerdormando2016-12-301-1/+4
| | | | for the internal checker, primarily.
* stop pushing NULl byte into watcher streamdormando2016-12-271-4/+6
| | | | also fixes logger L_DEBUG prints.
* Missing necessary header for atomic_inc_64_nv() used in logger.cAlexander Pyhalov2016-12-191-0/+4
|
* make logger use uriencode out of util.cdormando2016-08-191-39/+3
|
* don't fail on systems without 64bit atomicsdormando2016-07-011-2/+2
| | | | | | | | | We only explicitly tested for short atomics. Then I added 8byte atomic usage in the logger: now separately test for those.. since some systems can have one but not the other, I'd rather use atomics where available then fall back to a mutex when necessary. Reproduced this on a raspberry pi 2.
* split up logger_thread_parse_entrydormando2016-07-011-57/+76
| | | | | | now a main function + helpers, with a separate write function. drives me a lot less crazy now.
* first refactor of logger_thread_parse_entrydormando2016-07-011-94/+46
| | | | | | Was in a working state for release, now doing some work to clean up the routine. Is much shorter and clearer now. Function needs to be split into several functions from here.
View Lorry Controller Status