summaryrefslogtreecommitdiff
path: root/logger.h
Commit message (Collapse)AuthorAgeFilesLines
* log: Add a new watcher to watch for deletions.Hemal Shah2023-03-151-0/+10
| | | | | | | `watch deletions`: would log all keys which are deleted using either `delete` or `md` command. The log line would contain the command used, the key, the clsid and size of the deleted item. Items which result in delete miss or are marked as stale wouldn't show up in the logs
* proxy: add read buffer data to backend errorsdormando2023-02-151-0/+9
| | | | | | Errors like "trailing data" or "missingend" or etc are only useful if you're in a debugger and can break and inspect. This adds detail in uriencoding into the log message when applicable.
* proxy: replace proxycmds stream with proxyreqsdormando2022-04-081-8/+1
| | | | | | | | | delete the magic logging and require mcp.log_req* be used if you want those types of entries to appear. keeps a separate data stream from "proxyuser" just in case that's useful. proxycmds wasn't able to get enough context to autogenerate useful log lines, so I'd rather not have it in there at all.
* proxy: mcp.log_req* API interfacedormando2022-04-081-0/+13
| | | | | | Lua level API for logging full context of a request/response. Provides log_req() for simple logging and log_reqsample() for conditional logging.
* proxy: add some basic logging for backend errorsdormando2022-02-111-0/+1
|
* proxy: logging improvements + lua mcp.log()dormando2022-02-111-0/+4
| | | | | | | adds watch commands for: proxycmds - internal raw timing log (tbd?) proxyevents - config updates, internal errors, etc proxyuser - logs generated by mcp.log()
* proxy: initial commit.dormando2021-10-051-1/+13
| | | | | | | | | | | | | | | See BUILD for compilation details. See t/startfile.lua for configuration examples. (see also https://github.com/memcached/memcached-proxylibs for extensions, config libraries, more examples) NOTE: io_uring mode is _not stable_, will crash. As of this commit it is not recommended to run the proxy in production. If you are interested please let us know, as we are actively stabilizing for production use.
* Expose number of currently active watchers in stats1.6.11Kevin Lin2021-09-271-0/+1
| | | | | The stat key `log_watchers` indicates the number of active connected `watch` clients.
* Properly accommodate IPv6 addresses in LOG_CONNEVENTS logsKevin Lin2021-09-271-1/+1
|
* logger: simplify logging codedormando2021-08-091-18/+13
| | | | | | | | | | Adding a new type of log entry required updating code in a bunch of places. Now we use callbacks in the entry structure instead, cutting down the code and some awkwardness. Should also make it easier for plugins/extra systems to add log entries. Moving the writer/parsers to their own section makes the actual logging code easier to follow, as an added bonus.
* logger: avoid polling without watchersdormando2021-08-091-0/+1
| | | | | | | | | Inspired by #768 - logger thread would wake up once per second looking for watchers or log entries, but without any watchers. Now sleeps on a condition. Note that if I ever actually get logger to syslog/stdout/stderr/etc working properly this will still always poll.
* Implement LOG_CONNEVENTS watcher flag for connection state transitionsKevin Lin2021-08-071-0/+11
| | | | | | | | Add support for `watch connevents` to report opened (`conn_new`) and closed (`conn_close`) client connections. Event log lines indicate the connection's remote IP, remote port, and transport type. `conn_close` events additionally supply a reason for the closing the connection.
* Report item sizes for fetch, mutation, and eviction watchersKevin Lin2021-08-061-0/+3
| | | | | | This adds a new field `size` to logger entry lines for item_get, item_store, and eviction events indicating the size of the associated item in bytes.
* restartable cachedormando2019-09-171-1/+8
| | | | | | | | | | | | | | | "-e /path/to/tmpfsmnt/file" SIGUSR1 for graceful stop restart requires the same memory limit, slab sizes, and some other infrequently changed details. Most other options and features can change between restarts. Binary can be upgraded between restarts. Restart does some fixup work on start for every item in cache. Can take over a minute with more than a few hundred million items in cache. Keep in mind when a cache is down it may be missing invalidations, updates, and so on.
* log client connection id with fetchers and mutationsTharanga Gamaethige2019-08-271-0/+2
| | | | you can now monitor fetch and mutations of a given client
* Fix SIGBUS from alignment issues on 64bit ARMdormando2018-03-151-0/+1
| | | | | | | | | | | ARMv8 (and in general aarch64) has flipped some strictness requirements. However, at some point in history the NEED_ALIGN configure check code was optimized away by GCC. This fixes detection of alignment, as well as fixes an unaligned access that snuck in via the logging code. Also fixes a 64bit GCC atomics test that possibly never worked before.
* extstore: add evictions-related write logsdormando2017-12-151-2/+16
| | | | | | "watch evictions" will show a stream of evictions + writes to extstore. useful for analyzing the remaining ttl or key pattern of stuff being flushed.
* external storage base commitdormando2017-11-281-0/+8
| | | | | been squashing reorganizing, and pulling code off to go upstream ahead of merging the whole branch.
* add a real slab automover algorithmdormando2017-06-231-0/+1
| | | | converts the python script to C, more or less.
* Spelling fixesJosh Soref2017-05-231-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * accesses * amount * append * command * cyrillic * daemonize * detaches * detail * documentation * dynamically * enabled * existence * extra * implementations * incoming * increment * initialize * issue * javascript * number * optimization * overall * pipeline * reassign * reclaimed * response * responses * sigabrt * specific * specificity * tidiness
* log class id with fetchers and mutationsdormando2017-01-041-0/+3
| | | | you can now cross-reference slab classes of evicted prefixes with misses
* fix store log and add ttldormando2017-01-041-0/+4
| | | | | the TTL is relative so you won't see the exact TTL on entry. Also fixes two off-by-one's for the status values.
* add logging endpoint for LRU crawlerdormando2016-12-301-1/+2
| | | | for the internal checker, primarily.
* logger endpoints for first releasedormando2016-06-161-2/+22
| | | | | | | | | | | swapping "RAWCMDS" for internal hooks on when items are fetched or stored. This doesn't map 1:1 with commands, ie: a store is internally a fetch then store, so two log lines are generated. An ascii multiget one make one log line per key fetched. It's a good place to start. Need to come back and refactor parts of logger.c again, then convert all prints in the codebase to log entries.
* convert eviction log to new processdormando2016-06-161-0/+17
| | | | | | | | | | | | add LOGGER_LOG() macro wrapper for callers. eviction logging is now using the "more final" method, where data is copied into structs embedded into the per-worker circular buffer. This moves the expensive snprintf calls to the background thread, and also allows making the logging format switchable. also switched the format to key=value. it's still largely readable but much easier to parse.
* some global stats for bg loggerdormando2016-06-161-1/+9
| | | | | four obvious ones. have a handy place to do the summarization when the logger wakes up for its run, avoiding the locks.
* improve 'watch' command processing a littledormando2016-06-161-1/+0
| | | | | moves to its own function, respods "OK\r\n" to client, allows multiple arguments for multiple flags. Needs more thought before adding sampling.
* poll() smarter when full, print skipped countdormando2016-06-161-2/+2
| | | | | | | | | When lines are skipped to a watcher, a [skipped: n] is printed before resuming. Also polls full watcher directly once, and will wait until the outer loop to re-poll. Think the routine can be simplified... it works so will leave it and revisit later.
* use one bipbuffer per watcher. remove iovec code.dormando2016-06-161-5/+5
| | | | | | | | | | | | | after the previous commit's exercise, this greatly simplifies the writing process. The buffers are large but can be tuned with performance testing. Lines are now rendered into scratch space, and if event flags (eflags) match for a watcher, copy the bytes into that watcher's specific buffer. In most realistic cases, watchers will be watching different streams of information. While this does require a minimum of one memcpy instead of directly writing into the bipbuf as before, it removes all of the loops and management of iovecs required for filtering events on the write-to-socket end.
* remove "logger chunks", add individualized streamsdormando2016-06-161-11/+6
| | | | | | | | | | | | | | | | | | | Stop doing a multi-reader circular buffer structure on top of a circular buffer. Also adds individualized streams based off of the central buffer. Sadly this requires managing iovecs and dealing with partial writes into said iovecs. That makes things very complicated. Since it's not clear to me how to simplify it too much (as of this writing), one of the next commits should remove iovecs and instead give each watcher its own circular buffer. The parser thread will copy watched events into each buffer. The above would only be slower for the case of many watchers watching the same event streams. Given all of the extra loops required for managing the iovecs, and the more complicated syscall handling, it might even be the same speed to manage multiple buffers anyway. I completed this intermediary change since it simplifies the surrounding code and was educational to fiddle with iovecs again.
* switch from bit fields to bit manipulationdormando2016-06-161-14/+11
| | | | can't compare bit fields en-masse, which makes things too difficult.
* manage logger watcher flags.dormando2016-06-161-10/+15
| | | | | very temporary user control. allows to watch either fetchers or evictions, but not both, and always with timestamps.
* initial logger code.dormando2016-06-161-0/+114
Logs are written to per-thread buffers. A new background thread aggregates the logs, further processes them, then writes them to any "watchers". Logs can have the time added to them, and all have a GID so they can be put back into strict order. This is an early preview. Code needs refactoring and a more complete set of options. All watchers are also stuck viewing the global feed of logs, even if they asked for different data. As of this commit there's no way to toggle the "stderr" watcher.
View Lorry Controller Status