| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
"-e /path/to/tmpfsmnt/file"
SIGUSR1 for graceful stop
restart requires the same memory limit, slab sizes, and some other
infrequently changed details. Most other options and features can
change between restarts. Binary can be upgraded between restarts.
Restart does some fixup work on start for every item in cache. Can take
over a minute with more than a few hundred million items in cache.
Keep in mind when a cache is down it may be missing invalidations,
updates, and so on.
|
|
|
|
| |
you can now monitor fetch and mutations of a given client
|
| |
|
|
|
|
|
| |
Modified Logger and Crawler to use the correct buffer length when
they are printing URI encoded keys. Fixes #471
|
|
|
|
| |
I have no idea why this worked without the include? fixes #478
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Most of the work done by Tharanga. Some commits squashed in by
dormando. Also reviewed by dormando.
Tested, working, but experimental implementation of TLS for memcached.
Enable with ./configure --enable-tls
Requires OpenSSL 1.1.0 or better.
See `memcached -h` output for usage.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
ARMv8 (and in general aarch64) has flipped some strictness requirements.
However, at some point in history the NEED_ALIGN configure check code was
optimized away by GCC.
This fixes detection of alignment, as well as fixes an unaligned access that
snuck in via the logging code.
Also fixes a 64bit GCC atomics test that possibly never worked before.
|
|
|
|
|
|
| |
"watch evictions" will show a stream of evictions + writes to extstore.
useful for analyzing the remaining ttl or key pattern of stuff being
flushed.
|
|
|
|
|
|
| |
if < 2 free pages left, "evict" objects which haven't been hit at all.
should be better than evicting everything if we can continue compacting.
|
|
|
|
|
| |
been squashing reorganizing, and pulling code off to go upstream ahead
of merging the whole branch.
|
|
|
|
|
|
|
| |
This wasn't a bug, but add -1 here makes it more clear. The function will
early exit in any cases where it would've overflowed here.
thanks to shqking on github for the report
|
|
|
|
| |
converts the python script to C, more or less.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
when trying to manually run a crawl, the internal autocrawler is now blocked
from restarting for 60 seconds.
the internal autocrawl now independently schedules LRU's, and can re-schedule
sub-LRU's while others are still running. should allow much better memory
control when some sub-lru's (such as TEMP or WARM) are small, or slab classes
are differently sized.
this also makes the crawler drop its lock frequently.. this fixes an issue
where a long crawl happening at the same time as a hash table expansion could
hang the server until the crawl finished.
to improve still:
- elapsed time can be wrong in the logger entry
- need to cap number of entries scanned. enough set pressure and a crawl may
never finish.
|
|
|
|
|
|
|
|
|
|
|
| |
The logger and lru maintainer threads both adjust how long they sleep
based on how busy they are. This adjustment should be exponential, to
more quickly adjust to workloads.
The logger thread in particular would only adjust 50 microseconds at a
time, and was capped at 100 milliseconds of sleep, causing many
unnecessary wakeups on an otherwise idle dev machine. Adjust this cap to
1 second.
|
|
|
|
| |
you can now cross-reference slab classes of evicted prefixes with misses
|
|
|
|
|
| |
the TTL is relative so you won't see the exact TTL on entry. Also fixes two
off-by-one's for the status values.
|
|
|
|
|
| |
should be looking for 1% of potential reclaims, not 1% of all the items it
saw. Otherwise could be wedged into doing nothing.
|
|
|
|
| |
for the internal checker, primarily.
|
|
|
|
| |
also fixes logger L_DEBUG prints.
|
| |
|
| |
|
|
|
|
|
|
|
|
|
| |
We only explicitly tested for short atomics. Then I added 8byte atomic usage
in the logger: now separately test for those.. since some systems can have one
but not the other, I'd rather use atomics where available then fall back to
a mutex when necessary.
Reproduced this on a raspberry pi 2.
|
|
|
|
|
|
| |
now a main function + helpers, with a separate write function.
drives me a lot less crazy now.
|
|
|
|
|
|
| |
Was in a working state for release, now doing some work to clean up the
routine. Is much shorter and clearer now. Function needs to be split into
several functions from here.
|
|
|
|
|
|
|
| |
the defaults should just about always be used, but this is now tunable for
completeness. Also handy for writing tests.
This also hopefully makes the watcher.t test not flaky anymore.
|
|
|
|
| |
‘snprintf’
|
|
|
|
| |
and a note for missing information.
|
|
|
|
| |
tests pass now.
|
|
|
|
|
|
|
|
|
|
|
| |
swapping "RAWCMDS" for internal hooks on when items are fetched or stored.
This doesn't map 1:1 with commands, ie: a store is internally a fetch then
store, so two log lines are generated. An ascii multiget one make one log line
per key fetched.
It's a good place to start. Need to come back and refactor parts of logger.c
again, then convert all prints in the codebase to log entries.
|
|
|
|
|
|
|
|
|
|
|
|
| |
add LOGGER_LOG() macro wrapper for callers.
eviction logging is now using the "more final" method, where data is copied
into structs embedded into the per-worker circular buffer. This moves the
expensive snprintf calls to the background thread, and also allows making the
logging format switchable.
also switched the format to key=value. it's still largely readable but much
easier to parse.
|
|
|
|
| |
makes a map on start of bgthread to speed up conversion a little.
|
|
|
|
|
|
|
| |
put header sections for utils/bgthread/api, edit function names slightly so
it's clear what's only called from the bgthread.
remaining FIXME/TODO's appear noncritical. might do those in the future.
|
|
|
|
|
| |
four obvious ones. have a handy place to do the summarization when the logger
wakes up for its run, avoiding the locks.
|
|
|
|
|
| |
moves to its own function, respods "OK\r\n" to client, allows multiple
arguments for multiple flags. Needs more thought before adding sampling.
|
|
|
|
|
|
|
|
|
| |
When lines are skipped to a watcher, a [skipped: n] is printed before
resuming. Also polls full watcher directly once, and will wait until the
outer loop to re-poll.
Think the routine can be simplified... it works so will leave it and revisit
later.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
after the previous commit's exercise, this greatly simplifies the writing
process. The buffers are large but can be tuned with performance testing.
Lines are now rendered into scratch space, and if event flags (eflags) match
for a watcher, copy the bytes into that watcher's specific buffer.
In most realistic cases, watchers will be watching different streams of
information. While this does require a minimum of one memcpy instead of
directly writing into the bipbuf as before, it removes all of the loops and
management of iovecs required for filtering events on the write-to-socket end.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Stop doing a multi-reader circular buffer structure on top of a circular
buffer. Also adds individualized streams based off of the central buffer.
Sadly this requires managing iovecs and dealing with partial writes into said
iovecs. That makes things very complicated. Since it's not clear to me how to
simplify it too much (as of this writing), one of the next commits should
remove iovecs and instead give each watcher its own circular buffer. The
parser thread will copy watched events into each buffer.
The above would only be slower for the case of many watchers watching the same
event streams. Given all of the extra loops required for managing the iovecs,
and the more complicated syscall handling, it might even be the same speed to
manage multiple buffers anyway.
I completed this intermediary change since it simplifies the surrounding code
and was educational to fiddle with iovecs again.
|
|
|
|
| |
can't compare bit fields en-masse, which makes things too difficult.
|
|
|
|
|
| |
very temporary user control. allows to watch either fetchers or evictions, but
not both, and always with timestamps.
|
|
Logs are written to per-thread buffers. A new background thread aggregates the
logs, further processes them, then writes them to any "watchers".
Logs can have the time added to them, and all have a GID so they can be put
back into strict order.
This is an early preview. Code needs refactoring and a more complete set of
options. All watchers are also stuck viewing the global feed of logs, even if
they asked for different data.
As of this commit there's no way to toggle the "stderr" watcher.
|