| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Stop doing a multi-reader circular buffer structure on top of a circular
buffer. Also adds individualized streams based off of the central buffer.
Sadly this requires managing iovecs and dealing with partial writes into said
iovecs. That makes things very complicated. Since it's not clear to me how to
simplify it too much (as of this writing), one of the next commits should
remove iovecs and instead give each watcher its own circular buffer. The
parser thread will copy watched events into each buffer.
The above would only be slower for the case of many watchers watching the same
event streams. Given all of the extra loops required for managing the iovecs,
and the more complicated syscall handling, it might even be the same speed to
manage multiple buffers anyway.
I completed this intermediary change since it simplifies the surrounding code
and was educational to fiddle with iovecs again.
|
| |
|
|
| |
can't compare bit fields en-masse, which makes things too difficult.
|
| |
|
|
|
| |
very temporary user control. allows to watch either fetchers or evictions, but
not both, and always with timestamps.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Logs are written to per-thread buffers. A new background thread aggregates the
logs, further processes them, then writes them to any "watchers".
Logs can have the time added to them, and all have a GID so they can be put
back into strict order.
This is an early preview. Code needs refactoring and a more complete set of
options. All watchers are also stuck viewing the global feed of logs, even if
they asked for different data.
As of this commit there's no way to toggle the "stderr" watcher.
|
| |
|
|
| |
total_items is pretty easy to overflow. Upped some of the others just in case.
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
APPEND_NUM_FMT_STAT is a macro that expands to multiple statements, so it must have braces when used in a conditional.
|
| | |
|
| |
|
|
|
|
|
|
| |
gcc 4.4.7 with -Werror fails without this change:
`memcached.c: In function ‘process_bin_get_or_touch’:
memcached.c:1286: error: ignoring return value of ‘fwrite’, declared
with attribute warn_unused_result`
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
| |
re GH#128
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
|
|
|
|
| |
Apparently I never tested this back in 2014. The $(.TARGET:R) stuff is being
evaluated into nothing on my system and I have no idea where that extension
came from... So just type out the file names explicitly.
|
| |
|
|
| |
re: GH#142.
|
| |
|
|
|
|
|
|
|
|
| |
musl libc will warn if you include sys/signal.h instead of signal.h as
specified by posix. Build will fail due to -Werror explicitly beeing
set.
Fix it by use the posix location.
fixes #138
|
| |
|
|
|
| |
- avoid initializing the gnu stack (foreign)
- make memcached easier to cross-compile
|
| |
|
|
|
|
|
|
|
| |
on 32bit hardware with different pointer/slab class sizes, the tests would
fail. made a few adjustments to ensure reassign rescues happen and make items
not be near the default slab class borders.
This makes the tests pass, but needs further improvements for reliability..
ie: "fill until evicts", count slab pages for reassignment/etc.
|
| |
|
|
|
|
|
|
|
|
|
| |
single CPU VM builders could fail:
- spawn LRU crawler thread.
- signal LRU crawler.
- LRU crawler thread now waits on condition.
- Crawler thread misses condition, sits forever.
Might also want to move the "stats.lru_crawler_running" bit to be updated when
the crawler thread picks up the work to do, somehow.
|
| |
|
|
|
|
|
|
|
|
|
| |
mem_alloced was getting increased every time a page was assigned out of either
malloc or the global page pool. This means total_malloced will inflate forever
as pages are reused, and once limit_maxbytes is surpassed it will stop
attempting to malloc more memory.
The result is we would stop malloc'ing new memory too early if page reclaim
happens before the whole thing fills. The test already caused this condition,
so adding the extra checks was trivial.
|
| |
|
|
|
|
| |
previously the slab mover would evict items if the new chunk was within the
slab page being moved. now it will do an inline reclaim of the chunk and try
until it runs out of memory.
|
| |
|
|
|
|
| |
gross oversight putting two conditions into the same variable. now can tell if
we're evicting because we're hitting the bottom of the free memory pool, or if
we keep trying to rescue items into the same page as the one being cleared.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
class 255 is now a legitimate class, used by the NOEXP LRU when the
expirezero_does_not_evict flag is enabled. Instead, we now force a single bit
ITEM_SLABBED when a chunk is returned to the slabber, and
ITEM_SLABBED|ITEM_FETCHED means it's been cleared for a page move.
item_alloc overwrites the chunk's flags on set. The only weirdness was
slab_free |='ing in the ITEM_SLABBED bit. I tracked that down to a commit in
2003 titled "more debugging" and can't come up with a good enough excuse for
preserving an item's flags when it's been returned to the free memory pool. So
now we overload the flag meaning.
|
| |
|
|
|
| |
if we're deciding to move pages right on the chunk boundary it's too easy to
cause flapping.
|
| |
|
|
|
| |
uses the slab_rebal struct to summarize stats, more occasionally grabbing the
global lock to fill them in, instead.
|
| |
|
|
|
|
| |
During an item rescue, item size was being added to the slab class when the
new chunk requested, and then not removed again from the total if the item was
successfully rescued. Now just always remove from the total.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If item does not have ITEM_SLABBED bit, or ITEM_LINKED bit, logic was falling
through, defaulting to MOVE_PASS. If an item has had storage allocated via
item_alloc(), but haven't completed the data upload, it will sit in this mode.
With MOVE_PASS for an item in this state, if no other items trip the busy
re-scan of the page the mover will consider the page completely wiped even
with the outstanding item.
The hilarious bit is I'd clearly thought this through: the top comment states
the if this, then this, or that... with the "or that" logic completely
missing. Add one line of code and it survived a 5 hour torture test, where
before it crashed after 30-60 minutes.
Leaves some handy debug code #ifdef'ed out. Also moves the memset wipe on page
move completion to only happen if the page isn't being returned to the global
page pool, as the page allocator does a memset and chunk-split.
Thanks to Scott Mansfield for the initial information eventually leading to
this discovery.
|
| |
|
|
| |
Thanks Devon :)
|
| |
|
|
|
| |
some new variables and change to the '1' mode. little sad nobody noticed I'd
accidentally removed the '2' mode for a few versions.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If any slab classes have more than two pages worth of free chunks, attempt to
free one page back to a global pool.
Create new concept of a slab page move destination of "0", which is a global
page pool. Pages can be re-assigned out of that pool during allocation.
Combined with item rescuing from the previous patch, we can safely shuffle
pages back to the reassignment pool as chunks free up naturally. This should
be a safe default going forward. Users should be able to decide to free or
move pages based on eviction pressure as well. This is coming up in another
commit.
This also fixes a calculation of the NOEXP LRU size, and completely removes
the old slab automover thread. Slab automove decisions will now be part of the
lru maintainer thread.
|
| |
|
|
|
|
|
| |
used to take the newest page of the page list and replace the oldest page with
it. so only the first page we move from a slab class will actually be "old".
instead, actually burn the slight CPU to shuffle all of the pointers down one.
Now we always chew the oldest page.
|
| |
|
|
|
|
|
|
|
|
|
| |
During a slab page move items are typically ejected regardless of their
validity. Now, if an item is valid and free chunks are available in the same
slab class, copy the item over and replace it.
It's up to external systems to try to ensure free chunks are available before
moving a slab page. If there is no memory it will simply evict them as normal.
Also adds counters so we can finally tell how often these cases happen.
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Test is a port of a golang test submitted by Scott Mansfield.
There used to be an "angry birds mode" to slabs_automove, which attempts to
force a slab move from "any" slab into the one which just had an eviction.
This is an imperfect but fast way of responding to shifts in memory
requirements.
This change adds it back in plus a test which very quickly attempts to set
data in via noreply. This isn't the end of improvements here. This commit is a
starting point.
|
| |
|
|
|
| |
add a handful of missing stats docs (doh) and shorted the STATS_LOCK a little
bit. the rest seems fine.
|
| | |
|
| |
|
|
|
| |
Update documentation to more accurately reflect the value
that should be passed along as the parameter via command line.
|
| |
|
|
|
|
| |
'buf' could be freed after calling atoi, providing a gentler way to exit
Signed-off-by: Yongyue Sun <abioy.sun@gmail.com>
|
| | |
|
| | |
|
| | |
|
| | |
|
