| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
| |
|
|
|
|
|
|
| |
Extention to the existing slab reallocation model to better cope with eviction patterns that appear in short timeframe bursts.
This means that if more than 25% of the total evictions in the window appear in this slab it becomes a candidate for
needing more pages.
|
| |
|
| |
|
|
|
|
|
| |
regex call unescaping key reassigns $2
-- store/reference it as $exp
|
|
|
|
| |
`lru_crawler metadump` uses `uriencode`.
|
|
|
| |
pool_high was added in https://github.com/memcached/memcached/commit/e4caa1917b17f599603c29d4af58a68e73a1f428 , and the use of it was removed in https://github.com/memcached/memcached/commit/aa552fed9d9cffd10197bb3b2a62ad82ece62d5b#diff-b43c9d46afe4cc3ee89f2f819428a082 . As reported on https://lgtm.com/projects/g/memcached/memcached/alerts/?mode=list
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
| |
* automover
* avoiding
* compress
* fails
* successfully
* success
* tidiness
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The major things this does are adding systemd support to the rpm .spec
file, and adding systemd instancing support. This means that it is
possible to run multiple memcached instances without having to do
any additional configuration or hack on init scripts.
To use:
systemctl start memcached@11211 memcached@11311 memcached@11411
sysconfig files at /etc/sysconfig/memcached.<port> will be read as
appropriate, to allow differing configurations per-port. Defaults
will be read from /etc/sysconfig/memcached before the port-specific
settings are read.
You can also still start memcached the standard way just by doing
"systemctl start memcached". This will read /etc/sysconfig/memcached
and nothing else.
The "enhanced security" lines in the systemd unit file will be commented
out on systems where we know systemd isn't knew enough (fedora < 26 and
Redhat/CentOS 7), and enabled on other systems.
There are two versions of the .service file included, one for standard
memcached invocations and one for instanced invocations. The two are
very similar, but not identical. Ideally, we'd only have one version
in the source tree and we'd massage it with sed or somesuch during the
rpm build, but couldn't think of a super clean way to do that, so erred
on the side of simplicity.
A decent amount of spec file work was needed to enable this functionality.
In the process, I also cleaned up several additional aspects of the spec
file (like using %{name} in places where it was appropriate). I also
commented out the automatic restart in the %postun section, for two main
reasons:
1. The try-restart for instanced memcached will produce an error if
instanced memcached isn't in use, which is probably quite confusing to
people who aren't using that functionality and are just trying to update
their package. (There's workarounds for this, but I try to keep pre/post
scripts as simple as humanly possible)
2. Automatic restarts on updates means the cache gets flushed, which
means you can no longer safely use large-scale management tools (like
puppet or chef) to roll out new versions, at least not without a lot
of planning first. Not automatically dumping someone's caches feels
safer, here.
|
|
|
|
|
|
|
| |
not as easy to do the "global pool filled once" check, since the underlying
memcached can be restarted but the script will just reconnect. simplest thing
to do is to keep the evictions check, which stops the algorithm on a sudden
reset.
|
|
|
|
|
|
|
| |
allows reassigning memory from global page pool to a specific class.
this allows simplifying the algorithm to rely on moving memory to/from
global, removing hacks around relaxing free memory requirements.
|
|
|
|
|
|
|
|
|
|
| |
wasn't checking for "dirty" flag on a class before deciding it was too free,
so it would assign/remove/assign/remove/assign/remove, and tend to
over-remove slightly. Over time the global page pool would simply grow.
Also adjusts timing so draining the global page pool works better. multiple
classes can be drained into at once by nature of zeroing out the free limits
one at a time.
|
|
|
|
|
|
|
|
|
|
|
| |
wasn't aggressively enough reclaiming excess free memory if it was wobbling
around a specific number.
should drop it with some stop and go but will fix and should have a window
sized delay to absorb wobbles in free memory due to crawlers freeing things.
might be able to further tweak by preventing the reclaim if it's
*consistently* dropping for the whole window. a task for the future :P
|
|
|
|
|
| |
need to hold at least 1.5 pages free so balancing won't early evict in most
situations.
|
|
|
|
|
|
|
|
| |
using the new per-class free chunk limiter and ignoring item age, we can now
create a highly responsive page mover algorithm. It makes much fewer page
moves than before and keeps more of the free memory target in each class.
adjusts the per-class limits once per minute, and otherwise balancers by age.
|
|
|
|
| |
python types still escape me.
|
|
|
|
|
|
|
| |
if we want to disable automove and run an external algo that examines the
page pool count...
removes temp hack from external script.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
first pass at an extstore-friendly automove algorithm. With a static item_age
value, extra RAM may be unused during all or parts of the day, and the
existing balancer may not give enough memory to small slab classes.
This does a lot of page moving and can be further improved.
- leave some of the "buffer" as actual free space in each slab class. only
move pages back to global if one gets way too much memory.
- use a PID controller to slew adjustments.
but it does work, so I'll probably convert this to C and revise the algo
later.
|
|
|
|
|
| |
free_chunks_end stopped working years ago when the page mover was being
optimzed. check now looks for used_chunks == total_chunks.
|
|
|
|
| |
converts the python script to C, more or less.
|
|
|
|
|
|
|
|
|
|
|
|
| |
added a devtools/slab_loadgen script for some basic testing of load patterns.
can easily add more features (typical variance/reads/inline reload/etc) to
further tune algo but was helpful enough as-is. Was able to restart the
program with changed configs and watch rebalancer fix itself.
rebalancer algo is now much better at pulling slab classes closer together and
not ping-ponging, which previous version was doing even with a steady state of
sets.
|
| |
|
|
|
|
| |
reported by hobbs. my python is so bad :(
|
|
|
|
| |
external algorithm testing thing.
|
|
|
|
| |
whoops, forgot to run the test when cherry-picking :/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
one can check status of sole memcached daemon if there’re no instances configured (this is default behaviour on request made in common way w/o xtra args) or check status of some instance from bunch
an outcome is short diagnostic message along w/ appropriate ret code
— status of sole memcached daemon
/etc/init.d/memcached status
/sbin/service memcached status
— status of memcached instance ’instance001’
/etc/init.d/memcached status instance001
/sbin/service memcached status instance001
— status of memcached instance ’instance014’
/etc/init.d/memcached status instance014
/sbin/service memcached status instance014
|
|
|
| |
Disable use of namespaces, control groups, addresses families that aren't used, kernel module loading, and setting of kernel tunables.
|
|
|
|
|
|
| |
the parent process is the only one using that capability, once privileges are
dropped it's gone anyway. This prevents raising the connection limit from
breaking.
|
|
|
| |
MemoryDenyWriteExecute=true will prevent attempts to create memory mappings that are both writable and executable at the same time. This option improves service security, as it makes harder for software exploits to change running code dynamically.
|
|
|
|
| |
From http://code.google.com/p/memcached/issues/detail?id=283
|
| |
|
| |
|
|
|
| |
memcached should be isolated and restricted as much as possible to improve security so that if memcached is compromised, the damage will be limited.
|
| |
|
| |
|
|
|
|
| |
This is (these are) automatically started and stopped by the main job.
|
| |
|
|
|
|
| |
This is necessary for an Upstart job, because the call out to rm -f causes Perl to fork off a process, and Upstart think that new process is the main one (not the exec'd memcached process later on).
|
| |
|
|
|
|
|
| |
Add human parseable strings to the errors for slabs ressign. Also prevent
reassigning memory to the same source and destination.
|
|
|
|
|
|
|
|
|
|
| |
Enable at startup with -o slab_reassign,slab_automove
Enable or disable at runtime with "slabs automove 1\r\n"
Has many weaknesses. Only pulls from slabs which have had zero recent
evictions. Is slow, not tunable, etc. Use the scripts/mc_slab_mover example to
write your own external automover if this doesn't satisfy.
|
| |
|
|
|
|
|
| |
after the doublefork was added, the pidfile was now of the intermediary pid,
not the one forked after setsid.
|
|
|
|
| |
previously hardcoded to 40. now will iterate up through all of them.
|
|
|
|
|
| |
The debianish start script routes the STDERR/STDOUT to make "logfiles", but in
doing so doesn't break free of the launching session. Patch fixes that.
|
|
|
|
|
|
|
|
| |
Patch by Gordon Franke. Whitespace fixes and comments added by Dormando.
There are many better ways to do this, but it doesn't break the default and
the "status" command never existed. Servers can be started and stopped
individually, and people seem to like the idea.
|
| |
|