diff options
| author | Joan Touzet <joant@atypical.net> | 2017-08-20 17:17:52 -0400 |
|---|---|---|
| committer | Joan Touzet <joant@atypical.net> | 2017-08-20 17:17:52 -0400 |
| commit | d2f739dcf6307d839d010791a59d6e035448a896 (patch) | |
| tree | 514728ac31edaa0f7c289eafcb3cdfdfe3984cdc | |
| parent | d46e3ee992a7f9ab76cdb3dd23f6abf2e91c079d (diff) | |
| download | couchdb-reduce_default_max_dbs_open.tar.gz | |
Reduce default max_dbs_open to 100reduce_default_max_dbs_open
With `q=8` being the default shard setting, and the Linux default of
1024 file handles per process out of the box on virtually every distro,
the maximum number of databases an unmodified user environment could
conceivably open simultaneously is 128. (Assuming one view/index per
database open as well, that number drops to 64.)
When we hit these limits (enfile / emfile Erlang errors) CouchDB starts
acting very erratically, leading to lots of scary errors and in some
cases inconsistent databases. We have a recent report of mem3 dying as a
result, which took the ets tables with it - and this occurred with just
32 databases, with multiple views . In another case a user reported
pereceived database corruption (which was actually a combination of
error reports + app-server layer errors).
While we should recommend ulimit/fs.file-max be bumped in production
use, and consequently bumping max_dbs_open as well, we should defend
against these kinds of errors for users who compile and install from
source (our official distribution option). max_dbs_open = 100 makes this
kind of error less likely.
This also matches our couchdb-documentation repository, which still
reflects a setting of 100 and was never updated to match our increased
500 limit.
The alternate approach is for us to better handle enfile / emfile errors
and react accordingly, but there are are a significant number of touch
points in the code where this can cause issues.
| -rw-r--r-- | rel/overlay/etc/default.ini | 2 | ||||
| -rw-r--r-- | src/couch/src/couch_server.erl | 2 |
2 files changed, 2 insertions, 2 deletions
diff --git a/rel/overlay/etc/default.ini b/rel/overlay/etc/default.ini index 30b2efa5c..d7295455e 100644 --- a/rel/overlay/etc/default.ini +++ b/rel/overlay/etc/default.ini @@ -9,7 +9,7 @@ view_index_dir = {{view_index_dir}} ; util_driver_dir = ; plugin_dir = os_process_timeout = 5000 ; 5 seconds. for view and external servers. -max_dbs_open = 500 +max_dbs_open = 100 delayed_commits = false ; Method used to compress everything that is appended to database and view index files, except ; for attachments (see the attachments section). Available methods are: diff --git a/src/couch/src/couch_server.erl b/src/couch/src/couch_server.erl index 24016e05c..2a10b021b 100644 --- a/src/couch/src/couch_server.erl +++ b/src/couch/src/couch_server.erl @@ -28,7 +28,7 @@ -include_lib("couch/include/couch_db.hrl"). --define(MAX_DBS_OPEN, 500). +-define(MAX_DBS_OPEN, 100). -define(RELISTEN_DELAY, 5000). -record(server,{ |