Add tuning advise for os_cache_dirty_max, os_cache_max.

3 files changed, 82 insertions, 5 deletions

diff --git a/examples/c/ex_all.c b/examples/c/ex_all.c
index 0168552d8b9..e2486ffe454 100644
--- a/examples/c/ex_all.c
+++ b/examples/c/ex_all.c
@@ -1037,6 +1037,18 @@ main(void)
 	(void)conn->close(conn, NULL);
 #endif
 
+	/*! [os_cache_dirty_max configuration] */
+	ret =
+	    wiredtiger_open(home, NULL, "create,os_cache_dirty_max=1GB", &conn);
+	/*! [os_cache_dirty_max configuration] */
+	(void)conn->close(conn, NULL);
+
+	/*! [os_cache_max configuration] */
+	ret =
+	    wiredtiger_open(home, NULL, "create,os_cache_max=1GB", &conn);
+	/*! [os_cache_max configuration] */
+	(void)conn->close(conn, NULL);
+
 	/*! [Statistics configuration] */
 	ret = wiredtiger_open(home, NULL, "create,statistics=true", &conn);
 	/*! [Statistics configuration] */
diff --git a/src/docs/spell.ok b/src/docs/spell.ok
index 814b38d7c1f..84f1c4e53f0 100644
--- a/src/docs/spell.ok
+++ b/src/docs/spell.ok
@@ -117,6 +117,7 @@ env
 eof
 erlang
 errno
+fadvise
 failchk
 fd's
 fieldname
@@ -225,6 +226,7 @@ objectsin
 ol
 oltp
 oob
+os
 ovfl
 pcoll
 pdf
diff --git a/src/docs/tuning.dox b/src/docs/tuning.dox
index 5c0cfa118d7..9f2ba1801bb 100644
--- a/src/docs/tuning.dox
+++ b/src/docs/tuning.dox
@@ -159,12 +159,18 @@ An example of configuring page sizes:
 
 @snippet ex_file.c file create
 
-@section tuning_direct_io Direct I/O
+@section tuning_system_buffer_cache System buffer cache
 
-WiredTiger optionally supports direct I/O, based on the non-standard \c
-O_DIRECT flag to the POSIX 1003.1 open system call.   Configuring direct
-I/O may be useful for applications wanting to minimize the operating
-system cache effects of I/O to and from WiredTiger's buffer cache.
+@subsection tuning_system_buffer_cache_direct_io Direct I/O
+
+WiredTiger optionally supports direct I/O.  Configuring direct I/O may
+be useful for applications wanting to:
+- minimize the operating system cache effects of I/O to and from
+WiredTiger's buffer cache,
+- avoid double-buffering of blocks in WiredTiger's cache and the
+operating system buffer cache, and
+- avoid stalling underlying solid-state drives by writing a large number
+of dirty blocks.
 
 Direct I/O is configured using the "direct_io" configuration string to
 the ::wiredtiger_open function.  An example of configuring direct I/O
@@ -172,6 +178,63 @@ for WiredTiger's data files:
 
 @snippet ex_all.c Configure direct_io for data files
 
+Direct I/O implies a writing thread waits for the write to complete
+(which is a slower operation than writing into the system buffer cache),
+and configuring direct I/O is likely to decrease overall application
+performance.
+
+Direct I/O is based on the non-standard \c O_DIRECT flag to the POSIX
+1003.1 open system call and may not available on all platforms.
+
+@subsection tuning_system_buffer_cache_os_cache_dirty_max os_cache_dirty_max
+
+As well as direct I/O, WiredTiger supports two additional configuration
+options related to the system buffer cache:
+
+The first is \c os_cache_dirty_max, the maximum dirty bytes an object
+is allowed to have in the system buffer cache.  Once this many bytes
+from an object are written into the system buffer cache, WiredTiger will
+attempt to schedule writes for all of the dirty blocks the object has
+in the system buffer cache.  This configuration option allows
+applications to flush dirty blocks from the object, avoiding stalling
+any underlying drives when the object is subsequently flushed to disk
+as part of a durability operation.
+
+An example of configuring \c os_cache_dirty_max:
+
+@snippet ex_all.c os_cache_dirty_max configuration
+
+The \c os_cache_dirty_max configuration may not be used in combination
+with direct I/O.
+
+The \c os_cache_dirty_max configuration is based on the non-standard
+Linux \c sync_file_range system call and may not available on all
+platforms.
+
+@subsection tuning_system_buffer_cache_os_cache_max os_cache_max
+
+The second configuration option related to the system buffer cache is
+\c os_cache_max, the maximum bytes an object is allowed to have in the
+system buffer cache.  Once this many bytes from an object are either
+read into or written from the system buffer cache, WiredTiger will
+attempt to evict all of the object's blocks from the buffer cache.  This
+configuration option allows applications to evict blocks from the system
+buffer cache to limit double-buffering and system buffer cache overhead.
+
+An example of configuring \c os_cache_max:
+
+@snippet ex_all.c os_cache_max configuration
+
+The \c os_cache_max configuration may not be used in combination with
+direct I/O.
+
+The \c os_cache_max configuration is based on the POSIX 1003.1 standard
+\c posix_fadvise system call and may not available on all platforms.
+
+Configuring direct I/O, \c os_cache_dirty_max or \c os_cache_max all
+have the side effect of turning off memory-mapping of objects in
+WiredTiger.
+
 @section tuning_checksums Checksums
 
 WiredTiger checksums file reads and writes, by default.  In read-only