WT-2548 Cap the amount of data handed to raw compression.

Previously, we had special case code for zlib to bound the compression
ratio to an arbitrary factor.  This is driven by synthetic workloads
where data can be compressed on disk to 1% or less of the in-memory
size.  If WiredTiger allows such on-disk pages to grow, they can end up
larger than the maximum page size permitted in memory and either
immediately force eviction or in extreme cases grow larger than the
total cache size.

This change bounds the amount of data handed to raw compression based on
the maximum page size in memory before attempting in-memory splits.

2 files changed, 8 insertions, 9 deletions

diff --git a/ext/compressors/zlib/zlib_compress.c b/ext/compressors/zlib/zlib_compress.c
index 4ff0d8576eb..9aede2ed907 100644
--- a/ext/compressors/zlib/zlib_compress.c
+++ b/ext/compressors/zlib/zlib_compress.c
@@ -307,17 +307,9 @@ zlib_compress_raw(WT_COMPRESSOR *compressor, WT_SESSION *session,
 		/*
 		 * If there's more compression to do, save a snapshot and keep
 		 * going, otherwise, use the current compression.
-		 *
-		 * Don't let the compression ratio become insanely good (which
-		 * can happen with synthetic workloads).  Once we hit a limit,
-		 * stop so the in-memory size of pages isn't hugely larger than
-		 * the on-disk size, otherwise we can get into trouble where
-		 * every update to a page results in forced eviction based on
-		 * the in-memory size, even though the data fits into a single
-		 * on-disk block.
 		 */
 		last_slot = curr_slot;
-		if (zs.avail_out > 0 && zs.total_in <= zs.total_out * 20) {
+		if (zs.avail_out > 0) {
 			if ((ret = deflateCopy(&last_zs, &zs)) != Z_OK)
 				return (zlib_error(
 				    compressor, session, "deflateCopy", ret));
diff --git a/src/reconcile/rec_write.c b/src/reconcile/rec_write.c
index 873482df131..c42b310ef32 100644
--- a/src/reconcile/rec_write.c
+++ b/src/reconcile/rec_write.c
@@ -2575,6 +2575,13 @@ __rec_split_raw_worker(WT_SESSION_IMPL *session,
 		    dsk->type == WT_PAGE_COL_VAR)
 			r->raw_recnos[slots] = recno;
 		r->raw_entries[slots] = entry;
+
+		/*
+		 * Don't create an image so large that any future update will
+		 * cause a split in memory.
+		 */
+		if (len > (size_t)btree->splitmempage)
+			break;
 	}
 
 	/*