3.0.3, Apr 25 2015 ------------------ commit 04f9af1c4fb9a9287786948a818c88eef68cb8a4 Author: Keith Bostic Date: Thu Apr 23 17:52:42 2015 -0400 If a file is marked cache-resident, it can never be evicted, this fixes SERVER-18192. commit 96f85a0987217074dd91de689a499cd0a4c16bbb Author: Keith Bostic Date: Thu Apr 23 15:08:06 2015 -0400 Always clear WT_BTREE_NO_HAZARD when cache_resident is not configured, otherwise we can (1) create a file with cache_resident configured (setting both no-eviction and no-hazard), (2) drop the file, then (3) re-create the file without cache_resident configured (clearing only no-eviction), and the result will be a file that can be evicted and where we aren't maintaining hazard pointers, and nothing good is going to happen after that. commit 9166bca3d07d6592c1426c2c33bd56b6be0667e0 Author: Alex Gorrod Date: Thu Apr 23 05:43:35 2015 +0000 Fix a deadlock related to handle locking. If one thread does a __wt_session_release_btree of a handle at the same time as another thread does a __wt_session_get_btree both wanting exclusive access to the file. It was possible for one thread to get the dhandle list lock and wait on the handle lock, which another thread held the handle lock waiting for the handle list lock. Temporarily fix by doing a try-lock on the __wt_session_get_btree path, long term the solution is to get rid of the __conn_dhandle_open_lock method, and split get and lock into two passes. Refs BF-716 commit 3e254079484ce35a3cb70c48478c69defdb8f012 Author: Alex Gorrod Date: Thu Apr 23 05:42:08 2015 +0000 Fix a deadlock related to LSM. There are cases where closing a file with an existing checkpoint could self deadlock. Check in the meta tracking whether we've already visited a checkpoint handle. Refs WT-716 commit 610f629949726b16f938ded85188bb6a21820f7e Author: Keith Bostic Date: Mon Apr 20 10:40:54 2015 -0400 Create a "clear a single walk" function and call it from __evict_clear_walks and __evict_clear_all_walks, that way we can use the WT_WITH_DHANDLE macro, and the comment about clearing the eviction reference before releasing the page appears everywhere it should. commit 3eceb85ce623dcce9273f7b722bb64f509dbe24d Author: Michael Cahill Date: Mon Apr 20 16:15:41 2015 +1000 Clear session->dhandle so that future error messages don't dereference freed memory. commit 23ce8bae4d6507d6b6d599cb73a26a6c856cce98 Author: Michael Cahill Date: Mon Apr 20 15:48:29 2015 +1000 Clear eviction walks in all trees before the eviction server exits. commit a4bce0e0bc05d528f118b645d0d1915db00cdcf3 Author: Michael Cahill Date: Mon Apr 20 14:11:57 2015 +1000 Move the "cache is empty" check to where the cache is destoyed, not when the eviction server is exiting. commit c8fdd9c676c4a24bee6328a56cf7fd074cd045e0 Author: Michael Cahill Date: Mon Apr 20 13:25:36 2015 +1000 Shut down the eviction server before closing file handles to avoid a race. refs WT-1893 commit abb0bb80cc6dce29b8db61c6747c228c2701ae5a Author: Susan LoVerso Date: Fri Apr 17 10:49:41 2015 -0400 Look for any number of non-data-changing log records to determine if we can skip recovery. WT-1892 commit ee02428d1fdf1118c482688ec870a9da69bee45a Author: Michael Cahill Date: Fri Apr 17 12:28:52 2015 +1000 If an LSM search-near operation lands on a deleted item, make a copy of the key before stepping to the next record. refs WT-1891 commit 54e856d57da291c5f84da6d0d0ab56280d9956dc Author: Susan LoVerso Date: Thu Apr 16 12:22:31 2015 -0400 Remove use of unneeded tmp_fh. WT-1872 commit 6a32905c397e57643b15e5a3038dbcb99a8a8dc8 Author: Alex Gorrod Date: Thu Apr 16 06:18:31 2015 +0000 Fix a deadlock in LSM with schema operations. There is special code in LSM to co-ordinate schema operations on tables (drop, rename, etc). The code does a dance dropping and acquiring locks, to allow utility operations to drain for the tree while waiting for it to close. We were doing the dance with the schema and dhandle list locks. We needed to include the table lock, or parallel cursor opens could block: The cursor open is waiting for the table lock: __wt_spin_lock src/include/mutex.i:175 __schema_add_table src/schema/schema_list.c:26 __wt_schema_get_table src/schema/schema_list.c:98 __wt_curtable_open src/third_party/wiredtiger/src/cursor/cur_table.c:875 __wt_open_cursor src/session/session_api.c:240 The LSM table drop is waiting for the schema lock: __wt_spin_lock src/include/mutex.i:175 __lsm_tree_close src/lsm/lsm_tree.c:107 __wt_lsm_tree_drop src/lsm/lsm_tree.c:943 __wt_schema_drop src/schema/schema_drop.c:174 __drop_table src/schema/schema_drop.c:124 __wt_schema_drop src/schema/schema_drop.c:176 __session_drop src/session/session_api.c:528 commit 790646183cc5dd056bbf95c4563c20c51602a808 Author: Alex Gorrod Date: Thu Apr 16 04:11:36 2015 +0000 Fix a bug in LSM where updates with overwrite could be skipped. References JIRA BF-829 The issue was that we were not looking in all chunks of an LSM tree before deciding whether to apply an update (insert or remove). commit b9e2e76511a24505014369aaf0e1ec286e9c473d Author: Michael Cahill Date: Tue Apr 14 16:30:31 2015 +1000 Merge bulk cursor close with regular file cursor close: we were missing a decrement that kept bulk-loaded files pinned. commit ed3158e71f0bd2716269a5771fd162b60b9a1cc0 Author: daveh86 Date: Mon Apr 13 12:59:29 2015 +1000 Allow forced eviction of pages already queued for eviction commit 9c83351f63afc2e032e492e3030df4f3b1cd6883 Author: Mark Benvenuto Date: Sun Apr 12 19:02:32 2015 -0400 Disable fallocate on Windows since SetEndofFile does not ignore truncation requests like POSIX fallocate. commit 61a7d81ad26db1f2bfb65258d9b8ae4a4ca25b34 Author: Keith Bostic Date: Sun Apr 12 12:44:05 2015 -0400 When using ftruncate as the file-extension call, we must use WT_FH.size as the starting point of the extension (not offset), and we have to read the size value after acquiring the lock that prevents racing with writers. Split the extension functionality out into a separate function and try to make it a little simpler to understand. Reference #1871. commit f26f1c1c59d5cbbc8f5f543215d8fc636e7175d2 Author: Keith Bostic Date: Sun Apr 12 10:16:34 2015 -0400 The OS layer ftruncate() code sets the WT_FH file size, so when ftruncate is used to extend the file, we skip over the bytes added to the file during block allocation, and verify eventually fails because there are unverified blocks in the file. Reference #1871. commit d20f20f1ac324030986b3ee23e1edf96486c92b4 Author: Alex Gorrod Date: Fri Apr 10 05:59:04 2015 +0000 Update file_manager=(close_idle_time=,close_scan_interval=) max values. The old maximum for both was 1000, the new value is 100 thousand. Setting such large values is not recommended, but there is no internal limitation on them. commit c36a3308f685d3b85efe9ac6ee0835f0974574b4 Author: Keith Bostic Date: Thu Apr 9 14:32:40 2015 -0400 Don't ignore sweeping entirely when we reach the open-file-count limit, just ignore the in-use files. commit 46ef2555bbc51ce6453536e72202782be4949855 Author: Keith Bostic Date: Thu Apr 9 11:06:50 2015 -0400 Don't keep sweeping once we reach the minimum number of handles. commit 1fdfcc62726d25a7cceeeefff174a8e1bf9f9e67 Author: Keith Bostic Date: Thu Apr 9 12:39:03 2015 -0400 const: At condition ret == -1, the value of ret must be equal to -1. CID 72082 (#1 of 1): Redundant test (DEADCODE) dead_error_condition: The condition ret == -1 must be true. commit d04e3c25d46a5c4426e1c6d4881cd9e250014931 Author: Keith Bostic Date: Thu Apr 9 12:19:24 2015 -0400 Remove unnecessary atomic operation, fixing CID 69810 along the way. CID 69810 (#1 of 1): Parse warning (PW.CONVERSION_TO_POINTER_ADDS_BITS) 1. conversion_to_pointer_adds_bits: conversion from smaller integer to pointer commit d585bdab980508e590cf70508f053182c556d6f2 Author: Keith Bostic Date: Thu Apr 9 12:00:04 2015 -0400 CID 72073 (#1 of 1): Redundant test (DEADCODE) dead_error_condition: The condition session != NULL must be true. commit 21907f9193e30f51a59fcbaddfbc46cb7732d3b7 Author: Keith Bostic Date: Thu Apr 9 11:52:13 2015 -0400 Remove unnecessary error labels, fixing SERVER-17948 along the way. Coverity analysis defect 72088: Redundant test: File: /src/third_party/wiredtiger/src/log/log.c Function __log_decompress /src/third_party/wiredtiger/src/log/log.c, line: 363 At condition "ret != 0", the value of "ret" must be equal to 0. commit a29f4d2f40eee784950147af848fdbf277328b7f Author: Keith Bostic Date: Thu Apr 9 11:31:11 2015 -0400 SERVER-17954: Coverity analysis defect 72085: Redundant test At condition "ret != 0", the value of "ret" cannot be equal to 0. commit 1298fd6db4f0c1d6133b3e547b2e2db51ec4a708 Author: Keith Bostic Date: Wed Apr 8 14:38:49 2015 -0400 Fix places where we were using the wrong link for traversing hash buckets. commit 7d0e5fe3dfa39f7ff377a1d4660bc2bc36dc0df8 Author: Mark Benvenuto Date: Wed Apr 8 10:19:33 2015 -0400 Enable test/fops for Windows, and add to CI - Added Windows shim for gettimeofday commit c6270b677499525067d5d729a6fbdce6ad2f533a Author: Mark Benvenuto Date: Wed Apr 8 13:28:41 2015 -0400 test/format for Windows - Fixed an issue where fallocate was setting fh->size (incorrectly copied from ftruncate implementation) commit fac74b4665d6dfa3aebecf741c914fd1678fa628 Author: Keith Bostic Date: Wed Apr 8 08:47:06 2015 -0400 Rename file_manager.open_handles to file_manager.close_handle_minimum, try and match existing naming for file_manager configuration options. commit 709cc8d7ac85d31aeae2387b192092910e6cf854 Author: Keith Bostic Date: Wed Apr 8 08:09:23 2015 -0400 Don't sweep unless there's a reason we need to close file handles. Add a new configuration option, file_manager.open_handles that sets a minimum number of file handles that must be open before sweep runs, default is 250. Reference #1856, SERVER-17907. commit b79dcdebf0b1987b59fa70e50c8c61e5e0a64e64 Author: Keith Bostic Date: Tue Apr 7 11:52:12 2015 -0400 If the underlying split buffer grows, existing boundary references may point into freed memory, switch the boundary "first byte" field from a pointer to a buffer offset to avoid the problem, reference #1852. commit 684fd71475cbc6b15290945af0160fac0313ad6b Author: Susan LoVerso Date: Wed Apr 1 15:44:42 2015 -0400 Reset eol if we continue. Return NOTFOUND if checksum mismatch. #1840 commit 9c29e0f13268c03038704372c069353c81357791 Author: Susan LoVerso Date: Tue Mar 31 16:49:47 2015 -0400 Modify log_scan callback args to send in next LSN. #1837 commit 003e6c3598408c7670f65a8720622c38fdaf148d Author: Mark Benvenuto Date: Thu Mar 26 11:17:57 2015 -0400 Use Standard C type uint64_t for zlib commit e12b9e0b005da7364330f4d3409256ded26ba90d Author: Keith Bostic Date: Thu Mar 26 18:31:15 2015 -0400 Add checks for Windows builds: _M_AMD64 and _MSC_VER. commit a609c82bfa6fbf80cd5fb853e1d97c16878a0180 Author: Susan LoVerso Date: Mon Mar 23 15:25:23 2015 -0400 Set checkpoint LSN to existing log record. #1700 3.0.2, Mar 26 2015 ------------------ commit 17bd2356a5b17893e626749be399fb8fda23db1e Author: Mark Benvenuto Date: Fri Mar 20 01:07:59 2015 -0400 Use beginthreadex, and ensure we use the correct C calling conventions in all callbacks commit 269e847ad64dd12dfcadb58f84f905069e5b8dce Author: Michael Cahill Date: Wed Mar 25 15:47:46 2015 +1100 Split sweep into two passes: one that walks the handle list without locking and finds candidate handles to close, and a second pass holding the handle list lock that removes dead handles. refs #1814, #1811, #1808 commit 065a435f6b1d1b3fcb640d59c3109d0c2e24d308 Author: Alex Gorrod Date: Tue Mar 24 16:53:28 2015 +1100 Add statistics to track internal/overflow/leaf pages in cache. It is often useful to know if a cache is full of internal pages. I could have added page count tracking, but byte count tracking seems more useful (enough to justify the extra performance overhead). commit f4616895f1ac35aaa50fc1a68e0c6dd9e0cf7717 Author: Michael Cahill Date: Mon Mar 23 17:00:15 2015 +1100 If a walk for pages to evict ends on a page requiring forced eviction, don't keep it pinned. refs SERVER-16662, SERVER-17382 commit 49ddcca712db3a50c390f48b518f0835f28bc9d4 Author: Keith Bostic Date: Sat Mar 21 16:17:51 2015 -0400 Separate out the code to flush a file handle and rename the file into place. commit a0e88c41781c5b7ea0155fb57a58c91a964af4a3 Author: Keith Bostic Date: Fri Mar 20 19:08:45 2015 -0400 If the underlying FILE was opened for writing, flush and fsync it during close. commit 8d918f0ef8588056ecf729e72ffdd8bc0a79fd6c Author: Michael Cahill Date: Fri Mar 20 22:54:13 2015 +1100 Fix a race closing eviction helper threads: close the main eviction thread first, so the number of helper threads can be read safely. Partial fix for #1698 commit 294b0bce296bebf9790418e5575f59be5bec000c Author: Michael Cahill Date: Fri Mar 20 17:19:28 2015 +1100 Only align buffers on Linux if direct I/O is configured. Clarify why we round up the allocation size: it usually won't matter because we allocate multiples of the alignment size anyway. commit ee7456799277e19ff4771de79ac4eafac43aff9b Author: Mark Benvenuto Date: Thu Mar 19 16:16:15 2015 -0400 Visual Studio 2015 Preview support commit 9c60b462a25a36915207dde240579c5aa673f2c7 Author: Alex Gorrod Date: Fri Mar 20 00:09:42 2015 +0000 Update the eviction server to not set stuck if it isn't full. When only looking for pages to force out, the cache isn't really stuck. The trouble is that if we set stuck the eviction server doesn't clear walks which can lead to it always holding a reference t othe page we (really) want to evict. refs #1777 commit f0138353697d8706bd1f26e83b9fff8f4e83af8d Author: Keith Bostic Date: Thu Mar 19 13:04:17 2015 -0400 Create the base configuration file in a temporary file and rename it into place so a crash can't result in a corrupted base configuration file. refs #1775, #1776, SERVER-17571 commit 2e15cd6fc322c90c763394c52056bfebd4153aad Author: Michael Cahill Date: Thu Mar 19 10:25:31 2015 +1100 Avoid EBUSY errors from truncate caused by checkpoints. refs #1643 commit 3188352d623de85803db3dd6e5b5188822e2db4f Author: Michael Cahill Date: Wed Mar 18 13:00:53 2015 +1100 If the session-level attempt to lock a handle gets EBUSY, fall back to the slow path. The sweep server relies on there being a retry loop if an exclusive operation such as verify conflicts with a sweep. refs #1404 commit 5d8641568ac27e21d6671cb646a818708eb0aa28 Author: Susan LoVerso Date: Tue Mar 17 15:38:57 2015 -0400 Set the end of the log if we find a zero hole. refs #1766, SERVER-17569, SERVER-17613 commit f38e325f87d94f178c932328632857361fc3c92c Author: Alex Gorrod Date: Tue Mar 17 04:30:11 2015 +0000 Implement review feedback. The new configuration strings are: file_manager=(close_idle_time=30,close_scan_interval=10) Update the sweep test to take advantage of new configurations and shorten the runtime. commit 87dba78b65bc869c426363d86d02134aa8f8f2ac Author: Michael Cahill Date: Tue Mar 17 14:39:49 2015 +1100 Round up the "in-memory size" for WT_UPDATEs to a multiple of 32. They are a very common case, and for tiny updates, this seems to to a better job of matching tcmalloc's behavior at least. refs SERVER-17424 commit d277a088385401ff6e6664dd688a2fcab2ac4087 Author: Keith Bostic Date: Mon Mar 16 11:42:56 2015 -0400 Sanity check all splits for at least 100 keys on the page, if there are huge keys and a too-small cache, there's nothing to be done. commit d4aa136a20c9baf151c3185058ce089679e0de0e Author: Alex Gorrod Date: Mon Mar 16 06:10:12 2015 +0000 Ensure we free memory when statistics cursor open fails. refs #1760 commit 66622ac33357d53212ddfa362fdf3c6b439bf34b Author: Alex Gorrod Date: Mon Mar 16 05:55:45 2015 +0000 Update the split deepen sanity check. Allow a page that is using more than 1/4 of the cache to be split. refs #1759 commit f99114f785985d152ba8ddfa735771574e6bff5d Author: Michael Cahill Date: Mon Mar 16 16:43:27 2015 +1100 Fix the search for a free hazard pointer slot. Previously, it gave up early once all slots were allocated, even if they weren't all in use. commit 15fe04460c1f057811e0f3e02feb249c710d05ce Author: Michael Cahill Date: Mon Mar 16 16:40:49 2015 +1100 Track the number of hazard pointers active in the eviction server. Don't start a new walk if we are close to the limit. refs SERVER-17551 commit e6d8c8a56c4dc83f206d43fcaa21902b35b1e4e6 Author: Alex Gorrod Date: Mon Mar 16 05:20:22 2015 +0000 Add a new statistic tracking range of IDs pinned by a transaction. refs #1746 commit f954ab6b3fb6a51ecf3cf625cd3f5c5f818e0fde Author: Keith Bostic Date: Sat Mar 14 12:31:00 2015 -0400 Pass a allocated memory size to __wt_page_inmem(). commit 308dc500adcde7b62c8dbb9aa0fb401795445546 Author: Keith Bostic Date: Sat Mar 14 10:51:21 2015 -0400 Don't drop core if duplicate symbols are detected (caller's table gets free'd twice). commit dc19643859063a03b985d97e2f24b1262ec4b15d Author: Keith Bostic Date: Fri Mar 13 15:35:28 2015 -0400 Use SCNxxx #defines in sscanf() calls, not PRIxxx. We can't cast a pointer to a WT_LSN.offset to a pointer to a different type, we can't know WT_LSN.offset is the same size as that type. commit 99a992ed5207822ce2bc2fd69cb5b37408e6080f Author: Keith Bostic Date: Fri Mar 13 14:17:34 2015 -0400 Switch sscanf from SCNu46 to SCNi64 so we handle both hexadecimal and integer symbol/frequency values Add testing support for out-of-range symbol/frequency values. refs #1536 commit 48a6f6e76aafbf7b33dbb22325ddb4e8e27603a1 Author: Keith Bostic Date: Fri Mar 13 10:04:06 2015 -0400 Fix problems with configuration value parsing, break out the Huffman configuration file parsing code into a single routine. commit e7ef6d0c3d107f374de9924d981d731fe36fb4ec Author: Alex Gorrod Date: Fri Mar 13 16:32:37 2015 +1100 Update memory allocation accounting for new pages and realloc. They used to not always account for alignment overhead. commit 1e97f9730a6fd70c03784e292cb0435c8fb82354 Author: Don Anderson Date: Thu Mar 12 14:02:24 2015 -0400 Fix to track deleted cells. Remove unused vars. commit 5778e4098862665d4cdaf2cda3ef40d47d951efb Author: Alex Gorrod Date: Thu Mar 12 16:35:27 2015 +1100 Add ability to configure sweep server timings via API. Still needs some test code. commit a1c9f8f8833482c9773fc30b7b9a7b5f1ab09014 Author: Don Anderson Date: Wed Mar 11 22:48:12 2015 -0400 Made some adjustments to stats so that the btree entries is an accurate total for row and column stores. Entries that appear in internal pages (pointing to leaf pages or other internal pages) are no longer counted. Added and modified tests to check the number of btree entries reported by stats against the number of KV pairs expected to be in btree. refs #1733 commit 6d7c061a8292021195b32260a8b41d3e92e1958f Author: Susan LoVerso Date: Wed Mar 11 11:11:57 2015 -0400 Support no-logging mode in wt command for debug. refs #1732 commit 4a802bd592b6c8506b07900bf89a9d9fe53cfc25 Author: Keith Bostic Date: Wed Mar 11 09:30:39 2015 -0400 If the application only has 1 outstandin async operation at a time, we won't let them specify that. Change the minimum ops_max value to 1, but set the minimum maximum bound in the code to 10 as before. commit 3a412a2371e80ed9e26e546b6fe3c09d2d4e4091 Author: Keith Bostic Date: Wed Mar 11 09:29:55 2015 -0400 If the application re-uses a WT_SYNC_OP structure, we assert: return EINVAL instead, it's easier to debug than a core dump. commit 16119f98fee9525bf1990ccbcc979b9f08a8b000 Author: Michael Cahill Date: Wed Mar 11 20:24:20 2015 +1100 Don't cache an ikey before winning the race to split into a parent. refs #1582 commit 4d0b4093e3d3c9fa0be2bbb01467579f05deddc5 Author: Alex Gorrod Date: Wed Mar 11 03:44:22 2015 +0000 Update visibility check to know about checkpoints. This should allow us to evict more pages while a checkpoint is in progress. We can evict dirty pages from a file once the checkpoint is finished in that file. Similarly for the row store specific obsolete update check. refs #1745 commit 272daddb3734cd196cd303df4271f7e9e8f00cd3 Author: Alex Gorrod Date: Wed Mar 11 02:55:56 2015 +0000 Add a global checkpoint generation and track it per data handle. Not yet used, but will be used to allow more eviction during checkpoints. refs #1745 commit 89db28287d7b212efb31203eca253fbf144ea207 Author: Alex Gorrod Date: Tue Mar 10 07:19:59 2015 +0000 Mark statistics with the right aggregation flags (specifically cache dirty tracking and log preallocation). This helps wtstats graph generator. refs #1742 commit e51cc35a88ed497f911a25fec9f4bfcc6617d6a2 Author: Mark Benvenuto Date: Mon Mar 9 15:40:59 2015 -0400 SERVER-17471: Use uint64_t instead of long since long is compiler specific commit 105903c1beb0d16b40e243ed3624417c4fa0702c Author: Keith Bostic Date: Mon Mar 9 12:19:50 2015 -0400 Remove pockets of case-sensitivity for configuration strings. commit 1e8c2b89de09462253231213329674b8769d11fe Author: Keith Bostic Date: Sun Mar 8 13:05:51 2015 -0400 Change the statistics cursor WT_CURSOR.reset method to re-load statistics values. refs #1533 commit 769dc5976fe0d2448fb4fde511c5bc29eea39bfb Author: Keith Bostic Date: Tue Mar 3 16:45:09 2015 -0500 Change checkpoints to do first-fit allocation when we start writing the actual checkpoint blocks, that way if we delete significant space, the checkpoint blocks won't prevent file truncation. 3.0.1, Mar 9 2015 ----------------- commit d654795bb763b95d14604b9b65d09ae79b8ee5b6 Author: Alex Gorrod Date: Mon Mar 9 05:35:33 2015 +0000 Add a test case for checkpoint consistency. We recently fixed a bug where updating a page after the write leaves phase of a checkpoint had completed, then either checkpointed again or closed the database. We would end up with inconsistent data. That was the case because we weren't marking trees dirty all the time. This test case reproduces the scenario so we don't reintroduce the failure. refs #1735 SUPPORT-1248, SERVER-17506 commit 0315ee75f712ed0ccddca0616339de93b17835a3 Author: Alex Gorrod Date: Mon Mar 9 11:21:06 2015 +1100 Bump release version to 2.5.2 commit 4235c69d37474fb4e14673e0ea99337659db948d Author: Michael Cahill Date: Mon Mar 9 08:32:50 2015 +1100 When skipping a dirty page during a checkpoint, make sure the tree is marked dirty. refs SUPPORT-1248, SERVER-17506 commit 8382d14f32efc53b19aecd596cab3ba0d682b22d Author: Keith Bostic Date: Fri Mar 6 16:10:37 2015 -0500 Only increment/decrement from the current position on the first position set by our caller, that is, the position passed-in from our caller is the only position the caller has already seen. refs SERVER-17345. commit ed9c48c7b8fa5dd3362e417fda8337f1690585ed Author: Michael Cahill Date: Fri Mar 6 22:48:45 2015 +1100 Ignore empty child pages in column stores. commit 553a351ca6a81179c7f1db9c04d6f96aef0545ac Author: Alex Gorrod Date: Thu Mar 5 05:46:15 2015 +0000 Don't allow LSM bloom create to block waiting for space in the cache. We are only accessing one page at a time, and allowing the bloom create to block can lead to LSM not making progress. Refs: #1720 commit 5285b768de3237b4379f1526176efdeb55860971 Author: Michael Cahill Date: Thu Mar 5 16:15:08 2015 +1100 Remove the special "discard" path for trees marked clean: always go through eviction, which checks page modify flags. This may mean additional writes to internal pages when discarding a tree, but means that if a discard stops part-way through, the remaining in-memory tree has not lost any context. refs SUPPORT-1248, SERVER-17510 commit 00edf7a47115923bc9f3eaa7eee84c9f7d6b0d77 Author: Michael Cahill Date: Thu Mar 5 16:15:07 2015 +1100 If a page is split by eviction, mark the tree dirty to avoid having dirty pages in a tree marked clean. refs SUPPORT-1248 commit 57d3eba53fb91a0287374b9642b7cd4ef644854a Author: Susan LoVerso Date: Wed Mar 4 10:00:59 2015 -0500 Add fsync before closing log files and after header. #1717 refs SERVER-17451 commit d970bfe6b1bed7d1919b800bf2d65a3789b74d6f Author: Keith Bostic Date: Tue Mar 3 09:38:59 2015 -0500 Don't set eviction_workers_min/eviction_workers_max in the connection structure before checking the values are OK. commit 90f3f34a97440b6788a1a558e560a33fd116f166 Author: Susan LoVerso Date: Tue Mar 3 09:26:38 2015 -0500 Move writing into log worker thread. #1683 commit 1266bbb1143b22fec6b1c255b3aade5d0506477e Author: Alex Gorrod Date: Tue Mar 3 03:37:18 2015 +0000 Fix a bug in the reconfigure API related to shared cache quotas. While fixing the bug simplify the code flow for reconfigure and caches/shared caches. refs #1712 commit 1ebd617e6dfcf542983d62d9666c5a328dd41bb8 Author: Keith Bostic Date: Mon Mar 2 15:31:05 2015 -0500 Add overflow key/value counts to the statistics code refs #1520, #1703 commit a2166ead528ef61da478db67e3c0209a6bef5ac6 Author: Alex Gorrod Date: Mon Mar 2 14:18:37 2015 +1100 Allow memory_page_max to be at most 1/4 of the cache size not 1/2. If we let a single page grow to half the cache size, it's too easy for the cache to get pinned full if it is tiny to start with. commit b036921625e415bb66ac458922b81a9fae07740b Author: Alex Gorrod Date: Mon Mar 2 14:17:12 2015 +1100 Take all eviction candidates if we are aggressive. commit 0ddd3face0b99f5653001825bf3df0662ffcdc10 Author: Michael Cahill Date: Mon Mar 2 13:07:08 2015 +1100 Reverse the direction of the LRU walk each time the eviction server goes to sleep. Keep looking for candidates if eviction is stuck. Don't give up our walk position if eviction is stuck. commit cff10ba30f2eac379197e5d7dea49da2b8159890 Author: Don Anderson Date: Fri Feb 27 15:05:24 2015 -0500 Added test for wt dump on an index. refs #1709 commit a9f0e3ac769a060a8c3b06bac71fbed0e5f46cc6 Author: Michael Cahill Date: Fri Feb 27 16:59:35 2015 +1100 If we encounter a dirty page when closing a clean tree, switch to checkpointing. refs SERVER-17319, #1643?, #1404? commit bfcf5987b2b6f08d931d620330aed46837e3a8c2 Author: Michael Cahill Date: Thu Feb 26 16:32:09 2015 +1100 Add some paranoia to LSM around transaction checks: make sure we have allocated an ID before using it for anything. Recheck switch transactions before doing update checks in old chunks. refs #1641, #1701, #1702 commit da0bc67c821282e9fd0da725279811b59b25a675 Author: Michael Cahill Date: Thu Feb 26 16:32:09 2015 +1100 Add some paranoia around setting row-store internal keys. refs #1582 commit 5ea91f6ed0e0677530c5ab8215f81eb48ea307b8 Author: Alex Gorrod Date: Thu Feb 26 04:51:25 2015 +0000 Several optimizations to large page eviction: * Don't update the read generation on page in if it's set to oldest. * Clear the walk positions before the eviction server sleeps. * If only looking for pages that would block add them all to the queue. * If evicting dirty pages use the worker threads, not the server. commit 0eecd0a2d97771380ecbd7fd27bd44988db1148c Author: Alex Gorrod Date: Thu Feb 26 02:08:36 2015 +0000 Fix a bug in checkpoint, where it could get an EBUSY return. The case that could return EBUSY was when checkpointing with a specific target, while that target was open exclusively or for a bulk load. Refs #1404 #1589 commit 392a540deec817c5d6738b8e848a68882df3ac8a Author: Susan LoVerso Date: Wed Feb 25 15:03:50 2015 -0500 If the LSN given doesn't exist, don't return an error, but do force recovery. Fix recover.sh to grep the CONFIG. #1700 commit 59b699b7085868d1b12a41ae4cd7a01f25f6e865 Author: Alex Gorrod Date: Tue Feb 24 23:55:20 2015 +0000 Handle the case where a large record pushes us over a page boundary and we spill across two pages correctly. The previous fix could let us span additional space. Fix another bug in the fixup code - ensure there is enough space in the temporary buffer when fixing up after the fist page is full. refs #1697 commit ad8b58188961943e74c57b85e3b976aa03b79617 Author: Susan LoVerso Date: Tue Feb 24 18:54:18 2015 -0500 Set flags to SLOT_INIT_FLAGS on free. refs #1683 commit 45e4c049044efc96c682f390466a35d22dac555f Author: Keith Bostic Date: Tue Feb 24 16:17:32 2015 -0500 Move WT_PAGE.u.row.d next to WT_PAGE.u.row.entries, it reduces cache misses inside of row-search. refs #1665 commit 3c6d7adf422f432ae117e2292dcce00cc3b531a3 Author: Keith Bostic Date: Tue Feb 24 16:07:16 2015 -0500 Turn off key prefix-compression and rebuild the key before doing the boundary split. The size of the key is likely to increase by a few bytes, and if the value is large enough to consume almost all of the buffer, we overflow the space available. refs #1697 commit ca9ab16c320f6f154ff1fd3d0b65316f87e8bddc Author: Alex Gorrod Date: Tue Feb 24 06:51:14 2015 +0000 Fix a bug in reconciliation. In cases where we decide to squeeze one more large entry onto a page, we could have attempted to decrement space available negative. commit 777c35b074fce3656c14ca5770b424a65c719134 Author: Susan LoVerso Date: Thu Feb 19 12:32:16 2015 -0500 Add log worker thread to advance write_lsn. refs #1683 commit ae686a225a011bac07119e2d66e837e08d5a3a0e Author: Keith Bostic Date: Tue Feb 10 12:46:04 2015 -0500 Sasha says (my paraphrasing): I simply rearranged the fields in the WT_PAGE struct, so that u.row.d and u.row.entries are close together at the very end of the definition of union u, and the "uint8_t type;" immediately follows that. 3.0-RC10, Feb 24 2015 --------------------- commit 2fdfb2bbed56e42e1717e567828c68d0b2eb868d Author: Michael Cahill Date: Tue Feb 24 14:44:48 2015 +1100 Review places that set/clear session->dhandle, replace with macros. Change callers to save/restore if they need to release a handle after a call. commit ce89b608835561b11ce4e525a5ebdad86558f115 Author: Keith Bostic Date: Mon Feb 23 19:55:56 2015 -0500 The statistics server has open handles and may be opening underlying handles, make sure we don't overwrite them, reference #1694. commit 8827d909d7f14e9ad767d909d65598508ff0025c Author: Keith Bostic Date: Mon Feb 23 09:39:21 2015 -0500 Add an explicit barrier after setting the checkpointing value. (I'm pretty sure it's not actually necessary to have a barrier in the current code, but the barrier we rely on is in a different function and isn't always called, depending the eviction configuration of this file, I'd just rather be safe than try and debug this some time in the future.) commit 10abb2c47cffbcc215c3507256cee3e2cae2dd5f Author: Michael Cahill Date: Mon Feb 23 15:23:54 2015 +1100 If eviction is walking a file with only one live page (e.g., the last page), be careful to step over it, or eviction's hazard reference can make forced eviction stall. SERVER-17344 commit f72367f97357cf3e77f0d57d39992686b400ebe7 Author: Keith Bostic Date: Sun Feb 22 15:40:37 2015 -0500 Make sure we have a local copy of the start/stop cursor keys before calling into the underlying Btree range truncate function SERVER-17141. commit 09d345951d43e7ed928980048e5c5c927406a66b Author: Keith Bostic Date: Fri Feb 20 11:25:22 2015 -0500 If WT_SESSION.truncate isn't given a start cursor, instantiate one, it's always faster to traverse Btree objects in a forward direction. commit ce38bc5b40f1aa3ffc07cdd2cf993e32143135b5 Author: Keith Bostic Date: Fri Feb 20 08:57:57 2015 -0500 We don't need to do a search in the row-store cursor truncate setup code, the WT_SESSION.truncate API code already did one. This does not mean WT_SESSION.truncate can't return WT_NOTFOUND in some cases, the first thing cursor truncate does is a cursor remove call in order to acquire the page's write generation information -- that remove call does the usual "discard my current reference and get a new one" work, which allows another deleting thread to race and remove one of the truncate's start/stop keys. refs SERVER-17141 3.0-RC9, Feb 18 2015 -------------------- commit bf3ee2cd064b46cf0175d75950c825aa1f42c694 Author: Michael Cahill Date: Wed Feb 18 12:53:41 2015 +1100 Flip cache overhead to apply to the allocated bytes rather than the total size. Include the overhead in stats so that tools (e.g., mongostat) report accurate cache full and dirty percentages. This also makes eviction triggers and targets meaningful: with the default trigger of 95% and overhead 8%, eviction was previously never triggered until the cache was completely full. commit f9e6f942cf73c8a53aaadbc587c1b7efad6cc832 Author: Keith Bostic Date: Tue Feb 17 11:33:13 2015 -0500 Coverity notes the TXN_API_END_RETRY macro has an unnecessary test for "(ret == 0)" at the end of the do {} while loop. commit c34a56f357e21d134a2d9d0fefc032544069d8d7 Author: Michael Cahill Date: Tue Feb 17 21:56:34 2015 +1100 Allow the maximum number of eviction threads to be reconfigured. This was previously permitted by the API, but the array of thread contexts was not correctly resized, leading to segfaults. refs SERVER-17293 commit 67527fc235406469e69dbaec3dcd571469e660c0 Author: Michael Cahill Date: Tue Feb 17 21:50:53 2015 +1100 Make the eviction walk incremental: don't spend too long in any one file, fix tracking of whether we are making progress. commit 788265ed273c63183053e6325a9aa03c89c02860 Author: Michael Cahill Date: Tue Feb 17 21:48:44 2015 +1100 Combine the various checks for whether a page can be evicted into one place. commit 748e7b0c58b358b14340bacae41f9c46f3c06f7e Author: Michael Cahill Date: Tue Feb 17 21:47:57 2015 +1100 Skip hot pages during write leaves: checkpoint will have to visit them anyway. commit a9de0f7ac8ad373d7aef6a480c69a2a7e0b55c59 Author: Michael Cahill Date: Tue Feb 17 21:18:41 2015 +1100 Run recovery after crashing test/format in the recovery test. commit 4733961a3c1fa37988178d1b1dd4eb44d83b63f6 Author: Thomas Rueckstiess Date: Mon Feb 16 12:50:26 2015 +1100 fixes and improvements for wtperf parsing - convert wtperf microsec to millisec - don't skip monitor* files when parsing directory - parsing code reorganization - renamed wtperf stats fixture - added tests commit dc396e1cd64871219b9e5a1b6558707feb70706e Author: Keith Bostic Date: Sun Feb 15 13:44:30 2015 -0500 Clear the btree object statistics we're about to count, otherwise each cursor gets a cumulative value. commit 195b144bb37814b31cfa413029cda0b28f13f261 Author: Keith Bostic Date: Fri Feb 13 12:06:36 2015 -0500 Don't map WT_NOTFOUND to ENOENT unless a uri was specified, that's the only interesting case. Reference SERVER-17141. commit e9d7fee2c2c08985b8e2d2716e899853c5198290 Author: Thomas Rueckstiess Date: Fri Feb 13 17:22:05 2015 +1100 added support to parse wtperf files. they go into a separate stats section named "wtperf". commit 29d0d26fd1cd76392ea8225c1c4022ca54443737 Author: Keith Bostic Date: Thu Feb 12 18:05:33 2015 -0500 Ignore unexpected information in the metadata entry, the metadata entry might have been created by a future release, with unknown options. commit 05f07753059a4fa7f0f1bab7a107a9e6d17bf4af Author: Keith Bostic Date: Thu Feb 12 15:42:35 2015 -0500 Remove the requirement of a HAVE_DIAGNOSTIC build for the verify commands to work (except for dump_offsets, that requires the btree debugging code and so won't work anywhere but a HAVE_DIAGNOSTIC build). commit 006ed9f17c7fc0fe65dc43717ed0239b3bac564c Author: Keith Bostic Date: Thu Feb 12 15:31:36 2015 -0500 Add support for a new verify debug option, "dump_shape", that reports the levels of the tree. __wt_config_gets() returns WT_NOTFOUND when there's no entry, don't fail every command when DIAGNOSTIC #defined and debug options aren't set. commit 46b7721215856d08ca3a37f7ffc27c57b1d4c1d7 Author: Susan LoVerso Date: Thu Feb 12 13:27:32 2015 -0500 Add recover config setting and use it in the wt command. #1651 commit 0305a51ffba383af13d6078d409a03b249c502c5 Author: Don Anderson Date: Wed Feb 11 10:21:04 2015 -0500 Add test to detect file ID problems in recovery. Refs #1622. commit fc0ff5a9ea09e54512353d2275126cb54dbc5451 Author: Susan LoVerso Date: Tue Feb 10 13:02:28 2015 -0500 Allow 'wt' command to run with or without recovery. #1651 commit a26d87a53eb2ac2dcae9312b7979499c34c11613 Author: Keith Bostic Date: Mon Feb 2 19:20:33 2015 -0500 Replace wiredtiger_strerror_r with WT_SESSION.strerror, reference #1516. commit 33c146b51fdac86999e2eaa67f5636490eb441fb Author: Michael Cahill Date: Thu Feb 12 13:44:35 2015 +1100 Disable aggregation across all open checkpoints if statistics cursors don't specify a checkpoint. commit 04ec3d021d2f8b08b69d3ea5d0f243f468c71f2e Author: Michael Cahill Date: Thu Feb 12 13:00:49 2015 +1100 Move server thread waits to the beginning of their loops: check that we're still running before waiting. This makes more sense to me, but also fixes a problem introduced recently where the checkpoint server could hang on shutdown if the signal from the closing thread got lost. commit 85aae87cb1e019b0bcac4854e6508f11104f5339 Author: Keith Bostic Date: Wed Feb 11 12:34:03 2015 -0500 Mimic Alex's fix in 152a0ef, to fsync created files to disk, for truncated files, never surprise the upper-layer. commit f445f3bf63e3fa096479c5963f75d91e02f9b616 Author: Michael Cahill Date: Wed Feb 11 17:49:04 2015 +1100 If logs crossed the threshold size while we were taking a checkpoint, don't take another one immediately. Should help with SERVER-17206, where we saw two checkpoints every iteration. commit 0d85a9716b786de5fc90c00fb31765ade8aefd1f Author: Michael Cahill Date: Wed Feb 11 17:48:03 2015 +1100 Check if a page was recently split before doing forced eviction. We used to do this, but it got lost in a recent reorg of __wt_page_release. This change should mean that after an in-memory split, application threads that are appending have time to move to the new page at the end of the tree, rather than getting stuck trying to force out a page. SERVER-16938, SERVER-17121 commit 545d064fd4cbb0b35dc536e772c60b26a193d3f2 Author: Michael Cahill Date: Wed Feb 11 17:45:26 2015 +1100 When doing truncates, if we see a clean page in memory, try to evict it before truncating. It should be cheap (just freeing memory), and if the eviction succeeds, the fast truncate code can kick in and mark the whole page deleted immediately. Otherwise, truncate will mark each record on the page deleted, and the next time through will try to force that page out, which has to go through reconciliation to figure out that all of the records are deleted. SERVER-17157 commit 9bbb8595abd6ac962a0debf20a6cdcef73d83855 Author: Michael Cahill Date: Tue Feb 10 16:09:36 2015 +1100 Allow size-limited LSM trees to have Bloom filters, based on the normal configuration. commit c040f84a765c7c39f03e173a555eb50f85e2e698 Author: Michael Cahill Date: Tue Feb 10 15:58:02 2015 +1100 Re-enable the global setting to disable LSM merges. refs #1657 commit 8f14899ba0ce5b1a8df689e3c68db9a68bfeee66 Author: Alex Gorrod Date: Tue Feb 10 04:36:57 2015 +0000 Fix a bug when re-opening an LSM tree. We could have attempted to update the last chunk that is already on disk. commit d8263d46c1aa136d24ef194a8f62f0b02b92b9b0 Author: Michael Cahill Date: Tue Feb 10 15:11:59 2015 +1100 Improve LRU eviction of large pages: don't give up because a large page has recent updates: push on and try to do eviction and restore updates. commit eb02caa2564a18e857d18ef4b3f25683b438111c Author: Michael Cahill Date: Tue Feb 10 15:01:37 2015 +1100 Fix a local variable read when looking for pages evict racing with a page becoming dirty. refs #1660 commit da4d99e7ad57057a1b8397629d59a3c83c28de21 Author: Alex Gorrod Date: Tue Feb 10 02:32:25 2015 +0000 Fix a bug in LSM cursor open. The bug caused us to re-open more cursors than necessary in open. Related to fix: 439a655e commit eec16c3052af107bbe57aaf547eb8e70d2de4966 Author: Alex Gorrod Date: Tue Feb 10 00:26:37 2015 +0000 Don't do LSM merge throttling if merges are disabled. commit fcee4c8ce0b5db9d3340169deb321601b81f4a1b Author: Keith Bostic Date: Mon Feb 9 14:04:02 2015 -0500 Track splits during eviction by data-source as well as by connection. Don't double-count in-memory splits (we're incrementing cache_eviction_split in the underlying split-parent routine, not in the caller, so it's counting both normal and in-memory splits). Instead, cache_eviction_split is normal eviction splits, cache_inmem_split is in-memory splits, commit 3d1f9eace79b1aff84369d0caee245f9d6d96a60 Author: Alex Gorrod Date: Mon Feb 9 06:25:36 2015 +0000 Add a mode to LSM where we can limit the size of data in the tree. A feature request to allow for a high insert throughput into a table with a size limitation. Adds a new configuration option to WT_SESSION::create which is lsm=(chunk_count_limit=0), default to 0 which is disabled. Refs #1652 commit c63ba34c915d95c156aaf6c47a04fe6d361b91ad Author: Michael Cahill Date: Mon Feb 9 14:07:57 2015 +1100 Don't double-count the on-disk header size when setting split boundaries. refs #1655 commit 152a0efdbd3ea66b142f52eed3c9224437143eec Author: Alex Gorrod Date: Mon Feb 9 12:25:10 2015 +1100 Fix a bug in table create. A crash could cause recovery to break. Refs SERVER-17204 The bug is that we weren't doing an fsync of the file after it was created. Recovery assumes that if there are records for a particular file, then it will exist on disk. commit 4d50f5878073e582567848ae03ee506bb5058227 Author: Alex Gorrod Date: Mon Feb 9 00:43:27 2015 +0000 Remove obsolete updates every time we add a new update. We used to only do the check when the cache was full. That could lead to update chains growing immensely long, which is bad. Refs: #1647, SERVER-17195 commit e891a1f312850bcaaf5183f3fd2e091567044a96 Author: Keith Bostic Date: Sun Feb 8 17:59:49 2015 -0500 If we find a "removed" page, clean or dirty, leaf or internal, fast-path eviction, it helps with append-only workloads. commit ab2a7e9b397adf657081458e11f3dc472b10fd2b Author: Keith Bostic Date: Thu Feb 5 15:54:01 2015 -0500 There's a problem that went in in #1282, the key difference is that we are setting a split boundary at the end of the first page when there is more than a page worth of data. See also #1630 and #1631. This is an alternate approach to #1631: the real change is to fallthrough into the split case if the next item won't fit, callers of the split code can't handle failure from split, it has to create enough room for the next item to be entered into the buffer. commit 90a352717a45a40d047b33c9fb00e7174e1ae04f Author: Susan LoVerso Date: Thu Feb 5 12:59:08 2015 -0500 Initialize first_lsn if we have no logs. #1638 commit 7cc7efb75c90e778f9757b954ad3ec85912b58fd Author: Don Anderson Date: Thu Feb 5 12:20:55 2015 -0500 For wt printlog, make operations into a JSON array. Without that, any tool that parses JSON is almost certain to merge successive values of repeated fields. Refs #1438. commit 5bf11d893548804b890836a3d9ef4335c4319bb7 Author: Susan LoVerso Date: Wed Feb 4 15:46:30 2015 -0500 Add name_hash and hash bucket queues for fh and block. #1643 SERVER-17078 commit 3b0c18f612c9cf4d61bc13785ff7125fa67b265a Author: Alex Gorrod Date: Wed Feb 4 06:27:07 2015 +0000 Keep filling pages in reconciliation until we hit a boundary. This reverts some of a change for #1282 (without reverting the functionality in that change, AFAI can tell). Refs #1630 3.0-RC8 Feb 4 2015 ------------------ commit d8b7f0b8db92a2ad6d64b95cafeaf20f0a90c8ce Author: Michael Cahill Date: Wed Feb 4 16:00:11 2015 +1100 Updates should always mark pages dirty (before checking for obsolete updates to free). commit 0947f82e01587836277d911b147bc98eefb58507 Author: Keith Bostic Date: Tue Feb 3 10:28:00 2015 -0500 Fixes for split cache accounting: multi-page splits weren't correctly accounting for the allocated WT_REFs, insert splits weren't correctly handling the new right-page's instantiated key (the parent needs to be incremened by both the left- and right-hand page's keys, and cannot assume it's the same size as the original WT_REF's key), insert splits need to increment the parent page's WT_REF size by two, not one. commit df96addef5f3ffcb495b4bf54390cf3fd41ac924 Author: Michael Cahill Date: Mon Feb 2 16:45:46 2015 +1100 In recovery, track the maximum file ID in the metadata, regardless of whether there are any updates to roll forward. Previously, we tracked the largest file ID that was updated in the logs being rolled forward. It was usually the case that the most recently created file was also the most recently updated, so that calculation usually worked and wasn't detected until the repro in SERVER-17142 that created tables, did a clean shutdown and restart, then created more tables and did a dirty shutdown and restart, which was rolling forward updates into the wrong tables. refs SERVER-17142, SERVER-17131(?) commit 71f1559c91ed119082ebe42772da15e28915e1c8 Author: Michael Cahill Date: Tue Feb 3 10:40:27 2015 +1100 Start with clean trees so we can detect updates racing with sweep. Use a deleted ref to a leaf page that is created on first update, which is the same state the tree should be in if an empty leaf page is evicted. The only wrinkle is that bulk operations expect to find a leaf page in the tree: create it explicitly before the bulk insert starts. This was probably a bug before: if we had created a tree and kept it around for long enough with cache pressure before a bulk load started, the initial leaf page could have been evicted. commit 8545c4b3b7f5ed306215c82f1ad1cbe3664f0c50 Author: Keith Bostic Date: Mon Feb 2 17:13:09 2015 -0500 Make the "split to deepen the tree" configuration values real, stored in the metadata file, but they remain undocumented for now. commit fb769dafee4aca91a60a28cd89317c268ac79d4f Author: Keith Bostic Date: Mon Feb 2 16:36:22 2015 -0500 WT_CELL_ADDR_DEL is 0, so we can't test vtype against 0 to know if it's been set or not. Reference SERVER-16866. commit feca80738c1b9103b4faa04ddb0718344347f640 Author: Susan LoVerso Date: Mon Feb 2 13:53:21 2015 -0500 Wrap calls to functions using pindex with WT_WITH_PAGE_INDEX. commit 23f2e1ba0680a2e8fa7a081f1b46e1ae2ab220d4 Author: Michael Cahill Date: Mon Feb 2 17:18:03 2015 +1100 Once we decide to force-evict a page, do it directly rather than setting read_gen and hoping page release agrees. commit 5f00de07b5bad20a6ffb5ec7d412c4ca0b10c64f Author: Michael Cahill Date: Mon Feb 2 17:11:27 2015 +1100 split_gen paranoia: always increment split_gen once per split, use the allocated value to check for existing readers. Make sure that publishing a split_gen doesn't miss an update. commit 10a74d6af4f945e34368bc5754797ef1d684d8ab Author: Michael Cahill Date: Mon Feb 2 16:52:34 2015 +1100 If discarding a tree for sweep races with an update, give up the discard gracefully. refs #1618, SERVER-17048 commit a2d20dc49cac870977d91213a7fe6dabf362ce70 Author: Michael Cahill Date: Mon Feb 2 16:45:46 2015 +1100 In recovery, track the maximum file ID in the metadata, regardless of whether there are any updates to roll forward. Previously, we tracked the largest file ID that was updated in the logs being rolled forward. It was usually the case that the most recently created file was also the most recently updated, so that calculation usually worked and wasn't detected until the repro in SERVER-17142 that created tables, did a clean shutdown and restart, then created more tables and did a dirty shutdown and restart, which was rolling forward updates into the wrong tables. refs SERVER-17142, SERVER-17131(?) commit b0a828b262a2d0d3cf1361eed98aa25a1168a7a6 Author: Keith Bostic Date: Sat Jan 31 12:59:34 2015 -0500 We no longer calculate allocation overhead per allocation chunk, revert the workaround the problem with page memory size calculations during splits where we forced the new parent page memory size to 5% of its current value; reference #1564, #1565. This fixes a problem where 5% of a page's memory footprint isn't large enough to accommodate the cache decrements that will be done in the page's future, leading to page underflow. Minor cleanups: we no longer calculate allocation overhead per allocation chunk, the macro WT_MEMSIZE_ADD is no longer needed at all, and WT_MEMSIZE_TRANSFER is renamed to WT_MEM_TRANSFER. commit b640366c28fc66744e482c20c16973cb052aef8e Author: Keith Bostic Date: Fri Jan 30 10:19:31 2015 -0500 I believe we can race with pages being marked clean or dirty, which means we need to entirely divorce the page's dirty-byte count from page state: the page's dirty-byte count is just a value that tells us how many dirty bytes this page has contributed to the cache's total dirty-bytes count. Sync the cache's information to that value when possible, but don't worry if we can't. commit d02ea7246ec33e05b5fd60c499fea3ffe25c57d2 Author: Michael Cahill Date: Fri Jan 30 17:38:09 2015 +1100 Use reads to measure cache pressure with shared caches. We previously tracked writes, which were skewed by checkpoints. refs #1569 commit a326c3ba10e0d299944a650b890f8c2d851db34a Author: Keith Bostic Date: Thu Jan 29 17:19:06 2015 -0500 Simplify the cache calculation when a page is marked clean, use the page's dirty-byte count (which allows a race between the page being marked clean and being re-dirtied). This branch is still not correct, but appears to be able to run the CONFIG from #1582 without underflow for a much longer time). Reference #1605. commit 1c60c4966dd68ea2bf05ebe62e3f1d8de1a7d033 Author: Michael Cahill Date: Thu Jan 29 14:33:36 2015 +1100 Use a copy of the oldest transaction ID when sweeping cached overflow items. Otherwise, we could free structures that are still hooked into the skiplist. refs #1615 commit 42724267278c64f5af68b281c9ee5742d1a56d72 Author: Susan LoVerso Date: Wed Jan 28 10:31:51 2015 -0500 Adjust logging yield and timeout values. #1610 commit ae102f4fe604f7fd547dece8ee138e8292d4f07c Author: Michael Cahill Date: Wed Jan 28 17:40:06 2015 +1100 Cleanup accounting for update lists when restoring updates to evicted pages. Previously, we only accounted for the first update in a list. refs SERVER-16997 commit 4adf9c929b1b46f273239214b4e2757fcfdb8f96 Author: Mark Benvenuto Date: Fri Jan 23 18:29:15 2015 -0500 Windows Install Documentation commit 8faa218d27e7f21091f0b51a973f27047db1d950 Author: Mark Benvenuto Date: Tue Jan 27 13:47:24 2015 -0500 MCI configuration update commit 422cbb6cea5fa5be6829044215ae46dc10be5f70 Author: Mark Benvenuto Date: Mon Jan 26 16:11:29 2015 -0500 Add Install Target to SCons commit 41e7ab083d79a650e93a34d09e01e973ca4100d9 Author: Mark Benvenuto Date: Fri Jan 23 15:54:40 2015 -0500 WiredTiger DLL support - Examples that only depend on public API use DLL now commit 23b2493e75cd166075eaccdaef75c8beee4576db Author: Mark Benvenuto Date: Fri Jan 23 15:50:16 2015 -0500 Scons Improvements - Added --enable-attach, --enable-diagnostic, --enable-verbose - Renamed --enable-swig to --enable-python for consistency + swig cleanup - Renamed wiredtiger static library to libwiredtiger.lib commit 96ab0ef67eee20fa75fa6d52c97d98bc119b74ae Author: Mark Benvenuto Date: Thu Jan 15 15:07:01 2015 -0500 Struct alignment and packing for MSVC commit f3b65997ece52382eed91730416d5f919bea79cd Author: Susan LoVerso Date: Fri Jan 9 10:49:59 2015 -0500 Fix huffman config and add huffman tests. #1536 2.8-RC7 Jan 27 2015 ------------------- commit 2b4172f17008ff36dbeb50cadaf4fb97fc859e4e Author: Michael Cahill Date: Tue Jan 27 15:50:09 2015 +1100 Revert a workaround for splits during truncate. refs #1583, #1563, SERVER-16868 commit c2e108e2774ae79504579bcdca33f26fcff8cb07 Author: Michael Cahill Date: Tue Jan 27 09:58:32 2015 +1100 Change recovery to start from the checkpoint LSN in the metadata. Don't assert that we see a checkpoint complete in the available log: if the application crashes in between syncing the metadata and writing the final checkpoint record, there is no need to roll anything forward but we don't have the final checkpoint. refs #1529 commit 2555e80d2020ba9833c436a22d1031f6c5778a64 Author: Keith Bostic Date: Mon Jan 26 14:31:25 2015 -0500 Coverity CID 50796 (#1 of 1): Unintentional integer overflow (OVERFLOW_BEFORE_WIDEN) SERVER-17001 commit 1ce3b94d6e40d37a77e62eda500f286bd3816eb9 Author: Michael Cahill Date: Mon Jan 26 15:56:25 2015 +1100 Grab the table list lock while building the list of handles to checkpoint. This avoids a potential deadlock during compact operations and/or checkpoints with a target list (and an assertion about lock ordering in diagnostic builds). Note that nested locking is not ideal: the medium-term fix here is #1598. refs #1589, SERVER-16967 commit db3943563a87c3e4c42445ae9f3a07efacfdf4ac Author: Michael Cahill Date: Mon Jan 26 14:54:47 2015 +1100 Free WT_REFs deleted by truncate. We were doing this when a page spontaneously became empty, but not if the "fast truncate" code kicked in. refs SERVER-16921 commit 2063efb22c3c29b980f86f7fee77b6d03ba63ec1 Author: Keith Bostic Date: Fri Jan 23 16:21:06 2015 -0500 Don't count pages evicted by a worker thread as an "application thread" eviction; add a new statistic to distinguish between the server itself evicting pages and the eviction worker threads evicting. Don't increment the eviction counters unless we find a page to evict, __evict_lru_pages() gets called a huge number of times in any workload where eviction is happening. Reference SERVER-16997, SERVER-17020. commit 3abb99d58aaa46b0b3fcd338293a668422e3fcaf Author: Mark Benvenuto Date: Fri Jan 23 15:05:16 2015 -0500 Close Thread Handle after thread join on Windows commit 7d677aedfdcaa5458e900e556b662def460d0281 Author: Don Anderson Date: Fri Jan 23 08:52:13 2015 -0500 Fix drop index on a newly opened session. Fix __wt_schema_open_index to return WT_NOTFOUND when opening a single index. This fixes opening a cursor on a non-existant index. Refs #1567. commit 3626081dff24e1448281d10658752b996897ca82 Author: Keith Bostic Date: Thu Jan 22 18:08:15 2015 -0500 Add the cache_overhead configuration string to allow applications to configure their cache overhead. commit 4843cd78e7f90937ebdb23f84fbd7c133a7e5256 Author: Don Anderson Date: Thu Jan 22 10:40:11 2015 -0500 Prepend underscores to SWIG methods that could have name conflicts with WT internal names. refs #1574. commit ebb1d9402c0ce2911069b0437d71766b92c3dc12 Author: Susan LoVerso Date: Wed Jan 21 12:57:20 2015 -0500 Add log code to ensure write-no-sync. #1585 commit 44fa4fbff95d0689b20c3fe3f4a55202554f0d9f Author: Keith Bostic Date: Mon Jan 19 14:25:39 2015 -0500 Make compact more aggressive about finding blocks to move. 2.8-RC6, Jan 20 2015 -------------------- commit ab1d63d3aa2371ce53287c6c6c77833eb281a38a Author: Susan LoVerso Date: Tue Jan 20 15:37:46 2015 -0500 Check for valid log_fh handle in wt_log_write. #1580 commit e2de971061abea9451e92d60f0870136c9c0af42 Author: Keith Bostic Date: Tue Jan 20 13:24:06 2015 -0500 Quit page eviction immediately if we're trying to evict a tree, that is, an internal page that has other internal pages as children. commit 6f3c5a933ef8ce79efc03a22a8c03526ffb2197b Author: Keith Bostic Date: Mon Jan 19 12:38:24 2015 -0500 The size of the file is decreasing each time, so compacting 10 times, at 10%, is not sufficient to drive a file to its smallest size. The right fix is probably to get better information from the block manager as to exactly how much the size of the file has decreased, but that's messy, especially when you consider the checkpoints requires to get to that smallest size. For now, do 100 compaction attempts instead of 10, and depend on the no-progress state and/or the compaction timeout to limit the amount of time we spend here. commit 72172b088fba6769866aecabba8176303140f5c4 Author: Keith Bostic Date: Mon Jan 19 10:25:13 2015 -0500 Coverity 1264611, memory leak (WT_RET that should have been a WT_ERR). commit f61f984cf5241ac54bc2ea368c8c15e0cdfa91aa Author: Alex Gorrod Date: Sat Jan 17 22:25:02 2015 +0000 Fix a deadlock opening statistics cursors. Refs #1575 and JIRA SERVER-16738 commit c5fa51a0f18e4117d9f7b841de86eb35af751264 Author: Susan LoVerso Date: Sat Jan 17 09:07:04 2015 -0500 Log close thread needs to wait for any outstanding writes. #1571 commit 9cd8120f491595ad6ac1c25c4b154ad6556b5fe7 Author: Michael Cahill Date: Sat Jan 17 09:49:09 2015 +1100 Close the session for the log close server thread. Fixes a leak detected by address sanitizer. commit bd7364ea9a0542bee61db0a89e771faf814f6f53 Author: Alex Gorrod Date: Fri Jan 16 21:03:55 2015 +0000 Fix a bug in raw compression, where we were overflowing memory. We weren't growing the buffer enough when adding new items in. Refs SERVER-16664 commit 76addf73581c53f24462ab5fd724048aec36eaf3 Author: Michael Cahill Date: Sat Jan 17 05:48:03 2015 +1100 Have WT_CURSOR::equals return 1 when cursors are equal, 0 when not. commit b2841dfc015d9502e1def870605968144b935570 Author: Susan LoVerso Date: Thu Jan 15 21:58:39 2015 -0500 Add log thread to fsync and close log files. #1560 commit ebb93969ebfb6b9bb9dc60621933f2fbeac4b472 Author: Keith Bostic Date: Thu Jan 15 22:47:52 2015 +0000 Don't do memory adjustments for the WT_REF's WT_ADDR structures, we don't do those adjustments in other places we set addresses. Workaround the problem with page memory size calculations during splits by forcing the new parent page memory size to 5% of its current value; reference #1564. Minor cleanups/renaming of the code instantiating the WT_REF structures during a tree-deepening split to clarify what's going on there. commit e0031209183c880fb1a1b99399013e7675a75e88 Author: Alex Gorrod Date: Fri Jan 16 09:26:34 2015 +1100 Don't look at a page after it may be freed during split. During the process of doing a split we switch the ref to WT_REF_MEM - after which it's no longer safe to refer to the page. Shuffle the code so that we don't. SERVER-16868 commit b6d7532cbf823d537b8f1733169fe4de08173c09 Author: Susan LoVerso Date: Thu Jan 15 16:55:00 2015 -0500 Only advance sync_lsn to the end of our write. We waited until the log->sync_lsn is advanced into our file. It was a bug to set the sync_lsn to the current write_lsn as that can be too far ahead in a new log file when earlier log files aren't done yet. commit 85851933a938c53dfa57d1621cab1a959db672eb Author: Thomas Rueckstiess Date: Mon Dec 15 11:04:43 2014 +1100 wtstats.py: removed python-nvd3 dependencies, rewriting with HTML template commit 4c26d2324bae1d7030b0142d50dbd2ccf11ddeb6 Author: Keith Bostic Date: Thu Dec 11 19:32:50 2014 -0500 Add support for a WT_CURSOR.reconfigure method, reference #1381. 2.8-RC5 Jan 15 2015 ------------------- commit 2e54a27683c5e2fd88918575383c76d3f60c3c78 Author: Michael Cahill Date: Thu Jan 15 07:17:21 2015 +1100 Workaround a read-after-free involving eviction during truncates. We were implicitly relying on first_dirty_txn to prevent pages being immediately force-evicted by truncate. The bug is not fully understood, but this change restores the previous window where reads can complete before the page is evicted for real. refs BF-759 commit 8a1bfe3c35f0c1d90ea3e8e70c2aae8dff1fdbb3 Author: Susan LoVerso Date: Wed Jan 14 15:07:39 2015 -0500 Force log file closes to go in sequence. #1555 Update the sync_lsn after sync'ing and closing an earlier log file and make sure archive doesn't try to remove a file that is still in use. 2.8-RC4 Dec 22 2014 ------------------- commit fbb96d94cdba9a28f5c5d737ce6c96543f3289f4 Author: Michael Cahill Date: Mon Dec 22 15:59:46 2014 +1100 Use the original page's first_dirty_txn when restoring updates to match what we do for in-memory splits. refs #1475 commit 4df72e8e20139ddf667e1f0d3b6b7dcf91deb006 Author: Michael Cahill Date: Mon Dec 22 13:12:07 2014 +1100 Avoid EBUSY returns to verify and salvage caused by checkpoints. The "fix" involves blocking checkpoints while salvage or verify are in progress. refs #1404, SERVER-16457 commit 864f3495721b1311b49df19ee241bfca9adf0863 Author: Keith Bostic Date: Sun Dec 21 20:47:52 2014 -0500 Make the cache bytes-written and bytes-read match, both should ignore compression. Reference #1505. commit 995d6f8c26ae19013a1eb921fd871481ca643f47 Author: Michael Cahill Date: Mon Dec 22 12:42:46 2014 +1100 Eviction should do update-restore if upper layers are trying to force out a page, regardless of its size. Also, only look at ref->page after checking for exclusive access. It is possible (but very unlikely) that a child page pointer could be replaced in the window where we are checking hazard pointers. commit d4abc51ea61211f90f4b70a0486442264ededc27 Author: Alex Gorrod Date: Mon Dec 22 10:43:04 2014 +1100 Fix a bug where a custom extractor terminate was being called twice. Resolves issue #1503. Clarify the custom extractor and collator terminate documentation while I'm here. commit 16972ef63de1283d85146530c35f522b053e2c1e Author: Don Anderson Date: Fri Dec 19 09:56:47 2014 -0500 Remove version numbering from the pkg-config file. We don't create include files that are named by version. Programs linked using -lwiredtiger will follow the symlink to wiredtiger-a.b.c.so, so their referred library name is forever stamped as wiredtiger-a.b.c.so, which won't conflict even when we ship wiredtiger-a.b.d.so. Refs #1458. commit e913b0811114d65b543cd78824e809eb487fd330 Author: Michael Cahill Date: Fri Dec 19 17:19:58 2014 +1100 Check that handles are not being walked by eviction before discarding. refs #1497 commit 0d21e437917bc7cf08393852a3074957431ea30e Author: Alex Gorrod Date: Fri Dec 19 15:44:15 2014 +1100 Use the eviction server to write pages with READGEN_OLDEST set. Even before the eviction trigger has been reached. This should mean that we clear those pages out of cache earlier, and hopefully will save application threads from doing the evictions (at least sometimes). commit e0adfba3c4011c49b73ff3e4a165a4a938f69cb3 Author: Michael Cahill Date: Fri Dec 19 12:23:21 2014 +1100 Don't try to write leaves from the sweep server. Previously, this was done without locking the handle, and so could race with LSM discarding a handle. We know the handle has been idle, so there is a good chance that a checkpoint has run since the last update and the write leaves was wasted effort. If not, this change will keep the handle locked for longer preventing new opens, but it has been idle for a while anyway. refs #1495, #1497 (maybe) commit 87328a8d5c1d4a201a1df604ba32a87863948bbb Author: Mark Benvenuto Date: Thu Dec 18 14:51:51 2014 -0500 fix test_salvage on Windows commit 1953776ada137f3deae50169bf889d2063b353d3 Author: Michael Cahill Date: Thu Dec 18 16:45:06 2014 +1100 Don't try to set and clear session->split_gen in WT_INTL_FOREACH_BEGIN: there are too many of those loops to ensure that none of them skip clearing it. Instead, make sure all calls are wrapped in WT_WITH_APAGE_INDEX. refs SERVER-16546 commit 235f747e2df80d9899497595a2b649e7d6df8601 Author: Mark Benvenuto Date: Wed Dec 17 14:11:41 2014 -0500 snprintf - Implement a custom version of snprintf match the truncation behavior of C99 standard snprintf until MSVC supports snprintf. commit 857a6fd0c4b6b001c78cbbc507674e2129029dff Author: Keith Bostic Date: Wed Dec 17 15:51:41 2014 -0500 Check the block header checksum before we clear it, it should be the same as the passed-in checksum, and if just those 4 bytes are corrupted, we wouldn't noticed. SERVER-16457. commit dfa706056c4a359f7f894047bc9e5399efcec776 Author: Don Anderson Date: Tue Dec 16 15:58:34 2014 -0500 Some refactoring of python packing. More checks for standalone unit tests. Refs #1429. 2.8-RC3 Dec 17 2014 ------------------- commit bb064847e1c45f2b396d3f65f4e08cd10f33ed6e Author: Michael Cahill Date: Wed Dec 17 15:45:23 2014 +1100 Detect write-write conflicts before no-overwrite cursors decide to skip an update. refs SERVER-16351 commit 91abf8e35d5246a653bd9615ffd9723d87999c38 Author: Alex Gorrod Date: Wed Dec 17 13:36:02 2014 +1100 Add support for none configuration string to log compressor. To be consistent with block compression configuration. commit 5438fee4942b4dbf484799dad6e12e042d253e99 Author: Alex Gorrod Date: Wed Dec 17 12:09:33 2014 +1100 Return an error if a shared cache configuration is set, but not enabled. Check for a configuration via shared_cache=(size=). Refs #1487 commit 390a5b71b25492dc3030e908a65a11a04401852b Author: Keith Bostic Date: Tue Dec 16 15:32:09 2014 -0500 We documented that huffman_key and huffman_value took "none" as an argument, but they didn't. Worse, if key was set but not value, we'd set value anyway, free of charge, and if value was set but not key, we'd fail. I doubt this is a problem (it's pretty surprising if anyone would set key but not also set value). Use __wt_config_gets_none() to support the "none" setting, re-work the logic to ignore keys or values that are zero-length. Reference #1417. commit 662e26eeb31f76f2c4aeebf6690c9056612de32e Author: Susan LoVerso Date: Tue Dec 16 14:20:01 2014 -0500 Atomically create all log files and move them into place. #1482 commit 5c30d62dbf7ec0976d6ec4d2aed4ba272aadd499 Author: Keith Bostic Date: Mon Dec 15 19:41:05 2014 -0500 If we race with the logging thread and get to __wt_logmgr_destroy() while __log_archive_once() is still using conn->log_path, we can free it out from under the running server. If there's a logging thread running, don't free conn->log_path until we've joined that thread. Reference #1480. commit d77d35db407fd74c266bdb728b12c74fdab26ba2 Author: Keith Bostic Date: Mon Dec 15 18:33:54 2014 -0500 Even if we don't track any overflow pages during our read of the file, we still have to process the list of pages looking for leaf pages that reference unavailable overflow pages, no overflow pages doesn't imply there are no references to overflow pages. commit cedf8cfe69bf964629aab498feb20a0b1ab77bc0 Author: Don Anderson Date: Mon Dec 15 11:54:15 2014 -0500 Fix use of 'compressed' flag for printlog. Added printlog call to test case for log compression. Refs #1472. commit 3210b11cf7bfb79f3ed52cd1c17a13c644a82e7a Author: Susan LoVerso Date: Mon Dec 15 11:02:10 2014 -0500 Fix memory leak. Always free log_path. #1473 commit 097c61e5f3326bc71f2d645b3f539c5c6d1ae3fb Author: Alex Gorrod Date: Mon Dec 15 05:58:06 2014 +0000 Allow printlog to work without recovery. It now works even if compression or a custom path are setup. commit b8921272755ce66d09ab2a001745573420bd41ac Author: Keith Bostic Date: Wed Dec 10 15:40:06 2014 -0500 We can't use the corrected page size to calculate the buffer's space available, but we don't have a page size either. We do know how much space we added, so use that to increment the space available. The raw compression handler can no longer pass a page size buffer to the underlying compression function, because a single key/value object could be larger than the page size, instead, pass a buffer of the same length as the source to compress, whatever that is. commit 402041727de02931be1dd385f3c970f31a53341c Author: Michael Cahill Date: Wed Dec 10 10:53:27 2014 +1100 Remove the min / max bounds on overflow sizes. commit 5088ee53fce569915e8de8c168da50cff7991ec1 Author: Keith Bostic Date: Tue Dec 9 16:22:20 2014 -0500 Separate the btree maximum key/value sizes from the underlying page size, reference #1282. Deprecate the internal_item_max and leaf_item_max configuration strings, replace with internal_key_max, leaf_key_max and leaf_value_max. Remove examples/c/ex_file.c (there's no real need for a "file" URI example, and it's easy to replace the one place the documentation used it). commit e1e187e8fdfb48526f2a62e3f0f48072c30db53e Author: Michael Cahill Date: Tue Dec 16 10:29:08 2014 +1100 Update the swept handles statistic any time we close the underlying handle, regardless of whether a session still references the data handle. reds #1460, #1466 commit 080b34fde5de97459c383c67ba93d9fdc88090a8 Author: Don Anderson Date: Fri Dec 12 11:13:22 2014 -0500 Allow pruning scenarios with different limits for default vs. long runs. The pruned scenario list now matches the original ordering. Tests now print with both the scenario number and the scenario args. This should make it easier to diagnose and debug problems that effect only certain scenarios. Refs #1461. commit d5b88e08e2f6e39d098cfff3c013f4aa035c88bc Author: Don Anderson Date: Thu Dec 11 15:13:11 2014 -0500 Changed python test suite to allow for shorter runs by default, with a --long (or -l) option for the complete runs. Txn02 in particular now runs only a small number of scenarios by default. Also added a @longtest("description") decorator for individual tests that can be marked to be run only under --long. Refs #1461. commit 2f37332e5bbd14823f0c78ad38672dbce074e87f Author: Alex Gorrod Date: Fri Dec 12 17:21:50 2014 +1100 Ensure metadata table is open at start of checkpoints. commit 2cb10882f4f7189a3c2de4d7e187117873fded32 Author: Alex Gorrod Date: Fri Dec 12 17:07:11 2014 +1100 Switch to stashing the metadata dhandle, not the btree. Also update checkpoint to use the saved handle instead of searching for it again. commit 42c05161cf8cc74606b259ceeeb41dd38ea7fb4e Author: Susan LoVerso Date: Thu Dec 11 15:24:14 2014 -0500 Use dhandle hash lists in more places. Adjust sweep timings. #1460 commit 7fb6315e45e74f0bef0a04505018e1ab0b68d144 Author: Michael Cahill Date: Thu Dec 11 23:06:19 2014 +1100 If LSM search_near finds a matching tombstone, step the whole LSM cursor next to find the closest key. We can't step individual chunk cursors, or we could return a record that is deleted in a more recent chunk. MongoDB BF-694, BF-700 commit 5f6bbc898564aefb312255555abd34202cb98815 Author: Michael Cahill Date: Wed Dec 10 14:39:25 2014 +1100 Track whether eviction is making progress regardless of whether the cache is 100% full. Otherwise we can get into a tight loop. Use the count of pages evicted rather than a flag, now that there are multiple eviction threads. commit cbe9e9bdbc508f95076b8097d41bb4cc799eab1c Author: Don Anderson Date: Tue Dec 9 19:11:52 2014 -0500 Change timing to allow archive thread to complete on tests that do archive. Since this can make each test run substantially longer, reduce the number of tests that are doing archive from ~4000 to something under 100. Refs #1452. commit 62af85890179abb9fda17a619fcd5ae69fb369e0 Merge: b83bf08 b24c7af Author: Michael Cahill Date: Wed Dec 10 11:03:48 2014 +1100 Merge pull request #1449 from wiredtiger/lsm-switch-simplify Improve and simplify the LSM switch logic commit 7e0f7d7b803f9af04ad10b2bec6ef5073aa79248 Author: Don Anderson Date: Tue Dec 9 16:04:48 2014 -0500 SESSION->drop with "force" of nonexistant index/colgroup should be silent. Refs #1436. commit be364821d75c0c42169d79c486fa582c777f7082 Author: Michael Cahill Date: Tue Dec 9 15:58:28 2014 +1100 Sweep old handles more aggressively: 1. don't have checkpoint or other periodic operations like statistics logging keep old handles alive; 2. don't wait for all sessions to empty the file from their cache before closing; 3. only update the time of death from the sweep thread. commit c96a4c954ccc73744f8a1fbcf2fea6debdfca018 Merge: cc8eb0b 2e332b9 Author: Alex Gorrod Date: Tue Dec 9 14:41:36 2014 +1100 Merge pull request #1443 from wiredtiger/cursor-open-optimize Cursor open optimize commit 993c8ede8ff64eac9e87d1adcd39f8575039222b Author: Michael Cahill Date: Tue Dec 9 13:21:13 2014 +1100 Improve and simplify the LSM switch logic: it is fine to keep writing into a chunk while it is being switched, until either it hits the hard chunk size limit or a switch transaction ID is chosen that is larger than the writer's. Fixes an assertion failure introduced by #1432 that could write to an old chunk after the switch completed. refs #1432, #1418 commit 5551461cd5f26249e4330c9f87b4945d7ec2bb34 Author: Alex Gorrod Date: Mon Dec 8 18:01:31 2014 +1100 If there are only two LSM worker threads, don't let the cache get full. Allow the first thread to do flushes as well as switches and drops if there are only two threads. Refs #1441, but this is a hang seen from that test/format configuration, not a segfault. commit 8f06d6b79dabed54ad1e05515bbdb31e23c4e991 Author: Don Anderson Date: Fri Dec 5 14:22:41 2014 -0500 Modify printlog output so that arbitrary strings are shown as using the JSON Unicode standard. refs #1438. commit 68090796dea07e7b2d3d5bee8d69aafcd8febe16 Author: Alex Gorrod Date: Fri Dec 5 04:36:25 2014 +0000 Avoid string comparisons when looking up tables. Save a hash value in the table so we can do integer comparisons rather than string comparisons. commit 4de5e3a71bfad1c2a9ef1eccccdd45ec02fecba9 Author: Michael Cahill Date: Fri Dec 5 13:01:51 2014 +1100 Force eviction if we see many consecutive deletes when scanning through a page. This fixes quadratic behavior in find-first+delete workloads. commit bbced52c939e16ad5662b3a177cef3e52abddd6e Author: Keith Bostic Date: Thu Dec 4 07:57:13 2014 -0500 In the final close, continue and remove the handle no matter what errors we see, otherwise the handle-close code in __wt_conn_dhandle_discard() can become infinite loops, where we repeatedly attempt to close the same file handles. Reference #1434. commit 46fa7f0b6397fe765c5e8c2853f9cd0b067bc808 Author: Keith Bostic Date: Wed Dec 3 13:46:01 2014 -0500 Changes in #1204, #1288 mistakenly changed the values for some error defines, which breaks backward compatibility. (The WT_DEADLOCK error name sorted differently from WT_ROLLBACK, and we were assigning error values based on the sort order in a script.) Revert the change, and make sure it doesn't happen again. commit 249e88485c75951a0584a7c7a8dd4b8f8b6a3382 Author: Keith Bostic Date: Wed Dec 3 06:25:16 2014 -0500 Support "none" in all configuration strings as an alternative to an empty string. Reference #1417. commit 63d7c7869f8c2ab5a3e6ee935d1e37f21d40755f Author: Don Anderson Date: Tue Dec 2 14:00:11 2014 -0500 Added log compression. When configured, we attempt to compress each log record. Added printlog output to show before/after compression sizes. Refs #1359.