Import wiredtiger: 5b5d7988560a20dbe827a1fa16c0481e6c0e2821 from branch mongodb-4.4

ref: 660f3363bd..5b5d798856
for: 4.5.1

WT-3821       Python test case fails due to pinned timestamp warning
WT-4230       Remove named snapshot support
WT-4458       Only sweep btree handles that have been evicted from cache
WT-4523       Fix type incompatibilities during alloc and free of memory for async ops
WT-4671       Remove Limbo pages
WT-4674       Instantiate birthmark records using in-memory metadata
WT-4676       Search LAS for the history and compare to in-memory update
WT-4677       Re-initialize prepared updates
WT-4678       Write the test workload for relevant history in cache
WT-4681       Check if expired content is being swept from lookaside
WT-4683       Reverse the modifies when writing to LAS
WT-4685       Add new statistics for relevant history in cache
WT-4868       Aggregate btree write gen from leaf pages in salvage
WT-4954       Document duplicate backup cursors
WT-5001       Migrate Jenkins “wiredtiger-test-format-stress-sanitizer-ppc” job to Evergreen
WT-5045       Add more statistics tracking where checkpoint spends time
WT-5052       Enhance lookaside search functionality to handle WT_MODIFY records
WT-5058       Stop initializing a page with updates from lookaside
WT-5066       Extend test_las06 to support column store.
WT-5072       Modify __wt_rec_upd_select() to pull an update from lookaside
WT-5073       Review update chain traversals
WT-5089       test_las02.py fails: Unable to locate the correct record
WT-5092       Modify lookaside schema to add timestamp and transaction id to the key
WT-5098       Modify __wt_update_alloc() to accept NULL WT_ITEM values
WT-5141       Handle multiple modifies for the same key on lookaside eviction
WT-5146       Returning wrong record value for variable length columnar table
WT-5158       Remove assertion in __wt_find_lookaside_upd for cursor comparison
WT-5167       Fixing the missing pieces of page instantiation from las
WT-5170       Convert the os_posix layer to use mmap for I/O instead of system calls
WT-5186       Handle reading of LAS modifies on top of a birthmark record
WT-5191       Avoid LAS sweep server remove records that don't have disk image with proper version
WT-5203       lookaside history should no longer be preferentially evicted
WT-5212       Backup data validation tests
WT-5214       Verify potential incremental failures
WT-5215       Stress testing of incremental backup
WT-5222       Make checkpoint always choose skew newest
WT-5225       Create persistent file for History store
WT-5227       Enable python test cursor04 on relevant history feature branch
WT-5228       Several tests fail with incorrect or missing record after reconciliation changes
WT-5233       Prepared update not visible in las06
WT-5236       Enhance workgen to support draft lookaside stress workload
WT-5240       Update test/format to perform incremental backups
WT-5246       Update WiredTiger backup documentation
WT-5253       Add History store file to checkpoint target list.
WT-5254       Add aggregate timestamp info to the WT_REF.
WT-5256       Fix relevant history build after develop merged brought LIMBO reference
WT-5259       Write wiki documentation on how to generate and use XRay traces
WT-5264       Add a delete record for every insert record into the WT history store to represent stop time pair
WT-5267       Enable storing start/stop time pair for each record in the history store
WT-5271       Remove WT_PAGE_LOOKASIDE structure.
WT-5279       Disable failing tests in the preferred way
WT-5280       Write all other versions to history store during checkpoint.
WT-5281       Assert that we're no longer writing tombstones to the history store
WT-5283       Eviction to create disk image with newest committed version.
WT-5284       Test to check eviction is writing the newest version to data store
WT-5286       Enable the data format to store time pair
WT-5289       Remove the code that does garbage collection from sweep server
WT-5290       Add new verbose flag and verbose logs for checkpoint garbage collection
WT-5291       Additional statistics for checkpoint garbage collection
WT-5292       Test the garbage collection by checkpoint
WT-5295       Disable remaining failing tests in durable history branch
WT-5298       Add a condition to verify the timestamp of a page during checkpoint of history store
WT-5300       Traverse the history store internal page for obsolete child pages by checkpoint
WT-5302       Begin writing validity window for new data format
WT-5303       Update checkpoint statistics for the new history store file.
WT-5308       Remove WT --enable-page-version-ts build configuration option
WT-5317       Eviction writes uncommitted metadata changes to disk
WT-5320       Save update restore to reuse lookaside eviction.
WT-5321       Update the stats and counters that represent lookaside inserts according to durable history
WT-5328       Evict obsolete history store page
WT-5333       Fix search not found issue due to skipped birthmark and rework txn_read
WT-5335       Read the start and stop time pairs back from cell when we reconstruct it on update chain
WT-5341       Clang sanitiser was triggered by checkpoint writing to the history store change
WT-5343       Show history store in the wt list
WT-5345       Show aggregated timestamp information for internal pages and leaf pages
WT-5347       Add ability for wt util to print historical versions of a key
WT-5353       Verify data continuity between history and data store
WT-5354       Verify history store key is not missing from the data store
WT-5355       Add the ability to dump a given btree as of a timestamp
WT-5361       Fix selecting aborted txn to write to disk
WT-5369       Don't use transaction ids from a page with a previous startup generation
WT-5385       Use WT_UPDATE_HISTORY_STORE to avoid repeatedly inserting updates to lookaside
WT-5386       Re-enable test_inmem01
WT-5388       Fix write squash statistic caused by change in reverse delta
WT-5392       Fall out of in-memory obsolete history page eviction
WT-5414       Change the usage from durable timestamp to commit timestamp during eviction.
WT-5416       Make sure that rollback to stable works on all objects in WT
WT-5417       Skip the clean pages of each tree with aggregated stop timestamp is less than stable timestamp
WT-5419       Create a TOMBSTONE update to represent the on-disk value state
WT-5420       Reconcile should consider TOMBSTONE update to replace or remove the on-disk value
WT-5421       Search the history store to find the value that can be replaced the on-disk value
WT-5422       Remove the history updates for reconciled data pages
WT-5423       Convert a history store modified value into a full update
WT-5424       Add statistics for rollback to stable operations
WT-5425       Add a new verbose flag to print the rollback to stable operation
WT-5426       Ensure rollback to stable is executed from recovery
WT-5427       Implement tests for rollback to stable
WT-5431       Merge develop branch to durable history branch
WT-5434       Update compatibility_test_for_mongodb_releases.sh for durable history
WT-5439       Free updates inserted into history store after hs transaction commit in eviction
WT-5441       Fix known visibility issues
WT-5446       Fix not writing to lookaside
WT-5448       Reconciliation wrongly overwrites the cell with default time pairs
WT-5451       Update reverse modify logic to always insert a modify into the history store
WT-5453       Fix cell packing for globally visible values.
WT-5454       Rename cache_overflow configuration option.
WT-5455       Fix test_hs03.py
WT-5457       single block reconciliation with saved updates can leak blocks
WT-5461       Search should not return onpage value if nothing is found by transaction read
WT-5462       Rebalance the Evergreen Python test buckets
WT-5464       Open reading history cursors on the user session
WT-5467       Make history store cursor part of each session, get rid of the cursor pool
WT-5469       Support mixing timestamped and non-timestamped updates
WT-5476       Continue writing tombstones when the tombstone is globally visable.
WT-5478       Insert records directly into history store without a txn
WT-5482       Increment cache usage when appending on-disk value to update list
WT-5484       Check visibility before saving updates for in memory reconciliation
WT-5489       page-read can race with threads locking in-memory page structures
WT-5491       Add an option to wt verify to confirm that no data exists after the stable timestamp
WT-5493       Re-enabling test_bug008
WT-5495       WT-5495 For column store, check the on-page value and history store even if the key is in the insert list
WT-5496       upd should be reset to NULL if all upd on the chain are not visible in txn read
WT-5500       Implement new history store format
WT-5501       Do not use default session to create a history store cursor when configuring
WT-5502       Re-add changes reverted by merge commit
WT-5503       We can only free updates inserted into history store after a full update.
WT-5508       Some of the txnids not cleared after restart
WT-5509       Infinite loop when reading from history store at early specific timestamp in test_util01
WT-5510       Fix test_hs01.py, test_hs06.py (test_hs_prepare_reads)
WT-5513       Don't consider TOMBSTONE/Stop time pair for history store reads
WT-5515       Enable test_hs06
WT-5516       Backup starts with base_write_gen 1
WT-5518       Split-parent code can race with other threads when checking the WT_REF.state
WT-5519       Apply version from datastore after finding a modify when possible.
WT-5522       Remove update free logic in hs_insert_updates.
WT-5523       Inserting history store needs to handle modify based on a tombstone
WT-5525       Free up 3B in the WT_REF structure
WT-5529       Improve usage of upd in txn.i
WT-5540       Call cursor disable bulk insert on first insert to history store.
WT-5541       Use snapshot isolation whenever we use history store cursors in verification
WT-5542       History store not using the on disk value as the base value for modify when key boundary crossed
WT-5547       Disable all the skipped rollback to stable tests for column store types
WT-5549       Fix the recovery rollback to stable and enable the passing tests for both row and variable columnar types
WT-5551       Fix the history store insert statistics
WT-5552       Checkpoint reconciliation and page splits free the WT_REF.addr field without locking
WT-5555       Update base write gen to the maximum write gen we have seen in recover
WT-5556       Verify of a file should verify its history store content too
WT-5558       Use durable timestamp from the update instead of start
WT-5563       Transactions ID are getting wiped which causes errors in WT Verify
WT-5565       Update fast truncate code to consider aggregated timestamps
WT-5566       Update rollback to stable tests to use new statistics
WT-5567       Fix an assert in txn_read always be true
WT-5569       Update WiredTiger source code to include 2020 copyright notices for durable history
WT-5570       Refactor the __upd_alloc_tombstone() according to the new use in durable history
WT-5574       Rolling back a prepared transaction with `cursor_copy` results in a use-after-free
WT-5575       Fix the test_durable_ts01 test to expect older data after recovery
WT-5576       Temporarily add lookaside score stat and cache_overflow config option
WT-5579       Fix Evergreen memory sanitizer test failure
WT-5581       Address sanitizer test failure running bloom filter testing
WT-5582       Long unit testing sweeping cursors failing
WT-5583       Applying operations in recovery encounters unexpected operation type
WT-5587       Limit how many checkpoints are dropped by a subsequent checkpoint
WT-5588       Only release cursor memory allocated by the API for cursor copy debug mode
WT-5589       force_stop on duplicate cursor open not returning error
WT-5590       Fix spellings so s_string passes
WT-5593       test/format assertion failure addr->size != 0
WT-5595       test/format data mismatch errors
WT-5596       Increase format stress testing scope.
WT-5597       Fix the history store file access during recovery
WT-5598       __verify_timestamp_to_pretty_string uses local buffer which is freed before access
WT-5602       Rollback transaction core dumped with  upd->start_ts >= unpack->stop_ts
WT-5603       test/format assertion failure while discarding in-memory page
WT-5605       Update test checkpoint to no longer use checkpoint cursors
WT-5607       Successful 'verify -h' calls are returning not found
WT-5610       Fix assertion for reconciling fixed length column store
WT-5611       Don't write updates with different commit and durable timestamps to data store
WT-5612       Remove history store values for non-timestamp tables
WT-5613       Remove birthmark update type
WT-5615       Coverity: Read of uninitialised value
WT-5616       Coverity: Memory leak
WT-5617       Coverity: Memory leak
WT-5618       Skip timestamp range overlap check if start timestamp is zero
WT-5620       Skip the history store TOMBSTONE only for rollback to stable operation
WT-5624       Incremental unit test should use offset/length ranges
WT-5625       corruption detected during validation - root page's aggregated timestamp incorrect
WT-5626       Remove assert which checks for newer updates in the history store
WT-5628       rollback_to_stable failed with no such file
WT-5630       Adding stage to turtle init to prevent startup on a too-low version
WT-5631       Recovery rollback to stable for timestamped logged tables
WT-5632       Don't write stop_ts of 1 for non-timestamped delete
WT-5633       Fix another assertion for reconciling fixed length column store
WT-5636       prefix compression is slow in the history-store access pattern
WT-5638       Ignore checking visibility of history store updates as they are implicitly committed
WT-5641       Clear history store content when deleting a key due to a globally visible tombstone
WT-5644       Appending onpage value to an aborted update triggers an assert
WT-5647       replace the WT_REF structure's WT_REF_READING state with a flag
WT-5648       Add a leaf or internal page type flag to the WT_REF structure
WT-5649       Refactor WT_REF locking, review all WT_REF.addr reads for locking issues
WT-5650       Fix a race condition between reading the WT_PAGE.modify field and the page being dirtied.
WT-5651       Fix the RTS assert to consider error scenarios of search
WT-5654       Add version information to the history store key format
WT-5658       Fix heap-use-after-free in parent split code
WT-5665       Data mismatch bug when running new version of test_checkpoint with timestamps
WT-5666       Deleting a chunk of the namespace changes the WT_REF type
WT-5667       Remove usage of checkpoint cursor in test/format
WT-5668       Prepare support with durable history: implement data format changes
WT-5669       Prepare support with durable history: backport data format changes to 4.2
WT-5672       Prepare support with durable history: implement rollback_to_stable
WT-5674       Prepare support with durable history: add/debug functional tests
WT-5678       Fix infinite looping behaviour in history store cursor positioning
WT-5680       segfault when dereferencing NULL addr while reconciling
WT-5682       Ensure that we can't apply modifies on top of tombstones
WT-5684       overflow values must be discarded when there is no update for a key
WT-5685       Evergreen does not match libraries in artifacts or spawns
WT-5688       Memory leak detected during page overflow read
WT-5689       reduce work required for the cursor-pinned test.
WT-5690       Rollback to stable assertion failure regarding update visibility
WT-5692       Revert a test change to fix a Python hang
WT-5695       Fixed incremental backup example to use O_CREAT in the backup range case
WT-5696       test_timestamp_abort fails with data mismatch
WT-5698       Disabling bulk cursor changes broke a Jenkins compile
WT-5699       Refactor incremental backup RANGE code
WT-5700       Add smoke test script for incremental backup stress test
WT-5701       If an out-of-order update masks an on-disk value, don't append it
WT-5704       Incremental backup smoke test core dumped
WT-5706       Fix csuite-incr-backup-test calculation of value sizes
WT-5707       Reduce the test load for checkpoint stress test
WT-5712       Ensure WT command line utility treats history store consistently
WT-5713       Fix failures so test_durable_rollback_stable.py can be enabled
WT-5719       Incremental backup metadata should quote the ID string
WT-5721       Remove LAS file when upgrading to 4.4
WT-5722       Incremental backup should do a name check on identifiers
WT-5729       Coverity analysis defect 114078: Unchecked return value from library
WT-5730       Coverity analysis defect 114077: Unchecked return value
WT-5733       Coverity analysis defect 114074: Logically dead code
WT-5734       Coverity analysis defect 114073: PW.MIXED_ENUM_TYPE
WT-5735       Coverity analysis defect 114070: Explicit null dereferenced
WT-5745       Don't copy a value into tombstones
WT-5747       Cope with updates out of timestamp order
WT-5753       Fix divide by zero error in test/csuite/test_incr_backup
WT-5754       Ensure the value is globally visible before encode it into WT_ROW
WT-5756       heap-use-after-free in __wt_row_modify
WT-5762       Make test_hs10 more robust
WT-5767       Fix search_near invocations for history store
WT-5771       make-check-msan-test failed with use-of-uninitialized-value error on RHEL 8.0
WT-5774       Move stress test tasks into a separate build variant
WT-5775       Fix leak of updates from history store
WT-5777       Add statistic for tracking history store deletions due to key removal
WT-5780       Fix timestamp_index_build.js in noPassthrough suite.
WT-5781       Fix basic.js in parallel suite
WT-5785       Fix potential deadlocks in eviction and verify code paths
WT-5786       Detect if a file is too small to read a descriptor block
WT-5789       million-collection-test throughput degradation after loading ~30K collections
WT-5792       Dump and verify can't see the history store
WT-5795       Disable assertion that inserts to the history store are unique
WT-5799       Dont assume we have ordered timestamps when doing rec append original value
WT-5800       Temporarily disable history store verify
WT-5806       Perform rollback to stable on a clean shutdown
WT-5809       Invariant failure: stable timestamp does not equal appliedThrough timestamp
WT-5813       Fix heap-use-after-free in __split_parent
WT-5820       Change format.sh to forcibly quit if a test runs out of disk space.
WT-5822       Don't evict metadata updates from a running checkpoint
WT-5825       Timeout failure in checkpoint-stress-test
WT-5826       Fix a leak when a page splits and some updates aren't restored
WT-5827       Enable test_schema_abort
WT-5828       Reconciliation discards salvaged overflow blocks with stop time pairs
WT-5830       Enable two c tests in evergreen
WT-5831       Fix the case of history store corruption
WT-5834       Incremental backup returning too large offset
WT-5843       Mark page dirty when reconciled page updates are aborted
WT-5844       Adjust error codes for when we detect files that are too small to contain a descriptor block
WT-5855       Clean up operation tracking documentation
WT-5857       Prepend homedir for system call pathnames in format backup test
WT-5858       Fix incremental backups both log/block based for first time
WT-5859       Fix a seg fault in verification
WT-5862       Add conditional around macro definition
WT-5868       Fix test_txn19 because of changing of error message
WT-5872       test/format not writing out isolation level
WT-5874       Disable test_wt2323_join_visiblity
WT-5879       Fix a bug reconciling metadata
WT-5880       test/format backup missing expected log file
WT-5885       Improve WT rollback to stable verbose output for better debugging
WT-5887       Turn off verbose output when test/format output is redirected to a file
WT-5890       Bump log/WT version to facilitate upgrade/downgrade floor for MongoDB
WT-5891       Disable testing of checkpoint cursors in wt4333_handle_locks
WT-5894       Don't persist durable timestamp if it is same as commit timestamp
WT-5896       Recovery sometimes attempts rollback to stable with an absent history store file
WT-5898       Fix memory leak in verifying history store
WT-5899       Restore WT_PAGE_LAS_UPDATE on-disk page flag value to avoid break backward compatibility
WT-5900       Display failed version numbers in error message for version mismatch failures
WT-5901       Fix branch checkout issue in release compatibility tests
WT-5902       Fix history store using incorrect key for lookups
WT-5910       Fix start_durable_ts when handling out-of-order updates
WT-5914       Only configure log-incremental backup if archiving is off in test/format
WT-5921       Fix wtperf_xray to detect builds that haven't been instrumented with XRay
WT-5922       Separate out XRay and xray_to_optrack build instructions
WT-5923       Avoid performing history store cleanup for modified trees
WT-5925       Rework format CONFIG file syntax for clarity
WT-5931       Recovery rollback to stable is leading to remove a record that is not expected

1 files changed, 0 insertions, 110 deletions

diff --git a/src/third_party/wiredtiger/test/suite/test_las04.py b/src/third_party/wiredtiger/test/suite/test_las04.py
deleted file mode 100644
index a9c27876958..00000000000
--- a/src/third_party/wiredtiger/test/suite/test_las04.py
+++ /dev/null
@@ -1,110 +0,0 @@
-#!/usr/bin/env python
-#
-# Public Domain 2014-2020 MongoDB, Inc.
-# Public Domain 2008-2014 WiredTiger, Inc.
-#
-# This is free and unencumbered software released into the public domain.
-#
-# Anyone is free to copy, modify, publish, use, compile, sell, or
-# distribute this software, either in source code form or as a compiled
-# binary, for any purpose, commercial or non-commercial, and by any
-# means.
-#
-# In jurisdictions that recognize copyright laws, the author or authors
-# of this software dedicate any and all copyright interest in the
-# software to the public domain. We make this dedication for the benefit
-# of the public at large and to the detriment of our heirs and
-# successors. We intend this dedication to be an overt act of
-# relinquishment in perpetuity of all present and future rights to this
-# software under copyright law.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
-# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
-# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
-# IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
-# OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
-# ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
-# OTHER DEALINGS IN THE SOFTWARE.
-#
-# test_las04.py
-#   Test file_max configuration and reconfiguration for the lookaside table.
-#
-
-import wiredtiger, wttest
-from wtscenario import make_scenarios
-
-# Taken from src/include/misc.h.
-WT_MB = 1048576
-
-class test_las04(wttest.WiredTigerTestCase):
-    uri = 'table:las_04'
-    in_memory_values = [
-        ('false', dict(in_memory=False)),
-        ('none', dict(in_memory=None)),
-        ('true', dict(in_memory=True))
-    ]
-    init_file_max_values = [
-        ('default', dict(init_file_max=None, init_stat_val=0)),
-        ('non-zero', dict(init_file_max='100MB', init_stat_val=(WT_MB * 100))),
-        ('zero', dict(init_file_max='0', init_stat_val=0))
-    ]
-    reconfig_file_max_values = [
-        ('non-zero', dict(reconfig_file_max='100MB',
-                          reconfig_stat_val=(WT_MB * 100))),
-        ('too-low', dict(reconfig_file_max='99MB', reconfig_stat_val=None)),
-        ('zero', dict(reconfig_file_max='0', reconfig_stat_val=0))
-    ]
-    scenarios = make_scenarios(init_file_max_values, reconfig_file_max_values,
-                               in_memory_values)
-
-    def conn_config(self):
-        config = 'statistics=(fast)'
-        if self.init_file_max is not None:
-            config += ',cache_overflow=(file_max={})'.format(self.init_file_max)
-        if self.in_memory is not None:
-            config += ',in_memory=' + ('true' if self.in_memory else 'false')
-        return config
-
-    def get_stat(self, stat):
-        stat_cursor = self.session.open_cursor('statistics:')
-        val = stat_cursor[stat][2]
-        stat_cursor.close()
-        return val
-
-    def test_las(self):
-        self.session.create(self.uri, 'key_format=S,value_format=S')
-
-        if self.in_memory:
-            # For in-memory configurations, we simply ignore any lookaside
-            # related configuration.
-            self.assertEqual(
-                self.get_stat(wiredtiger.stat.conn.cache_lookaside_ondisk_max),
-                0)
-        else:
-            self.assertEqual(
-                self.get_stat(wiredtiger.stat.conn.cache_lookaside_ondisk_max),
-                self.init_stat_val)
-
-        reconfigure = lambda: self.conn.reconfigure(
-            'cache_overflow=(file_max={})'.format(self.reconfig_file_max))
-
-        # We expect an error when the statistic value is None because the value
-        # is out of range.
-        if self.reconfig_stat_val is None:
-            self.assertRaisesWithMessage(
-                wiredtiger.WiredTigerError, reconfigure, '/below minimum/')
-            return
-
-        reconfigure()
-
-        if self.in_memory:
-            self.assertEqual(
-                self.get_stat(wiredtiger.stat.conn.cache_lookaside_ondisk_max),
-                0)
-        else:
-            self.assertEqual(
-                self.get_stat(wiredtiger.stat.conn.cache_lookaside_ondisk_max),
-                self.reconfig_stat_val)
-
-if __name__ == '__main__':
-    wttest.run()