diff options
| author | Ramon Fernandez <ramon@mongodb.com> | 2016-07-29 20:49:48 -0400 |
|---|---|---|
| committer | Ramon Fernandez <ramon@mongodb.com> | 2016-07-29 20:50:00 -0400 |
| commit | 8c33be6ca2523af862db28157d1b8bfdb3f6028a (patch) | |
| tree | e1ec6ae92226adea23cae0f1d4a4cb0344c89caa /src/third_party | |
| parent | 62d931bf4ba6a4d881e53e10dd176a80d8f3b8b3 (diff) | |
| download | mongo-8c33be6ca2523af862db28157d1b8bfdb3f6028a.tar.gz | |
Import wiredtiger-wiredtiger-2.8.0-601-gdddca65.tar.gz from wiredtiger branch mongodb-3.2
ref: 848e5f5..dddca65
for: 3.2.9
SERVER-24971 Excessive memory held by sessions when application threads do evictions
WT-2711 Change statistics log configuration options
WT-2730 cursor next/prev can return the wrong key/value pair when crossing a page boundary
WT-2760 Fix a bug in backup related to directory sync. Change the filesystem API to make durable the default
WT-2798 Crash vulnerability with nojournal after create during checkpoint
WT-2802 PPC test/format sanitizer heap-use-after free
Diffstat (limited to 'src/third_party')
65 files changed, 854 insertions, 7925 deletions
diff --git a/src/third_party/wiredtiger/NEWS.MONGODB b/src/third_party/wiredtiger/NEWS.MONGODB deleted file mode 100644 index 831237ba149..00000000000 --- a/src/third_party/wiredtiger/NEWS.MONGODB +++ /dev/null @@ -1,2523 +0,0 @@ -3.2.0, Date TBA ------------------- - -commit 9166bca3d07d6592c1426c2c33bd56b6be0667e0 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Thu Apr 23 05:43:35 2015 +0000 - - Fix a deadlock related to handle locking. - - If one thread does a __wt_session_release_btree of a handle at the same time - as another thread does a __wt_session_get_btree both wanting exclusive access - to the file. It was possible for one thread to get the dhandle list lock and - wait on the handle lock, which another thread held the handle lock waiting for - the handle list lock. - - Temporarily fix by doing a try-lock on the __wt_session_get_btree path, long term - the solution is to get rid of the __conn_dhandle_open_lock method, and split get - and lock into two passes. - - Refs BF-716 - -commit 3e254079484ce35a3cb70c48478c69defdb8f012 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Thu Apr 23 05:42:08 2015 +0000 - - Fix a deadlock related to LSM. There are cases where closing a file with - an existing checkpoint could self deadlock. - - Check in the meta tracking whether we've already visited a checkpoint handle. - - Refs WT-716 - -commit 1e80654b284b47b2dd9c302395ba908bf3a9b898 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Apr 22 13:54:03 2015 -0400 - - __wt_config_concat() is a special case of __wt_config_merge(), replace - the three calls to __wt_config_concat() with __wt_config_merge(). - -commit 1c412df22489dc3c18aa5390164ff68474293daf -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Apr 22 13:29:05 2015 -0400 - - Instead of doing a configuration line merge to remove configuration - values we don't included, followed by a call to concatenate additional - configuration values, do a single merge call including all of the - configuration values we care about. - - This change is slightly more defensive -- if a user tries to change one - of the configuration values we don't allow, we'll strip it, rather than - failing the load. - - Clean up memory handling a bit so we don't leak memory, someday Coverity - will thank me. - -commit 6093e42b21d7e947d7bd053f6691aa1fed1a7f99 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Apr 22 11:33:21 2015 -0400 - - Instead of rolling our own configuration removal code, use - __wt_config_merge(), reference WT-1898. - -commit 4322191125284717af1a0c6734b6ea123ca6c50d -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Tue Apr 21 15:45:30 2015 -0400 - - Run recovery earlier. WT-1897 - -commit 6155c465a519612e16cec5feb9fcf79fd0121d7f -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Tue Apr 21 13:15:06 2015 +1000 - - Revert a change to split large in-memory pages. - - Refs WT-1890 WT-1896 - -commit 610f629949726b16f938ded85188bb6a21820f7e -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Apr 20 10:40:54 2015 -0400 - - Create a "clear a single walk" function and call it from __evict_clear_walks - and __evict_clear_all_walks, that way we can use the WT_WITH_DHANDLE macro, - and the comment about clearing the eviction reference before releasing the - page appears everywhere it should. - -commit 6ea949933ff9e68d220738d18fa72eb7a91aab65 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Mon Apr 20 16:50:13 2015 +1000 - - Fix coverity reports in test/format code. - - CID 1295092: Integer handling issues (OVERFLOW_BEFORE_WIDEN) - /test/format/ops.c: 93 - CID 1295091: Null pointer dereferences (REVERSE_INULL) - /test/format/ops.c: 489 - -commit 3eceb85ce623dcce9273f7b722bb64f509dbe24d -Author: Michael Cahill <michael.cahill@mongodb.com> -Date: Mon Apr 20 16:15:41 2015 +1000 - - Clear session->dhandle so that future error messages don't dereference freed memory. - -commit 23ce8bae4d6507d6b6d599cb73a26a6c856cce98 -Author: Michael Cahill <michael.cahill@mongodb.com> -Date: Mon Apr 20 15:48:29 2015 +1000 - - Clear eviction walks in all trees before the eviction server exits. - -commit a4bce0e0bc05d528f118b645d0d1915db00cdcf3 -Author: Michael Cahill <michael.cahill@mongodb.com> -Date: Mon Apr 20 14:11:57 2015 +1000 - - Move the "cache is empty" check to where the cache is destoyed, not when the eviction server is exiting. - -commit c8fdd9c676c4a24bee6328a56cf7fd074cd045e0 -Author: Michael Cahill <michael.cahill@mongodb.com> -Date: Mon Apr 20 13:25:36 2015 +1000 - - Shut down the eviction server before closing file handles to avoid a race. - - refs WT-1893 - -commit fb4a089fa71876232478e1181d821f29dedc0bd8 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Sun Apr 19 12:43:47 2015 -0400 - - Add the version to the configuration stack before reading the base - configuration file: that way it's always set, but it's always set - to the value of the base configuration file. - -commit 7aee6d5dabb7942aeb685e797b103c47c9337186 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Sun Apr 19 11:56:22 2015 -0400 - - Add support to the __wt_config_merge() call to remove strings we don't - want in the final configuration string, then strip out "create=" and - "encryption=(secretkey=)". - -commit 071d545f47ff4e4badcf9b8c066a44dac7fa2f20 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Fri Apr 17 11:39:23 2015 -0400 - - Instead of explicitly writing the version into the base configuration - file, append it to the configuration list. - -commit ab5443aa13d62c71423c128853735f699935c01e -Author: Keith Bostic <keith@wiredtiger.com> -Date: Fri Apr 17 11:35:13 2015 -0400 - - There's no reason to loop through each application-specified configuration - string separately, there's a call to check them all for the same key. - - This also prevents a value being listed multiple times. Previously, if - you set buffer_alignment in the wiredtiger_open() configuration string, - but also in the WIREDTIGER_CONFIG environment variable, it would appear - twice in the base configuration file, with this change, it only appears - once. - -commit abb0bb80cc6dce29b8db61c6747c228c2701ae5a -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Fri Apr 17 10:49:41 2015 -0400 - - Look for any number of non-data-changing log records to determine if we - can skip recovery. WT-1892 - -commit e7f4ecd2055bab4d683eae119f8da95fa7acf21f -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Fri Apr 17 04:32:52 2015 +0000 - - Update API documentation to explain session usage. - - This allows users to account for internal WiredTiger session handle - usage. We already allocate additional handles for those we use - unconditionally. This allows users to do specific calculations based - on their session_max setting. - -commit ee02428d1fdf1118c482688ec870a9da69bee45a -Author: Michael Cahill <michael.cahill@mongodb.com> -Date: Fri Apr 17 12:28:52 2015 +1000 - - If an LSM search-near operation lands on a deleted item, make a copy of the key before stepping to the next record. - - refs WT-1891 - -commit 54e856d57da291c5f84da6d0d0ab56280d9956dc -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Thu Apr 16 12:22:31 2015 -0400 - - Remove use of unneeded tmp_fh. WT-1872 - -commit 6a32905c397e57643b15e5a3038dbcb99a8a8dc8 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Thu Apr 16 06:18:31 2015 +0000 - - Fix a deadlock in LSM with schema operations. - - There is special code in LSM to co-ordinate schema operations on - tables (drop, rename, etc). The code does a dance dropping and - acquiring locks, to allow utility operations to drain for the tree - while waiting for it to close. - - We were doing the dance with the schema and dhandle list locks. We - needed to include the table lock, or parallel cursor opens could block: - - The cursor open is waiting for the table lock: - __wt_spin_lock src/include/mutex.i:175 - __schema_add_table src/schema/schema_list.c:26 - __wt_schema_get_table src/schema/schema_list.c:98 - __wt_curtable_open src/third_party/wiredtiger/src/cursor/cur_table.c:875 - __wt_open_cursor src/session/session_api.c:240 - - The LSM table drop is waiting for the schema lock: - __wt_spin_lock src/include/mutex.i:175 - __lsm_tree_close src/lsm/lsm_tree.c:107 - __wt_lsm_tree_drop src/lsm/lsm_tree.c:943 - __wt_schema_drop src/schema/schema_drop.c:174 - __drop_table src/schema/schema_drop.c:124 - __wt_schema_drop src/schema/schema_drop.c:176 - __session_drop src/session/session_api.c:528 - -commit 790646183cc5dd056bbf95c4563c20c51602a808 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Thu Apr 16 04:11:36 2015 +0000 - - Fix a bug in LSM where updates with overwrite could be skipped. - - References JIRA BF-829 - - The issue was that we were not looking in all chunks of an LSM - tree before deciding whether to apply an update (insert or remove). - -commit 72ccd267fea9e491fcf3506e85191f71471cf51a -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Apr 15 09:44:13 2015 -0400 - - A WT_CURSOR.next operation followed by a WT_CURSOR.search has the same - problem as referenced in #1887: the key returned to the application is - in WT_CURSOR_BTREE.tmp, and that WT_ITEM is used as temporary storage - during the search of a row-store leaf page, so the search can overwrite - the search key while it's still in use. - - Change WT_CURSOR.next to return the ey in WT_CURSOR_BTREE.search_key. - - Rename WT_CURSOR_BTREE.search_key to be WT_CURSOR_BTREE.row_key, it's - exclusive to row-store, and no longer exclusive to search. - -commit 50f8bedf616a4009068516df865374f688c76c70 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Tue Apr 14 12:52:32 2015 -0400 - - When row-search builds keys during leaf-page search, the last key built - is left in WT_CURSOR_BTREE.search_key, and if it's an exact match, that - buffer's contents are returned to the application (or, in the case of - cursor, the search key can be overwritten during the search, that is, - the searched-for-key, and the temporary buffer where we're building the - on-page keys for comparison, are the same. - - Use the WT_CURSOR_BTREE.tmp buffer during the row-search instead of - WT_CURSOR_BTREE.search_key, and set WT_CURSOR_BTREE.search_key to the - returned key only when we've found an exact match and are returning to - our caller. - - Making WT_CURSOR_BTREE.search_key and WT_CURSOR_BTREE.tmp pointers so - it's easy to swap back-and-forth makes this change noisy; note the new - __wt_btcur_open() function to set things up when a Btree cursor is first - created. - - Reference #1887. - -commit 4cf1871ca6770b035a3b30aca29fa89dc11bfc18 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Tue Apr 14 06:30:41 2015 +0000 - - Make test/thread use WT_TEST not cwd, and add some more logging. - -commit b9e2e76511a24505014369aaf0e1ec286e9c473d -Author: Michael Cahill <michael.cahill@mongodb.com> -Date: Tue Apr 14 16:30:31 2015 +1000 - - Merge bulk cursor close with regular file cursor close: we were missing a decrement that kept bulk-loaded files pinned. - -commit a9e6a51f4ace1da5c73dd54c108d9a764fc5d8a4 -Author: Michael Cahill <michael.cahill@mongodb.com> -Date: Tue Apr 14 14:37:12 2015 +1000 - - Fix some 64-to-32-bit conversion warnings. - - include/cell.i|652 col 20| error: implicit conversion loses integer precision: 'uint64_t' (aka 'unsigned long long') to 'uint32_t' (aka 'unsigned int') - copy.v = unpack->v; - - include/cell.i|705 col 24| error: implicit conversion loses integer precision: 'uint64_t' (aka 'unsigned long long') to 'uint32_t' (aka 'unsigned int') - copy.v = unpack->v; - - reconcile/rec_write.c|2683 col 17| error: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'uint32_t' (aka 'unsigned int') - tmp->size = ((u_int)(28 + (btree)->block_header)) + len; - -commit 4528ddaec1d4b3382055e5c1c53fb7b1772133a2 -Author: Don Anderson <dda@ddanderson.com> -Date: Mon Apr 13 15:05:16 2015 -0400 - - Generated tables for config subcategories now contains the name of the - method they are applicable for. Refs #1879. - -commit 1ac393ca9adce1f6d5e4bec035e7c49b32e5a722 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Apr 13 13:00:57 2015 -0400 - - We can't pass the reconfiguration functions the base configuration array - because repeated calls will cause us to fallback to the default value. - - For example: - - conn->reconfigure(conn, "file_manager=(close_handle_minimum=37)"); - conn->reconfigure(conn, "file_manager=(close_handle_minimum=38)"); - conn->reconfigure(conn, "file_manager=(close_handle_minimum=39)"); - conn->reconfigure(conn, "eviction=(threads_max=10)"); - - The reconfigure call for eviction will reset close_handle_minimum back - to its default value, because it will find the base value, not the - reconfigured value. - - Try and make sure we don't mess this up again, use "cfg" instead of a - separately constructed local variable. - -commit ac37b924ac18f15726d3bd1984c61e89fbdd405e -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Apr 13 07:17:18 2015 -0400 - - Search the strings in reverse order, that way the first hit wins - and we don't search the base set until there's no other choice. - -commit 110164271dc688c90a092c0c95d37feed2cec188 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Mon Apr 13 05:48:19 2015 +0000 - - Add functionality to drop files at the end of a wtperf run. - - Helps figure out how long a drop takes. - -commit ed3158e71f0bd2716269a5771fd162b60b9a1cc0 -Author: daveh86 <howsdav@gmail.com> -Date: Mon Apr 13 12:59:29 2015 +1000 - - Allow forced eviction of pages already queued for eviction - -commit 9c83351f63afc2e032e492e3030df4f3b1cd6883 -Author: Mark Benvenuto <mark.benvenuto@mongodb.com> -Date: Sun Apr 12 19:02:32 2015 -0400 - - Disable fallocate on Windows since SetEndofFile does not - ignore truncation requests like POSIX fallocate. - -commit 61a7d81ad26db1f2bfb65258d9b8ae4a4ca25b34 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Sun Apr 12 12:44:05 2015 -0400 - - When using ftruncate as the file-extension call, we must use WT_FH.size - as the starting point of the extension (not offset), and we have to read - the size value after acquiring the lock that prevents racing with writers. - - Split the extension functionality out into a separate function and - try to make it a little simpler to understand. - - Reference #1871. - -commit f26f1c1c59d5cbbc8f5f543215d8fc636e7175d2 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Sun Apr 12 10:16:34 2015 -0400 - - The OS layer ftruncate() code sets the WT_FH file size, so when - ftruncate is used to extend the file, we skip over the bytes added to - the file during block allocation, and verify eventually fails because - there are unverified blocks in the file. - - Reference #1871. - -commit c27c201de9c766aea5249d3eeb85b8103ea6cefc -Author: Keith Bostic <keith@wiredtiger.com> -Date: Sat Apr 11 09:16:11 2015 -0400 - - Possible approach to avoiding lockout when when sweeping files: before - closing the file, flush the file from the cache. - - Add a new cache-operation, WT_SYNC_CLOSE_SWEEP, that walks the cache for - a handle and discards any clean pages it finds. - - Add call to __wt_txn_update_oldest() before checking if a handle can be - swept. - -commit d20f20f1ac324030986b3ee23e1edf96486c92b4 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Fri Apr 10 05:59:04 2015 +0000 - - Update file_manager=(close_idle_time=,close_scan_interval=) max values. - - The old maximum for both was 1000, the new value is 100 thousand. Setting - such large values is not recommended, but there is no internal limitation - on them. - -commit c36a3308f685d3b85efe9ac6ee0835f0974574b4 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Apr 9 14:32:40 2015 -0400 - - Don't ignore sweeping entirely when we reach the open-file-count limit, - just ignore the in-use files. - -commit 46ef2555bbc51ce6453536e72202782be4949855 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Apr 9 11:06:50 2015 -0400 - - Don't keep sweeping once we reach the minimum number of handles. - -commit 1fdfcc62726d25a7cceeeefff174a8e1bf9f9e67 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Apr 9 12:39:03 2015 -0400 - - const: At condition ret == -1, the value of ret must be equal to -1. - CID 72082 (#1 of 1): Redundant test (DEADCODE) - dead_error_condition: The condition ret == -1 must be true. - -commit d04e3c25d46a5c4426e1c6d4881cd9e250014931 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Apr 9 12:19:24 2015 -0400 - - Remove unnecessary atomic operation, fixing CID 69810 along the way. - - CID 69810 (#1 of 1): Parse warning (PW.CONVERSION_TO_POINTER_ADDS_BITS) - 1. conversion_to_pointer_adds_bits: conversion from smaller integer to pointer - -commit d585bdab980508e590cf70508f053182c556d6f2 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Apr 9 12:00:04 2015 -0400 - - CID 72073 (#1 of 1): Redundant test (DEADCODE) - dead_error_condition: The condition session != NULL must be true. - -commit 21907f9193e30f51a59fcbaddfbc46cb7732d3b7 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Apr 9 11:52:13 2015 -0400 - - Remove unnecessary error labels, fixing SERVER-17948 along the way. - - Coverity analysis defect 72088: Redundant test: - File: /src/third_party/wiredtiger/src/log/log.c - Function __log_decompress - /src/third_party/wiredtiger/src/log/log.c, line: 363 - At condition "ret != 0", the value of "ret" must be equal to 0. - -commit a29f4d2f40eee784950147af848fdbf277328b7f -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Apr 9 11:31:11 2015 -0400 - - SERVER-17954: Coverity analysis defect 72085: Redundant test - At condition "ret != 0", the value of "ret" cannot be equal to 0. - -commit 1298fd6db4f0c1d6133b3e547b2e2db51ec4a708 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Apr 8 14:38:49 2015 -0400 - - Fix places where we were using the wrong link for traversing hash buckets. - -commit 7d0e5fe3dfa39f7ff377a1d4660bc2bc36dc0df8 -Author: Mark Benvenuto <mark.benvenuto@mongodb.com> -Date: Wed Apr 8 10:19:33 2015 -0400 - - Enable test/fops for Windows, and add to CI - - - Added Windows shim for gettimeofday - -commit c6270b677499525067d5d729a6fbdce6ad2f533a -Author: Mark Benvenuto <mark.benvenuto@mongodb.com> -Date: Wed Apr 8 13:28:41 2015 -0400 - - test/format for Windows - - Fixed an issue where fallocate was setting fh->size (incorrectly copied from ftruncate implementation) - -commit fac74b4665d6dfa3aebecf741c914fd1678fa628 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Apr 8 08:47:06 2015 -0400 - - Rename file_manager.open_handles to file_manager.close_handle_minimum, - try and match existing naming for file_manager configuration options. - -commit 709cc8d7ac85d31aeae2387b192092910e6cf854 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Apr 8 08:09:23 2015 -0400 - - Don't sweep unless there's a reason we need to close file handles. - Add a new configuration option, file_manager.open_handles that sets - a minimum number of file handles that must be open before sweep runs, - default is 250. Reference #1856, SERVER-17907. - -commit b59dfec2609847bb22bb48f4e7eede8c45312ce9 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Apr 8 07:58:49 2015 -0400 - - Fix WT_STAT_ATOMIC_INCR, it didn't get upgraded to the new atomic - macros needed for the Windows port. - -commit 9f9fbb19ba19f90e4fc52d7568bd66427edb31e6 -Author: Pat Gunn <pgunn@mongodb.com> -Date: Tue Apr 7 22:56:55 2015 -0400 - - Add a Python ex_stat example - -commit b79dcdebf0b1987b59fa70e50c8c61e5e0a64e64 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Tue Apr 7 11:52:12 2015 -0400 - - If the underlying split buffer grows, existing boundary references may - point into freed memory, switch the boundary "first byte" field from a - pointer to a buffer offset to avoid the problem, reference #1852. - -commit 27b37db17e70ef73432e1fb48c530246fd753670 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Apr 6 07:27:25 2015 -0400 - - Fix a broken line, #1846. - -commit dbb58f0b8f7b26a70f71f6c67dba93c966b162e0 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Sun Apr 5 12:24:31 2015 -0400 - - Search the currently pinned page first in WT_CURSOR::search-near, - before descending the tree from the root. - -commit 353093e16eb4a955c4f2e2f4245577cd9156955f -Author: Keith Bostic <keith@wiredtiger.com> -Date: Sun Apr 5 08:36:57 2015 -0400 - - WT_CURSOR::search() near current position. - -commit 96022e89162f746d8252db637a0305aed1965cd1 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Sat Apr 4 10:09:17 2015 -0400 - - Inside wiredtiger_open(), we may have allocated scratch memory when using - the dummy session or the subsequently created real session, and we don't - want to tie down memory for the rest of the run in either of them. - -commit 928409dbb64e222f722c5f6692f1d7638ce9e617 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Fri Apr 3 20:37:45 2015 -0400 - - Use scratch buffers in wiredtiger_open(), but clear them out when we're - done (we have no idea what other functions are using scratch buffers when - called via wiredtiger_open(), so just make them work). - -commit d7e2351db8b855af3b5b8860e000528ac99d57a4 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Apr 2 19:44:16 2015 -0400 - - Windows doesn't have <x86intrin.h>, it does have <intrin.h> which is - included by <msvc.h>. - -commit d3dacbffd2a87ea79ec05fa569bdd5d34f90254c -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Apr 2 13:06:19 2015 -0400 - - Make --with-spinlock=pthread_logging option compile again. - - Remove the WT_SESSION_IMPL argument to the fprintf, vfprintf, fflush and - fclose functions, there are places we want to use it that don't have - session handles, and it's not currently needed. Clean up error handling - in the vfprintf function. - -commit fccb479aa9b97fb22d9ec2827c94ba49faa5ab79 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Apr 2 09:58:52 2015 -0400 - - Gcc 4.1 can't figure out that copy.v can't be used uninitialized. - Inline a length check in the short-key/value case to avoid it. - -commit dca44d7f2762052bf675a5edc0a2f63534c85cf1 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Apr 2 08:55:41 2015 -0400 - - Coverity 44262 (#1 of 1): Parse warning (PW.MISSING_INITIALIZER_ON_CONST) - 1. missing_initializer_on_const: const variable "__clear" requires an - initializer. - - This isn't a bug, but we only use WT_CLEAR_INLINE() in a single place - in WiredTiger, and we can probably do better by not clearing the unpack - structure at all. - -commit 0b60cfdfa3912d3048e3c5dbce56db7745c20964 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Apr 1 17:43:31 2015 -0400 - - Do a binary search of the base configuration options when checking - configuration at the API layer instead of a linear walk. - -commit 078cb46916b55c254abe1a966a2413410a5f6174 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Apr 1 15:57:04 2015 -0400 - - Use the existence of the <x86intrin.h> #include file to configure the - x86 vector instructions. Some old versions of gcc don't have it, but I - believe they're old enough that we don't care. - -commit 684fd71475cbc6b15290945af0160fac0313ad6b -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Wed Apr 1 15:44:42 2015 -0400 - - Reset eol if we continue. Return NOTFOUND if checksum mismatch. #1840 - -commit 507a3005b3bc4f9cc69153da5acb534702b734b8 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Apr 1 09:40:36 2015 -0400 - - Don't #include <x86intrin.h>, gcc 4.1.2 doesn't have it, use - <emmintrin.h> instead. - -commit 9c29e0f13268c03038704372c069353c81357791 -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Tue Mar 31 16:49:47 2015 -0400 - - Modify log_scan callback args to send in next LSN. #1837 - -commit 76cba586685828fc3a8598b48e70c7614818859c -Author: Keith Bostic <keith@wiredtiger.com> -Date: Tue Mar 31 12:11:04 2015 -0400 - - Coverity 1199719 (#1 of 1): Dereference after null check (FORWARD_NULL) - var_deref_op: Dereferencing null pointer "ref". - -commit f9edf738c6075601bb2885cd2aba3ea4a0134b5e -Author: Keith Bostic <keith@wiredtiger.com> -Date: Tue Mar 31 12:00:50 2015 -0400 - - Coverty 1129071 (#1 of 1): Dereference before null check (REVERSE_INULL) - check_after_deref: Null-checking "conn" suggests that it may be null, - but it has already been dereferenced on all paths leading to the check. - -commit 4317a14ffead9029c6524c3a5013e1d91b2f0a02 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Tue Mar 31 11:47:50 2015 -0400 - - Coverity 1129018, 1129019, 1129020, 1129021: Side effect in assertion - (ASSERT_SIDE_EFFECT). - -commit a5bb492b41e4121ff69776ba70072585aef405af -Author: Keith Bostic <keith@wiredtiger.com> -Date: Tue Mar 31 07:42:01 2015 -0400 - - When looking for the next-larger item (__col_insert_search_gt), stay - at the same level if the checked record is equal to the searched for - record (can't happen in the current use of this code, but it's the - correct thing to do in a skiplist). - - When looking for the next-smaller item (__col_insert_search_lt), the - search still has to be greater-than, reference #1835. - -commit f2055cab87688cbb26da659c8038dbb4f032eb30 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Tue Mar 31 07:41:40 2015 -0400 - - Add a __setitem__ to cursors in the Python API. - -commit 674170067c4016455e78d14ec24e3641d047f1c4 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Tue Mar 31 14:26:00 2015 +1100 - - Add a __setitem__ to cursors in the Python API, remove lots of boilerplate c.set_key ... c.set_value ... c.insert code. - -commit 4790f13cd8191d834f40c97a0c00d729f91acd1b -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Mar 30 09:52:34 2015 -0400 - - async_max_queue: collection statistic, aggregate doesn't apply, don't - clear. - - cache_eviction_maximum_page_size: collection statistic, aggregate - doesn't apply, don't clear. - - various txn_checkpoint_XXX statistics: collection statistic, aggregate - doesn't apply - - various Btree maximum size statistics: aggregate the maximum value, - rather than doing no aggregation at all. - - lsm_chunk_count: aggregate the count of LSM chunks. - - Add syntax checking code to stat.py to disallow aggregation flags - for connection-level statistics. - - Reference #1742. - -commit 0c9f1341e2fdb93d3bd4d3fc58176f6ad169825e -Author: Keith Bostic <keith@wiredtiger.com> -Date: Sat Mar 28 13:15:53 2015 -0400 - - When we find a record in the slot's update skiplist, but then want to - jump past the rest of the deleted records, we have to adjust based on - the starting record of the slot, use the page's repeat array to find - that starting record. - - Another run at the __col_insert_search_gt (the greater-than skiplist - search), hopefully it's finally correct. - -commit b5edc28c2588a0257daf50f43db7b5bf335c7ef5 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Fri Mar 27 14:40:36 2015 -0400 - - Cleanup, fix, simplify the cursor-insert-greater-than search, add a - corresponding less-than search and hook it into the WT_CURSOR.prev - function. This runs better than before, but there are still problems. - -commit c7cdb2e1f1960bc9432185df8fb7b507198889ff -Author: Keith Bostic <keith@wiredtiger.com> -Date: Fri Mar 27 10:38:17 2015 -0400 - - The gaps in column-store tables can be large enough we spend too much - time looping through the "deleted" records when writing out the page, - skip the boring part. Reference #1807. - -commit 488d064b45fb194ea2f3b9e2187214fb44b86a5f -Author: Keith Bostic <keith@wiredtiger.com> -Date: Fri Mar 27 10:07:16 2015 -0400 - - There can be huge gaps in the variable-length column-store name space - appearing as deleted records. If more than one deleted record, do the - work of finding the next useful record. Reference #1807. - -commit e1ee6432fb5a948ebd5cafe2ba9c2b79411458f5 -Author: Mark Benvenuto <mark.benvenuto@mongodb.com> -Date: Thu Mar 26 11:31:27 2015 -0400 - - Revert "Make LZ4 default off in scons builds for now." - - This reverts commit 67b71e1eaad1a5825f99a415d9851976f11dbfee. - -commit 003e6c3598408c7670f65a8720622c38fdaf148d -Author: Mark Benvenuto <mark.benvenuto@mongodb.com> -Date: Thu Mar 26 11:17:57 2015 -0400 - - Use Standard C type uint64_t for zlib - -commit e12b9e0b005da7364330f4d3409256ded26ba90d -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Mar 26 18:31:15 2015 -0400 - - Add checks for Windows builds: _M_AMD64 and _MSC_VER. - -commit 1b1371c83e3e78feab0921c88a63d960288d58e0 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Mar 26 11:09:28 2015 -0400 - - __wt_btree_size_overflow uses the page's WT_PAGE_INDEX, wrap the call - in WT_ENTER_PAGE_INDEX/WT_LEAVE_PAGE_INDEX. - -commit e4c9309756f8bcda4bd7b9be5232887cdea377be -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Mar 26 10:57:33 2015 -0400 - - Instead of asserting split-generation is set in WT_INTL_FOREACH_BEGIN, - move the test into the WT_INTL_INDEX_COPY macro (which is called by - WT_INTL_FOREACH_BEGIN), WT_INTL_INDEX_COPY is the code that reads the - actual WT_PAGE_INDEX value: if a page can split, it's not safe to look - at anything a WT_PAGE_INDEX references, and we have code that uses a - WT_PAGE_INDEX reference outside of the WT_INTL_FOREACH_BEGIN macro. - - Add two versions of the WT_INTL_INDEX_COPY macro, WT_INTL_INDEX_GET - which checks the split-generation, and WT_INTL_INDEX_GET_SAFE which - doesn't. - -commit 8bdb29fed9ef4a124e8c1c3c1a6c031f4ecbe130 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Mar 26 09:34:19 2015 -0400 - - Remove safe version of the WT_INTL_FOREACH_BEGIN macro (it was only used - in a single non-DIAGNOSTIC code path), wrap child page-index walk during - a deepen split with WT_ENTER_PAGE_INDEX/WT_LEAVE_PAGE_INDEX, reference - issue #1799. - -commit 87326b721a443936e118e72e58e4f51e14845132 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Mar 25 21:44:02 2015 -0400 - - Use the size of the vector chunk to decide if we'll execute vector - instructions, rather than some specific length. - -commit fc4eea6e3eb39cdf1b147d6a7b65af4f0d08f5c1 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Mar 25 18:20:23 2015 -0400 - - Instead of using _mm_cmplt_epi8 when we find a mismatch, fall into the - slow comparison code. - -commit a29b65429f2963bb3e3ff8ca4416f16ae8ca2e52 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Mar 25 16:28:34 2015 -0400 - - Cleanup #includes so the whole tree builds. - -commit 35b436dfd47895cc19310095f4f4e5f3b9b86501 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Mar 23 17:51:50 2015 -0400 - - We don't have to reinitialize userp and treep when falling into the - slow/remainder loop, they're correctly positioned. - -commit a307a61135394d5006bd3257325bffd82ea80f21 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Mar 23 17:14:30 2015 -0400 - - Don't call _mm_cmplt_epi8() until we don't compare equal. - -commit a609c82bfa6fbf80cd5fb853e1d97c16878a0180 -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Mon Mar 23 15:25:23 2015 -0400 - - Set checkpoint LSN to existing log record. #1700 - -commit 076336e9b244664440777cfeab9fe0d925d4c25e -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Mar 23 10:13:37 2015 -0400 - - Integrate Sasha's vectorized search code. - - -3.0.2, Mar 26 2015 ------------------- - -commit 17bd2356a5b17893e626749be399fb8fda23db1e -Author: Mark Benvenuto <mark.benvenuto@mongodb.com> -Date: Fri Mar 20 01:07:59 2015 -0400 - - Use beginthreadex, and ensure we use the correct C calling conventions in all callbacks - -commit 269e847ad64dd12dfcadb58f84f905069e5b8dce -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Wed Mar 25 15:47:46 2015 +1100 - - Split sweep into two passes: one that walks the handle list without locking and finds candidate handles to close, and a second pass holding the handle list lock that removes dead handles. - - refs #1814, #1811, #1808 - -commit 065a435f6b1d1b3fcb640d59c3109d0c2e24d308 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Tue Mar 24 16:53:28 2015 +1100 - - Add statistics to track internal/overflow/leaf pages in cache. - - It is often useful to know if a cache is full of internal pages. I could have added page count tracking, but byte count tracking seems more useful (enough to justify the extra performance overhead). - -commit f4616895f1ac35aaa50fc1a68e0c6dd9e0cf7717 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Mon Mar 23 17:00:15 2015 +1100 - - If a walk for pages to evict ends on a page requiring forced eviction, don't keep it pinned. - - refs SERVER-16662, SERVER-17382 - -commit 49ddcca712db3a50c390f48b518f0835f28bc9d4 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Sat Mar 21 16:17:51 2015 -0400 - - Separate out the code to flush a file handle and rename the file into place. - -commit a0e88c41781c5b7ea0155fb57a58c91a964af4a3 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Fri Mar 20 19:08:45 2015 -0400 - - If the underlying FILE was opened for writing, flush and fsync it during close. - -commit 8d918f0ef8588056ecf729e72ffdd8bc0a79fd6c -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Fri Mar 20 22:54:13 2015 +1100 - - Fix a race closing eviction helper threads: close the main eviction thread first, so the number of helper threads can be read safely. - - Partial fix for #1698 - -commit 294b0bce296bebf9790418e5575f59be5bec000c -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Fri Mar 20 17:19:28 2015 +1100 - - Only align buffers on Linux if direct I/O is configured. Clarify why we round up the allocation size: it usually won't matter because we allocate multiples of the alignment size anyway. - -commit ee7456799277e19ff4771de79ac4eafac43aff9b -Author: Mark Benvenuto <mark.benvenuto@mongodb.com> -Date: Thu Mar 19 16:16:15 2015 -0400 - - Visual Studio 2015 Preview support - -commit 9c60b462a25a36915207dde240579c5aa673f2c7 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Fri Mar 20 00:09:42 2015 +0000 - - Update the eviction server to not set stuck if it isn't full. - - When only looking for pages to force out, the cache isn't really stuck. The trouble is that if we set stuck the eviction server doesn't clear walks which can lead to it always holding a reference t othe page we (really) want to evict. - - refs #1777 - -commit f0138353697d8706bd1f26e83b9fff8f4e83af8d -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Mar 19 13:04:17 2015 -0400 - - Create the base configuration file in a temporary file and rename it into place so a crash can't result in a corrupted base configuration file. - - refs #1775, #1776, SERVER-17571 - -commit 2e15cd6fc322c90c763394c52056bfebd4153aad -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Thu Mar 19 10:25:31 2015 +1100 - - Avoid EBUSY errors from truncate caused by checkpoints. - - refs #1643 - -commit 3188352d623de85803db3dd6e5b5188822e2db4f -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Wed Mar 18 13:00:53 2015 +1100 - - If the session-level attempt to lock a handle gets EBUSY, fall back to the slow path. The sweep server relies on there being a retry loop if an exclusive operation such as verify conflicts with a sweep. - - refs #1404 - -commit 5d8641568ac27e21d6671cb646a818708eb0aa28 -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Tue Mar 17 15:38:57 2015 -0400 - - Set the end of the log if we find a zero hole. - - refs #1766, SERVER-17569, SERVER-17613 - -commit f38e325f87d94f178c932328632857361fc3c92c -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Tue Mar 17 04:30:11 2015 +0000 - - Implement review feedback. The new configuration strings are: - - file_manager=(close_idle_time=30,close_scan_interval=10) - - Update the sweep test to take advantage of new configurations and - shorten the runtime. - -commit 87dba78b65bc869c426363d86d02134aa8f8f2ac -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Tue Mar 17 14:39:49 2015 +1100 - - Round up the "in-memory size" for WT_UPDATEs to a multiple of 32. They are a very common case, and for tiny updates, this seems to to a better job of matching tcmalloc's behavior at least. - - refs SERVER-17424 - -commit d277a088385401ff6e6664dd688a2fcab2ac4087 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Mar 16 11:42:56 2015 -0400 - - Sanity check all splits for at least 100 keys on the page, if there are huge keys and a too-small cache, there's nothing to be done. - -commit d4aa136a20c9baf151c3185058ce089679e0de0e -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Mon Mar 16 06:10:12 2015 +0000 - - Ensure we free memory when statistics cursor open fails. - - refs #1760 - -commit 66622ac33357d53212ddfa362fdf3c6b439bf34b -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Mon Mar 16 05:55:45 2015 +0000 - - Update the split deepen sanity check. Allow a page that is using more than 1/4 of the cache to be split. - - refs #1759 - -commit f99114f785985d152ba8ddfa735771574e6bff5d -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Mon Mar 16 16:43:27 2015 +1100 - - Fix the search for a free hazard pointer slot. Previously, it gave up early once all slots were allocated, even if they weren't all in use. - -commit 15fe04460c1f057811e0f3e02feb249c710d05ce -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Mon Mar 16 16:40:49 2015 +1100 - - Track the number of hazard pointers active in the eviction server. Don't start a new walk if we are close to the limit. - - refs SERVER-17551 - -commit e6d8c8a56c4dc83f206d43fcaa21902b35b1e4e6 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Mon Mar 16 05:20:22 2015 +0000 - - Add a new statistic tracking range of IDs pinned by a transaction. - - refs #1746 - -commit f954ab6b3fb6a51ecf3cf625cd3f5c5f818e0fde -Author: Keith Bostic <keith@wiredtiger.com> -Date: Sat Mar 14 12:31:00 2015 -0400 - - Pass a allocated memory size to __wt_page_inmem(). - -commit 308dc500adcde7b62c8dbb9aa0fb401795445546 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Sat Mar 14 10:51:21 2015 -0400 - - Don't drop core if duplicate symbols are detected (caller's table gets free'd twice). - -commit dc19643859063a03b985d97e2f24b1262ec4b15d -Author: Keith Bostic <keith@wiredtiger.com> -Date: Fri Mar 13 15:35:28 2015 -0400 - - Use SCNxxx #defines in sscanf() calls, not PRIxxx. - - We can't cast a pointer to a WT_LSN.offset to a pointer to a different type, we can't know WT_LSN.offset is the same size as that type. - -commit 99a992ed5207822ce2bc2fd69cb5b37408e6080f -Author: Keith Bostic <keith@wiredtiger.com> -Date: Fri Mar 13 14:17:34 2015 -0400 - - Switch sscanf from SCNu46 to SCNi64 so we handle both hexadecimal and integer symbol/frequency values - - Add testing support for out-of-range symbol/frequency values. - - refs #1536 - -commit 48a6f6e76aafbf7b33dbb22325ddb4e8e27603a1 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Fri Mar 13 10:04:06 2015 -0400 - - Fix problems with configuration value parsing, break out the Huffman configuration file parsing code into a single routine. - -commit e7ef6d0c3d107f374de9924d981d731fe36fb4ec -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Fri Mar 13 16:32:37 2015 +1100 - - Update memory allocation accounting for new pages and realloc. - - They used to not always account for alignment overhead. - -commit 1e97f9730a6fd70c03784e292cb0435c8fb82354 -Author: Don Anderson <dda@ddanderson.com> -Date: Thu Mar 12 14:02:24 2015 -0400 - - Fix to track deleted cells. Remove unused vars. - -commit 5778e4098862665d4cdaf2cda3ef40d47d951efb -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Thu Mar 12 16:35:27 2015 +1100 - - Add ability to configure sweep server timings via API. - - Still needs some test code. - -commit a1c9f8f8833482c9773fc30b7b9a7b5f1ab09014 -Author: Don Anderson <dda@ddanderson.com> -Date: Wed Mar 11 22:48:12 2015 -0400 - - Made some adjustments to stats so that the btree entries is an accurate total for row and column stores. Entries that appear in internal pages (pointing to leaf pages or other internal pages) are no longer counted. Added and modified tests to check the number of btree entries reported by stats against the number of KV pairs expected to be in btree. - - refs #1733 - -commit 6d7c061a8292021195b32260a8b41d3e92e1958f -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Wed Mar 11 11:11:57 2015 -0400 - - Support no-logging mode in wt command for debug. - - refs #1732 - -commit 4a802bd592b6c8506b07900bf89a9d9fe53cfc25 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Mar 11 09:30:39 2015 -0400 - - If the application only has 1 outstandin async operation at a time, we won't let them specify that. Change the minimum ops_max value to 1, but set the minimum maximum bound in the code to 10 as before. - -commit 3a412a2371e80ed9e26e546b6fe3c09d2d4e4091 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Mar 11 09:29:55 2015 -0400 - - If the application re-uses a WT_SYNC_OP structure, we assert: return EINVAL instead, it's easier to debug than a core dump. - -commit 16119f98fee9525bf1990ccbcc979b9f08a8b000 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Wed Mar 11 20:24:20 2015 +1100 - - Don't cache an ikey before winning the race to split into a parent. - - refs #1582 - -commit 4d0b4093e3d3c9fa0be2bbb01467579f05deddc5 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Wed Mar 11 03:44:22 2015 +0000 - - Update visibility check to know about checkpoints. - - This should allow us to evict more pages while a checkpoint is in progress. We can evict dirty pages from a file once the checkpoint is finished in that file. Similarly for the row store specific obsolete update check. - - refs #1745 - -commit 272daddb3734cd196cd303df4271f7e9e8f00cd3 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Wed Mar 11 02:55:56 2015 +0000 - - Add a global checkpoint generation and track it per data handle. Not yet used, but will be used to allow more eviction during checkpoints. - - refs #1745 - -commit 89db28287d7b212efb31203eca253fbf144ea207 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Tue Mar 10 07:19:59 2015 +0000 - - Mark statistics with the right aggregation flags (specifically cache dirty tracking and log preallocation). This helps wtstats graph generator. - - refs #1742 - -commit e51cc35a88ed497f911a25fec9f4bfcc6617d6a2 -Author: Mark Benvenuto <mark.benvenuto@mongodb.com> -Date: Mon Mar 9 15:40:59 2015 -0400 - - SERVER-17471: Use uint64_t instead of long since long is compiler specific - -commit 105903c1beb0d16b40e243ed3624417c4fa0702c -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Mar 9 12:19:50 2015 -0400 - - Remove pockets of case-sensitivity for configuration strings. - -commit 1e8c2b89de09462253231213329674b8769d11fe -Author: Keith Bostic <keith@wiredtiger.com> -Date: Sun Mar 8 13:05:51 2015 -0400 - - Change the statistics cursor WT_CURSOR.reset method to re-load statistics values. - - refs #1533 - -commit 769dc5976fe0d2448fb4fde511c5bc29eea39bfb -Author: Keith Bostic <keith@wiredtiger.com> -Date: Tue Mar 3 16:45:09 2015 -0500 - - Change checkpoints to do first-fit allocation when we start writing the actual checkpoint blocks, that way if we delete significant space, the checkpoint blocks won't prevent file truncation. - - -3.0.1, Mar 9 2015 ------------------ - -commit d654795bb763b95d14604b9b65d09ae79b8ee5b6 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Mon Mar 9 05:35:33 2015 +0000 - - Add a test case for checkpoint consistency. - - We recently fixed a bug where updating a page after the write leaves phase of a checkpoint had completed, then either checkpointed again or closed the database. We would end up with inconsistent data. That was the case because we weren't marking trees dirty all the time. - - This test case reproduces the scenario so we don't reintroduce the failure. - - refs #1735 SUPPORT-1248, SERVER-17506 - -commit 0315ee75f712ed0ccddca0616339de93b17835a3 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Mon Mar 9 11:21:06 2015 +1100 - - Bump release version to 2.5.2 - -commit 4235c69d37474fb4e14673e0ea99337659db948d -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Mon Mar 9 08:32:50 2015 +1100 - - When skipping a dirty page during a checkpoint, make sure the tree is marked dirty. - - refs SUPPORT-1248, SERVER-17506 - -commit 8382d14f32efc53b19aecd596cab3ba0d682b22d -Author: Keith Bostic <keith@wiredtiger.com> -Date: Fri Mar 6 16:10:37 2015 -0500 - - Only increment/decrement from the current position on the first position set by our caller, that is, the position passed-in from our caller is the only position the caller has already seen. - - refs SERVER-17345. - -commit ed9c48c7b8fa5dd3362e417fda8337f1690585ed -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Fri Mar 6 22:48:45 2015 +1100 - - Ignore empty child pages in column stores. - -commit 553a351ca6a81179c7f1db9c04d6f96aef0545ac -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Thu Mar 5 05:46:15 2015 +0000 - - Don't allow LSM bloom create to block waiting for space in the cache. - - We are only accessing one page at a time, and allowing the bloom create to block can lead to LSM not making progress. - - Refs: #1720 - -commit 5285b768de3237b4379f1526176efdeb55860971 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Thu Mar 5 16:15:08 2015 +1100 - - Remove the special "discard" path for trees marked clean: always go through eviction, which checks page modify flags. This may mean additional writes to internal pages when discarding a tree, but means that if a discard stops part-way through, the remaining in-memory tree has not lost any context. - - refs SUPPORT-1248, SERVER-17510 - -commit 00edf7a47115923bc9f3eaa7eee84c9f7d6b0d77 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Thu Mar 5 16:15:07 2015 +1100 - - If a page is split by eviction, mark the tree dirty to avoid having dirty pages in a tree marked clean. - - refs SUPPORT-1248 - -commit 57d3eba53fb91a0287374b9642b7cd4ef644854a -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Wed Mar 4 10:00:59 2015 -0500 - - Add fsync before closing log files and after header. #1717 - - refs SERVER-17451 - -commit d970bfe6b1bed7d1919b800bf2d65a3789b74d6f -Author: Keith Bostic <keith@wiredtiger.com> -Date: Tue Mar 3 09:38:59 2015 -0500 - - Don't set eviction_workers_min/eviction_workers_max in the connection structure before checking the values are OK. - -commit 90f3f34a97440b6788a1a558e560a33fd116f166 -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Tue Mar 3 09:26:38 2015 -0500 - - Move writing into log worker thread. #1683 - -commit 1266bbb1143b22fec6b1c255b3aade5d0506477e -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Tue Mar 3 03:37:18 2015 +0000 - - Fix a bug in the reconfigure API related to shared cache quotas. - - While fixing the bug simplify the code flow for reconfigure and caches/shared caches. - - refs #1712 - -commit 1ebd617e6dfcf542983d62d9666c5a328dd41bb8 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Mar 2 15:31:05 2015 -0500 - - Add overflow key/value counts to the statistics code - - refs #1520, #1703 - -commit a2166ead528ef61da478db67e3c0209a6bef5ac6 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Mon Mar 2 14:18:37 2015 +1100 - - Allow memory_page_max to be at most 1/4 of the cache size not 1/2. - - If we let a single page grow to half the cache size, it's too easy for the cache to get pinned full if it is tiny to start with. - -commit b036921625e415bb66ac458922b81a9fae07740b -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Mon Mar 2 14:17:12 2015 +1100 - - Take all eviction candidates if we are aggressive. - -commit 0ddd3face0b99f5653001825bf3df0662ffcdc10 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Mon Mar 2 13:07:08 2015 +1100 - - Reverse the direction of the LRU walk each time the eviction server goes to sleep. Keep looking for candidates if eviction is stuck. Don't give up our walk position if eviction is stuck. - -commit cff10ba30f2eac379197e5d7dea49da2b8159890 -Author: Don Anderson <dda@ddanderson.com> -Date: Fri Feb 27 15:05:24 2015 -0500 - - Added test for wt dump on an index. - - refs #1709 - -commit a9f0e3ac769a060a8c3b06bac71fbed0e5f46cc6 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Fri Feb 27 16:59:35 2015 +1100 - - If we encounter a dirty page when closing a clean tree, switch to checkpointing. - - refs SERVER-17319, #1643?, #1404? - -commit bfcf5987b2b6f08d931d620330aed46837e3a8c2 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Thu Feb 26 16:32:09 2015 +1100 - - Add some paranoia to LSM around transaction checks: make sure we have allocated an ID before using it for anything. - - Recheck switch transactions before doing update checks in old chunks. - - refs #1641, #1701, #1702 - -commit da0bc67c821282e9fd0da725279811b59b25a675 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Thu Feb 26 16:32:09 2015 +1100 - - Add some paranoia around setting row-store internal keys. - - refs #1582 - -commit 5ea91f6ed0e0677530c5ab8215f81eb48ea307b8 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Thu Feb 26 04:51:25 2015 +0000 - - Several optimizations to large page eviction: - - * Don't update the read generation on page in if it's set to oldest. - * Clear the walk positions before the eviction server sleeps. - * If only looking for pages that would block add them all to the queue. - * If evicting dirty pages use the worker threads, not the server. - -commit 0eecd0a2d97771380ecbd7fd27bd44988db1148c -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Thu Feb 26 02:08:36 2015 +0000 - - Fix a bug in checkpoint, where it could get an EBUSY return. - - The case that could return EBUSY was when checkpointing with a specific target, while that target was open exclusively or for a bulk load. - - Refs #1404 #1589 - -commit 392a540deec817c5d6738b8e848a68882df3ac8a -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Wed Feb 25 15:03:50 2015 -0500 - - If the LSN given doesn't exist, don't return an error, but do force recovery. Fix recover.sh to grep the CONFIG. #1700 - -commit 59b699b7085868d1b12a41ae4cd7a01f25f6e865 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Tue Feb 24 23:55:20 2015 +0000 - - Handle the case where a large record pushes us over a page boundary and we spill across two pages correctly. The previous fix could let us span additional space. - - Fix another bug in the fixup code - ensure there is enough space in the temporary buffer when fixing up after the fist page is full. - - refs #1697 - -commit ad8b58188961943e74c57b85e3b976aa03b79617 -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Tue Feb 24 18:54:18 2015 -0500 - - Set flags to SLOT_INIT_FLAGS on free. refs #1683 - -commit 45e4c049044efc96c682f390466a35d22dac555f -Author: Keith Bostic <keith@wiredtiger.com> -Date: Tue Feb 24 16:17:32 2015 -0500 - - Move WT_PAGE.u.row.d next to WT_PAGE.u.row.entries, it reduces cache misses inside of row-search. - - refs #1665 - -commit 3c6d7adf422f432ae117e2292dcce00cc3b531a3 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Tue Feb 24 16:07:16 2015 -0500 - - Turn off key prefix-compression and rebuild the key before doing the boundary split. The size of the key is likely to increase by a few bytes, and if the value is large enough to consume almost all of the buffer, we overflow the space available. - - refs #1697 - -commit ca9ab16c320f6f154ff1fd3d0b65316f87e8bddc -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Tue Feb 24 06:51:14 2015 +0000 - - Fix a bug in reconciliation. In cases where we decide to squeeze one more large entry onto a page, we could have attempted to decrement space available negative. - -commit 777c35b074fce3656c14ca5770b424a65c719134 -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Thu Feb 19 12:32:16 2015 -0500 - - Add log worker thread to advance write_lsn. - - refs #1683 - -commit ae686a225a011bac07119e2d66e837e08d5a3a0e -Author: Keith Bostic <keith@wiredtiger.com> -Date: Tue Feb 10 12:46:04 2015 -0500 - - Sasha says (my paraphrasing): I simply rearranged the fields in the WT_PAGE struct, so that u.row.d and u.row.entries are close together at the very end of the definition of union u, and the "uint8_t type;" immediately follows that. - - -3.0-RC10, Feb 24 2015 ---------------------- - -commit 2fdfb2bbed56e42e1717e567828c68d0b2eb868d -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Tue Feb 24 14:44:48 2015 +1100 - - Review places that set/clear session->dhandle, replace with macros. Change callers to save/restore if they need to release a handle after a call. - -commit ce89b608835561b11ce4e525a5ebdad86558f115 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Feb 23 19:55:56 2015 -0500 - - The statistics server has open handles and may be opening underlying handles, make sure we don't overwrite them, reference #1694. - -commit 8827d909d7f14e9ad767d909d65598508ff0025c -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Feb 23 09:39:21 2015 -0500 - - Add an explicit barrier after setting the checkpointing value. (I'm pretty sure it's not actually necessary to have a barrier in the current code, but the barrier we rely on is in a different function and isn't always called, depending the eviction configuration of this file, I'd just rather be safe than try and debug this some time in the future.) - -commit 10abb2c47cffbcc215c3507256cee3e2cae2dd5f -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Mon Feb 23 15:23:54 2015 +1100 - - If eviction is walking a file with only one live page (e.g., the last page), be careful to step over it, or eviction's hazard reference can make forced eviction stall. - - SERVER-17344 - -commit f72367f97357cf3e77f0d57d39992686b400ebe7 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Sun Feb 22 15:40:37 2015 -0500 - - Make sure we have a local copy of the start/stop cursor keys before calling into the underlying Btree range truncate function - - SERVER-17141. - -commit 09d345951d43e7ed928980048e5c5c927406a66b -Author: Keith Bostic <keith@wiredtiger.com> -Date: Fri Feb 20 11:25:22 2015 -0500 - - If WT_SESSION.truncate isn't given a start cursor, instantiate one, it's always faster to traverse Btree objects in a forward direction. - -commit ce38bc5b40f1aa3ffc07cdd2cf993e32143135b5 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Fri Feb 20 08:57:57 2015 -0500 - - We don't need to do a search in the row-store cursor truncate setup code, the WT_SESSION.truncate API code already did one. - - This does not mean WT_SESSION.truncate can't return WT_NOTFOUND in some cases, the first thing cursor truncate does is a cursor remove call in order to acquire the page's write generation information -- that remove call does the usual "discard my current reference and get a new one" work, which allows another deleting thread to race and remove one of the truncate's start/stop keys. - - refs SERVER-17141 - - -3.0-RC9, Feb 18 2015 --------------------- - -commit bf3ee2cd064b46cf0175d75950c825aa1f42c694 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Wed Feb 18 12:53:41 2015 +1100 - - Flip cache overhead to apply to the allocated bytes rather than the total size. Include the overhead in stats so that tools (e.g., mongostat) report accurate cache full and dirty percentages. This also makes eviction triggers and targets meaningful: with the default trigger of 95% and overhead 8%, eviction was previously never triggered until the cache was completely full. - -commit f9e6f942cf73c8a53aaadbc587c1b7efad6cc832 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Tue Feb 17 11:33:13 2015 -0500 - - Coverity notes the TXN_API_END_RETRY macro has an unnecessary test for "(ret == 0)" at the end of the do {} while loop. - -commit c34a56f357e21d134a2d9d0fefc032544069d8d7 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Tue Feb 17 21:56:34 2015 +1100 - - Allow the maximum number of eviction threads to be reconfigured. This was previously permitted by the API, but the array of thread contexts was not correctly resized, leading to segfaults. - - refs SERVER-17293 - -commit 67527fc235406469e69dbaec3dcd571469e660c0 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Tue Feb 17 21:50:53 2015 +1100 - - Make the eviction walk incremental: don't spend too long in any one file, fix tracking of whether we are making progress. - -commit 788265ed273c63183053e6325a9aa03c89c02860 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Tue Feb 17 21:48:44 2015 +1100 - - Combine the various checks for whether a page can be evicted into one place. - -commit 748e7b0c58b358b14340bacae41f9c46f3c06f7e -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Tue Feb 17 21:47:57 2015 +1100 - - Skip hot pages during write leaves: checkpoint will have to visit them anyway. - -commit a9de0f7ac8ad373d7aef6a480c69a2a7e0b55c59 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Tue Feb 17 21:18:41 2015 +1100 - - Run recovery after crashing test/format in the recovery test. - -commit 4733961a3c1fa37988178d1b1dd4eb44d83b63f6 -Author: Thomas Rueckstiess <thomas@rueckstiess.net> -Date: Mon Feb 16 12:50:26 2015 +1100 - - fixes and improvements for wtperf parsing - - - convert wtperf microsec to millisec - - don't skip monitor* files when parsing directory - - parsing code reorganization - - renamed wtperf stats fixture - - added tests - -commit dc396e1cd64871219b9e5a1b6558707feb70706e -Author: Keith Bostic <keith@wiredtiger.com> -Date: Sun Feb 15 13:44:30 2015 -0500 - - Clear the btree object statistics we're about to count, otherwise each cursor gets a cumulative value. - -commit 195b144bb37814b31cfa413029cda0b28f13f261 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Fri Feb 13 12:06:36 2015 -0500 - - Don't map WT_NOTFOUND to ENOENT unless a uri was specified, that's the only interesting case. Reference SERVER-17141. - -commit e9d7fee2c2c08985b8e2d2716e899853c5198290 -Author: Thomas Rueckstiess <thomas@rueckstiess.net> -Date: Fri Feb 13 17:22:05 2015 +1100 - - added support to parse wtperf files. - - they go into a separate stats section named "wtperf". - -commit 29d0d26fd1cd76392ea8225c1c4022ca54443737 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Feb 12 18:05:33 2015 -0500 - - Ignore unexpected information in the metadata entry, the metadata entry might have been created by a future release, with unknown options. - -commit 05f07753059a4fa7f0f1bab7a107a9e6d17bf4af -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Feb 12 15:42:35 2015 -0500 - - Remove the requirement of a HAVE_DIAGNOSTIC build for the verify commands to work (except for dump_offsets, that requires the btree debugging code and so won't work anywhere but a HAVE_DIAGNOSTIC build). - -commit 006ed9f17c7fc0fe65dc43717ed0239b3bac564c -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Feb 12 15:31:36 2015 -0500 - - Add support for a new verify debug option, "dump_shape", that reports the levels of the tree. - - __wt_config_gets() returns WT_NOTFOUND when there's no entry, don't fail every command when DIAGNOSTIC #defined and debug options aren't set. - -commit 46b7721215856d08ca3a37f7ffc27c57b1d4c1d7 -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Thu Feb 12 13:27:32 2015 -0500 - - Add recover config setting and use it in the wt command. #1651 - -commit 0305a51ffba383af13d6078d409a03b249c502c5 -Author: Don Anderson <dda@ddanderson.com> -Date: Wed Feb 11 10:21:04 2015 -0500 - - Add test to detect file ID problems in recovery. Refs #1622. - -commit fc0ff5a9ea09e54512353d2275126cb54dbc5451 -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Tue Feb 10 13:02:28 2015 -0500 - - Allow 'wt' command to run with or without recovery. #1651 - -commit a26d87a53eb2ac2dcae9312b7979499c34c11613 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Feb 2 19:20:33 2015 -0500 - - Replace wiredtiger_strerror_r with WT_SESSION.strerror, reference #1516. - -commit 33c146b51fdac86999e2eaa67f5636490eb441fb -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Thu Feb 12 13:44:35 2015 +1100 - - Disable aggregation across all open checkpoints if statistics cursors don't specify a checkpoint. - -commit 04ec3d021d2f8b08b69d3ea5d0f243f468c71f2e -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Thu Feb 12 13:00:49 2015 +1100 - - Move server thread waits to the beginning of their loops: check that we're still running before waiting. This makes more sense to me, but also fixes a problem introduced recently where the checkpoint server could hang on shutdown if the signal from the closing thread got lost. - -commit 85aae87cb1e019b0bcac4854e6508f11104f5339 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Feb 11 12:34:03 2015 -0500 - - Mimic Alex's fix in 152a0ef, to fsync created files to disk, for truncated - files, never surprise the upper-layer. - -commit f445f3bf63e3fa096479c5963f75d91e02f9b616 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Wed Feb 11 17:49:04 2015 +1100 - - If logs crossed the threshold size while we were taking a checkpoint, don't take another one immediately. - - Should help with SERVER-17206, where we saw two checkpoints every iteration. - -commit 0d85a9716b786de5fc90c00fb31765ade8aefd1f -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Wed Feb 11 17:48:03 2015 +1100 - - Check if a page was recently split before doing forced eviction. We used to do this, but it got lost in a recent reorg of __wt_page_release. This change should mean that after an in-memory split, application threads that are appending have time to move to the new page at the end of the tree, rather than getting stuck trying to force out a page. - - SERVER-16938, SERVER-17121 - -commit 545d064fd4cbb0b35dc536e772c60b26a193d3f2 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Wed Feb 11 17:45:26 2015 +1100 - - When doing truncates, if we see a clean page in memory, try to evict it before truncating. It should be cheap (just freeing memory), and if the eviction succeeds, the fast truncate code can kick in and mark the whole page deleted immediately. Otherwise, truncate will mark each record on the page deleted, and the next time through will try to force that page out, which has to go through reconciliation to figure out that all of the records are deleted. - - SERVER-17157 - -commit 9bbb8595abd6ac962a0debf20a6cdcef73d83855 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Tue Feb 10 16:09:36 2015 +1100 - - Allow size-limited LSM trees to have Bloom filters, based on the normal configuration. - -commit c040f84a765c7c39f03e173a555eb50f85e2e698 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Tue Feb 10 15:58:02 2015 +1100 - - Re-enable the global setting to disable LSM merges. - - refs #1657 - -commit 8f14899ba0ce5b1a8df689e3c68db9a68bfeee66 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Tue Feb 10 04:36:57 2015 +0000 - - Fix a bug when re-opening an LSM tree. - - We could have attempted to update the last chunk that is already on disk. - -commit d8263d46c1aa136d24ef194a8f62f0b02b92b9b0 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Tue Feb 10 15:11:59 2015 +1100 - - Improve LRU eviction of large pages: don't give up because a large page has recent updates: push on and try to do eviction and restore updates. - -commit eb02caa2564a18e857d18ef4b3f25683b438111c -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Tue Feb 10 15:01:37 2015 +1100 - - Fix a local variable read when looking for pages evict racing with a page becoming dirty. - - refs #1660 - -commit da4d99e7ad57057a1b8397629d59a3c83c28de21 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Tue Feb 10 02:32:25 2015 +0000 - - Fix a bug in LSM cursor open. - - The bug caused us to re-open more cursors than necessary in open. - - Related to fix: 439a655e - -commit eec16c3052af107bbe57aaf547eb8e70d2de4966 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Tue Feb 10 00:26:37 2015 +0000 - - Don't do LSM merge throttling if merges are disabled. - -commit fcee4c8ce0b5db9d3340169deb321601b81f4a1b -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Feb 9 14:04:02 2015 -0500 - - Track splits during eviction by data-source as well as by connection. - - Don't double-count in-memory splits (we're incrementing cache_eviction_split in the underlying split-parent routine, not in the caller, so it's counting both normal and in-memory splits). Instead, cache_eviction_split is normal eviction splits, cache_inmem_split is in-memory -splits, - -commit 3d1f9eace79b1aff84369d0caee245f9d6d96a60 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Mon Feb 9 06:25:36 2015 +0000 - - Add a mode to LSM where we can limit the size of data in the tree. - - A feature request to allow for a high insert throughput into a table with a size limitation. - - Adds a new configuration option to WT_SESSION::create which is lsm=(chunk_count_limit=0), default to 0 which is disabled. - - Refs #1652 - -commit c63ba34c915d95c156aaf6c47a04fe6d361b91ad -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Mon Feb 9 14:07:57 2015 +1100 - - Don't double-count the on-disk header size when setting split boundaries. - - refs #1655 - -commit 152a0efdbd3ea66b142f52eed3c9224437143eec -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Mon Feb 9 12:25:10 2015 +1100 - - Fix a bug in table create. A crash could cause recovery to break. - - Refs SERVER-17204 - - The bug is that we weren't doing an fsync of the file after it was created. Recovery assumes that if there are records for a particular file, then it will exist on disk. - -commit 4d50f5878073e582567848ae03ee506bb5058227 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Mon Feb 9 00:43:27 2015 +0000 - - Remove obsolete updates every time we add a new update. - - We used to only do the check when the cache was full. That could lead to update chains growing immensely long, which is bad. - - Refs: #1647, SERVER-17195 - -commit e891a1f312850bcaaf5183f3fd2e091567044a96 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Sun Feb 8 17:59:49 2015 -0500 - - If we find a "removed" page, clean or dirty, leaf or internal, fast-path eviction, it helps with append-only workloads. - -commit ab2a7e9b397adf657081458e11f3dc472b10fd2b -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Feb 5 15:54:01 2015 -0500 - - There's a problem that went in in #1282, the key difference is that we are setting a split boundary at the end of the first page when there is more than a page worth of data. See also #1630 and #1631. This is an alternate approach to #1631: the real change is to fallthrough into the split case if the next item won't fit, callers of the split code can't handle failure from split, it has to create enough room for the next item to be entered into the buffer. - -commit 90a352717a45a40d047b33c9fb00e7174e1ae04f -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Thu Feb 5 12:59:08 2015 -0500 - - Initialize first_lsn if we have no logs. #1638 - -commit 7cc7efb75c90e778f9757b954ad3ec85912b58fd -Author: Don Anderson <dda@ddanderson.com> -Date: Thu Feb 5 12:20:55 2015 -0500 - - For wt printlog, make operations into a JSON array. Without that, any tool that parses JSON is almost certain to merge successive values of repeated fields. - Refs #1438. - -commit 5bf11d893548804b890836a3d9ef4335c4319bb7 -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Wed Feb 4 15:46:30 2015 -0500 - - Add name_hash and hash bucket queues for fh and block. #1643 - - SERVER-17078 - -commit 3b0c18f612c9cf4d61bc13785ff7125fa67b265a -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Wed Feb 4 06:27:07 2015 +0000 - - Keep filling pages in reconciliation until we hit a boundary. - - This reverts some of a change for #1282 (without reverting the functionality in that change, AFAI can tell). - - Refs #1630 - - -3.0-RC8 Feb 4 2015 ------------------- - -commit d8b7f0b8db92a2ad6d64b95cafeaf20f0a90c8ce -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Wed Feb 4 16:00:11 2015 +1100 - - Updates should always mark pages dirty (before checking for obsolete updates to free). - -commit 0947f82e01587836277d911b147bc98eefb58507 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Tue Feb 3 10:28:00 2015 -0500 - - Fixes for split cache accounting: multi-page splits weren't correctly accounting for the allocated WT_REFs, insert splits weren't correctly handling the new right-page's instantiated key (the parent needs to be incremened by both the left- and right-hand page's keys, and cannot assume it's the same size as the original WT_REF's key), insert splits need to increment the parent page's WT_REF size by two, not one. - -commit df96addef5f3ffcb495b4bf54390cf3fd41ac924 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Mon Feb 2 16:45:46 2015 +1100 - - In recovery, track the maximum file ID in the metadata, regardless of whether there are any updates to roll forward. - - Previously, we tracked the largest file ID that was updated in the logs being rolled forward. It was usually the case that the most recently created file was also the most recently updated, so that calculation usually worked and wasn't detected until the repro in SERVER-17142 that created tables, did a clean shutdown and restart, then created more tables and did a dirty shutdown and restart, which was rolling forward updates into the wrong tables. - - refs SERVER-17142, SERVER-17131(?) - -commit 71f1559c91ed119082ebe42772da15e28915e1c8 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Tue Feb 3 10:40:27 2015 +1100 - - Start with clean trees so we can detect updates racing with sweep. - - Use a deleted ref to a leaf page that is created on first update, which is the same state the tree should be in if an empty leaf page is evicted. The only wrinkle is that bulk operations expect to find a leaf page in the tree: create it explicitly before the bulk insert starts. This was probably a bug before: if we had created a tree and kept it around for long enough with cache pressure before a bulk load started, the initial leaf page could have been evicted. - -commit 8545c4b3b7f5ed306215c82f1ad1cbe3664f0c50 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Feb 2 17:13:09 2015 -0500 - - Make the "split to deepen the tree" configuration values real, stored - in the metadata file, but they remain undocumented for now. - -commit fb769dafee4aca91a60a28cd89317c268ac79d4f -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Feb 2 16:36:22 2015 -0500 - - WT_CELL_ADDR_DEL is 0, so we can't test vtype against 0 to know if it's - been set or not. Reference SERVER-16866. - -commit feca80738c1b9103b4faa04ddb0718344347f640 -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Mon Feb 2 13:53:21 2015 -0500 - - Wrap calls to functions using pindex with WT_WITH_PAGE_INDEX. - -commit 23f2e1ba0680a2e8fa7a081f1b46e1ae2ab220d4 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Mon Feb 2 17:18:03 2015 +1100 - - Once we decide to force-evict a page, do it directly rather than setting read_gen and hoping page release agrees. - -commit 5f00de07b5bad20a6ffb5ec7d412c4ca0b10c64f -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Mon Feb 2 17:11:27 2015 +1100 - - split_gen paranoia: always increment split_gen once per split, use the allocated value to check for existing readers. Make sure that publishing a split_gen doesn't miss an update. - -commit 10a74d6af4f945e34368bc5754797ef1d684d8ab -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Mon Feb 2 16:52:34 2015 +1100 - - If discarding a tree for sweep races with an update, give up the discard gracefully. - - refs #1618, SERVER-17048 - -commit a2d20dc49cac870977d91213a7fe6dabf362ce70 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Mon Feb 2 16:45:46 2015 +1100 - - In recovery, track the maximum file ID in the metadata, regardless of whether there are any updates to roll forward. - - Previously, we tracked the largest file ID that was updated in the logs being rolled forward. It was usually the case that the most recently created file was also the most recently updated, so that calculation usually worked and wasn't detected until the repro in SERVER-17142 that created tables, did a clean shutdown and restart, then created more tables and did a dirty shutdown and restart, which was rolling forward updates into the wrong tables. - - refs SERVER-17142, SERVER-17131(?) - -commit b0a828b262a2d0d3cf1361eed98aa25a1168a7a6 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Sat Jan 31 12:59:34 2015 -0500 - - We no longer calculate allocation overhead per allocation chunk, revert the workaround the problem with page memory size calculations during splits where we forced the new parent page memory size to 5% of its current value; reference #1564, #1565. This fixes a problem where 5% of a page's memory footprint isn't large enough to accommodate the cache decrements that will be done in the page's future, leading to page underflow. - - Minor cleanups: we no longer calculate allocation overhead per allocation chunk, the macro WT_MEMSIZE_ADD is no longer needed at all, and WT_MEMSIZE_TRANSFER is renamed to WT_MEM_TRANSFER. - -commit b640366c28fc66744e482c20c16973cb052aef8e -Author: Keith Bostic <keith@wiredtiger.com> -Date: Fri Jan 30 10:19:31 2015 -0500 - - I believe we can race with pages being marked clean or dirty, which means we need to entirely divorce the page's dirty-byte count from page state: the page's dirty-byte count is just a value that tells us how many dirty bytes this page has contributed to the cache's total dirty-bytes count. Sync the cache's information to that value when possible, but don't worry if we can't. - -commit d02ea7246ec33e05b5fd60c499fea3ffe25c57d2 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Fri Jan 30 17:38:09 2015 +1100 - - Use reads to measure cache pressure with shared caches. We previously tracked writes, which were skewed by checkpoints. - - refs #1569 - -commit a326c3ba10e0d299944a650b890f8c2d851db34a -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Jan 29 17:19:06 2015 -0500 - - Simplify the cache calculation when a page is marked clean, use the page's dirty-byte count (which allows a race between the page being marked clean and being re-dirtied). - - This branch is still not correct, but appears to be able to run the CONFIG from #1582 without underflow for a much longer time). - - Reference #1605. - -commit 1c60c4966dd68ea2bf05ebe62e3f1d8de1a7d033 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Thu Jan 29 14:33:36 2015 +1100 - - Use a copy of the oldest transaction ID when sweeping cached overflow items. Otherwise, we could free structures that are still hooked into the skiplist. - - refs #1615 - -commit 42724267278c64f5af68b281c9ee5742d1a56d72 -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Wed Jan 28 10:31:51 2015 -0500 - - Adjust logging yield and timeout values. #1610 - -commit ae102f4fe604f7fd547dece8ee138e8292d4f07c -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Wed Jan 28 17:40:06 2015 +1100 - - Cleanup accounting for update lists when restoring updates to evicted pages. Previously, we only accounted for the first update in a list. - - refs SERVER-16997 - -commit 4adf9c929b1b46f273239214b4e2757fcfdb8f96 -Author: Mark Benvenuto <mark.benvenuto@mongodb.com> -Date: Fri Jan 23 18:29:15 2015 -0500 - - Windows Install Documentation - -commit 8faa218d27e7f21091f0b51a973f27047db1d950 -Author: Mark Benvenuto <mark.benvenuto@mongodb.com> -Date: Tue Jan 27 13:47:24 2015 -0500 - - MCI configuration update - -commit 422cbb6cea5fa5be6829044215ae46dc10be5f70 -Author: Mark Benvenuto <mark.benvenuto@mongodb.com> -Date: Mon Jan 26 16:11:29 2015 -0500 - - Add Install Target to SCons - -commit 41e7ab083d79a650e93a34d09e01e973ca4100d9 -Author: Mark Benvenuto <mark.benvenuto@mongodb.com> -Date: Fri Jan 23 15:54:40 2015 -0500 - - WiredTiger DLL support - - Examples that only depend on public API use DLL now - -commit 23b2493e75cd166075eaccdaef75c8beee4576db -Author: Mark Benvenuto <mark.benvenuto@mongodb.com> -Date: Fri Jan 23 15:50:16 2015 -0500 - - Scons Improvements - - Added --enable-attach, --enable-diagnostic, --enable-verbose - - Renamed --enable-swig to --enable-python for consistency + swig cleanup - - Renamed wiredtiger static library to libwiredtiger.lib - -commit 96ab0ef67eee20fa75fa6d52c97d98bc119b74ae -Author: Mark Benvenuto <mark.benvenuto@mongodb.com> -Date: Thu Jan 15 15:07:01 2015 -0500 - - Struct alignment and packing for MSVC - -commit f3b65997ece52382eed91730416d5f919bea79cd -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Fri Jan 9 10:49:59 2015 -0500 - - Fix huffman config and add huffman tests. #1536 - -2.8-RC7 Jan 27 2015 -------------------- - -commit 2b4172f17008ff36dbeb50cadaf4fb97fc859e4e -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Tue Jan 27 15:50:09 2015 +1100 - - Revert a workaround for splits during truncate. - - refs #1583, #1563, SERVER-16868 - -commit c2e108e2774ae79504579bcdca33f26fcff8cb07 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Tue Jan 27 09:58:32 2015 +1100 - - Change recovery to start from the checkpoint LSN in the metadata. Don't assert that we see a checkpoint complete in the available log: if the application crashes in between syncing the metadata and writing the final checkpoint record, there is no need to roll anything forward but we don't have the final checkpoint. - - refs #1529 - -commit 2555e80d2020ba9833c436a22d1031f6c5778a64 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Jan 26 14:31:25 2015 -0500 - - Coverity CID 50796 (#1 of 1): Unintentional integer overflow (OVERFLOW_BEFORE_WIDEN) - SERVER-17001 - -commit 1ce3b94d6e40d37a77e62eda500f286bd3816eb9 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Mon Jan 26 15:56:25 2015 +1100 - - Grab the table list lock while building the list of handles to checkpoint. - - This avoids a potential deadlock during compact operations and/or checkpoints with a target list (and an assertion about lock ordering in diagnostic builds). - - Note that nested locking is not ideal: the medium-term fix here is #1598. - - refs #1589, SERVER-16967 - -commit db3943563a87c3e4c42445ae9f3a07efacfdf4ac -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Mon Jan 26 14:54:47 2015 +1100 - - Free WT_REFs deleted by truncate. We were doing this when a page spontaneously became empty, but not if the "fast truncate" code kicked in. - - refs SERVER-16921 - -commit 2063efb22c3c29b980f86f7fee77b6d03ba63ec1 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Fri Jan 23 16:21:06 2015 -0500 - - Don't count pages evicted by a worker thread as an "application thread" eviction; add a new statistic to distinguish between the server itself evicting pages and the eviction worker threads evicting. - - Don't increment the eviction counters unless we find a page to evict, __evict_lru_pages() gets called a huge number of times in any workload where eviction is happening. - - Reference SERVER-16997, SERVER-17020. - -commit 3abb99d58aaa46b0b3fcd338293a668422e3fcaf -Author: Mark Benvenuto <mark.benvenuto@mongodb.com> -Date: Fri Jan 23 15:05:16 2015 -0500 - - Close Thread Handle after thread join on Windows - -commit 7d677aedfdcaa5458e900e556b662def460d0281 -Author: Don Anderson <dda@ddanderson.com> -Date: Fri Jan 23 08:52:13 2015 -0500 - - Fix drop index on a newly opened session. - Fix __wt_schema_open_index to return WT_NOTFOUND when opening a single index. This fixes opening a cursor on a non-existant index. - Refs #1567. - -commit 3626081dff24e1448281d10658752b996897ca82 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Jan 22 18:08:15 2015 -0500 - - Add the cache_overhead configuration string to allow applications to configure their cache overhead. - -commit 4843cd78e7f90937ebdb23f84fbd7c133a7e5256 -Author: Don Anderson <dda@ddanderson.com> -Date: Thu Jan 22 10:40:11 2015 -0500 - - Prepend underscores to SWIG methods that could have name conflicts - with WT internal names. refs #1574. - -commit ebb1d9402c0ce2911069b0437d71766b92c3dc12 -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Wed Jan 21 12:57:20 2015 -0500 - - Add log code to ensure write-no-sync. #1585 - -commit 44fa4fbff95d0689b20c3fe3f4a55202554f0d9f -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Jan 19 14:25:39 2015 -0500 - - Make compact more aggressive about finding blocks to move. - -2.8-RC6, Jan 20 2015 --------------------- - -commit ab1d63d3aa2371ce53287c6c6c77833eb281a38a -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Tue Jan 20 15:37:46 2015 -0500 - - Check for valid log_fh handle in wt_log_write. #1580 - -commit e2de971061abea9451e92d60f0870136c9c0af42 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Tue Jan 20 13:24:06 2015 -0500 - - Quit page eviction immediately if we're trying to evict a tree, that is, an internal page that has other internal pages as children. - -commit 6f3c5a933ef8ce79efc03a22a8c03526ffb2197b -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Jan 19 12:38:24 2015 -0500 - - The size of the file is decreasing each time, so compacting 10 times, at 10%, is not sufficient to drive a file to its smallest size. The right fix is probably to get better information from the block manager as to exactly how much the size of the file has decreased, but that's messy, especially when you consider the checkpoints requires to get to that smallest size. For now, do 100 compaction attempts instead of 10, and depend on the no-progress state and/or the compaction timeout to limit the amount of time we spend here. - -commit 72172b088fba6769866aecabba8176303140f5c4 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Jan 19 10:25:13 2015 -0500 - - Coverity 1264611, memory leak (WT_RET that should have been a WT_ERR). - -commit f61f984cf5241ac54bc2ea368c8c15e0cdfa91aa -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Sat Jan 17 22:25:02 2015 +0000 - - Fix a deadlock opening statistics cursors. - - Refs #1575 and JIRA SERVER-16738 - -commit c5fa51a0f18e4117d9f7b841de86eb35af751264 -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Sat Jan 17 09:07:04 2015 -0500 - - Log close thread needs to wait for any outstanding writes. #1571 - -commit 9cd8120f491595ad6ac1c25c4b154ad6556b5fe7 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Sat Jan 17 09:49:09 2015 +1100 - - Close the session for the log close server thread. Fixes a leak detected by address sanitizer. - -commit bd7364ea9a0542bee61db0a89e771faf814f6f53 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Fri Jan 16 21:03:55 2015 +0000 - - Fix a bug in raw compression, where we were overflowing memory. - - We weren't growing the buffer enough when adding new items in. - - Refs SERVER-16664 - -commit 76addf73581c53f24462ab5fd724048aec36eaf3 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Sat Jan 17 05:48:03 2015 +1100 - - Have WT_CURSOR::equals return 1 when cursors are equal, 0 when not. - -commit b2841dfc015d9502e1def870605968144b935570 -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Thu Jan 15 21:58:39 2015 -0500 - - Add log thread to fsync and close log files. #1560 - -commit ebb93969ebfb6b9bb9dc60621933f2fbeac4b472 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Jan 15 22:47:52 2015 +0000 - - Don't do memory adjustments for the WT_REF's WT_ADDR structures, we don't do those adjustments in other places we set addresses. - - Workaround the problem with page memory size calculations during splits by forcing the new parent page memory size to 5% of its current value; reference #1564. - - Minor cleanups/renaming of the code instantiating the WT_REF structures during a tree-deepening split to clarify what's going on there. - -commit e0031209183c880fb1a1b99399013e7675a75e88 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Fri Jan 16 09:26:34 2015 +1100 - - Don't look at a page after it may be freed during split. - - During the process of doing a split we switch the ref to WT_REF_MEM - after which it's no longer safe to refer to the page. Shuffle the code so that we don't. - - SERVER-16868 - -commit b6d7532cbf823d537b8f1733169fe4de08173c09 -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Thu Jan 15 16:55:00 2015 -0500 - - Only advance sync_lsn to the end of our write. We waited until the log->sync_lsn is advanced into our file. It was a bug to set the sync_lsn to the current write_lsn as that can be too far ahead in a new log file when earlier log files aren't done yet. - -commit 85851933a938c53dfa57d1621cab1a959db672eb -Author: Thomas Rueckstiess <thomas@rueckstiess.net> -Date: Mon Dec 15 11:04:43 2014 +1100 - - wtstats.py: removed python-nvd3 dependencies, rewriting with HTML template - -commit 4c26d2324bae1d7030b0142d50dbd2ccf11ddeb6 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Dec 11 19:32:50 2014 -0500 - - Add support for a WT_CURSOR.reconfigure method, reference #1381. - -2.8-RC5 Jan 15 2015 -------------------- - -commit 2e54a27683c5e2fd88918575383c76d3f60c3c78 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Thu Jan 15 07:17:21 2015 +1100 - - Workaround a read-after-free involving eviction during truncates. We were implicitly relying on first_dirty_txn to prevent pages being immediately force-evicted by truncate. The bug is not fully understood, but this change restores the previous window where reads can complete before the page is evicted for real. - - refs BF-759 - -commit 8a1bfe3c35f0c1d90ea3e8e70c2aae8dff1fdbb3 -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Wed Jan 14 15:07:39 2015 -0500 - - Force log file closes to go in sequence. #1555 - - Update the sync_lsn after sync'ing and closing an earlier log file and make sure archive doesn't try to remove a file that is still in use. - -2.8-RC4 Dec 22 2014 -------------------- - -commit fbb96d94cdba9a28f5c5d737ce6c96543f3289f4 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Mon Dec 22 15:59:46 2014 +1100 - - Use the original page's first_dirty_txn when restoring updates to match what we do for in-memory splits. - - refs #1475 - -commit 4df72e8e20139ddf667e1f0d3b6b7dcf91deb006 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Mon Dec 22 13:12:07 2014 +1100 - - Avoid EBUSY returns to verify and salvage caused by checkpoints. The "fix" involves blocking checkpoints while salvage or verify are in progress. - - refs #1404, SERVER-16457 - -commit 864f3495721b1311b49df19ee241bfca9adf0863 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Sun Dec 21 20:47:52 2014 -0500 - - Make the cache bytes-written and bytes-read match, both should - ignore compression. Reference #1505. - -commit 995d6f8c26ae19013a1eb921fd871481ca643f47 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Mon Dec 22 12:42:46 2014 +1100 - - Eviction should do update-restore if upper layers are trying to force out a page, regardless of its size. Also, only look at ref->page after checking for exclusive access. It is possible (but very unlikely) that a child page pointer could be replaced in the window where we are checking hazard pointers. - -commit d4abc51ea61211f90f4b70a0486442264ededc27 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Mon Dec 22 10:43:04 2014 +1100 - - Fix a bug where a custom extractor terminate was being called twice. - - Resolves issue #1503. - - Clarify the custom extractor and collator terminate documentation while I'm here. - -commit 16972ef63de1283d85146530c35f522b053e2c1e -Author: Don Anderson <dda@ddanderson.com> -Date: Fri Dec 19 09:56:47 2014 -0500 - - Remove version numbering from the pkg-config file. We don't create include files that are named by version. Programs linked using -lwiredtiger will follow the symlink to wiredtiger-a.b.c.so, so their referred library name is forever stamped as wiredtiger-a.b.c.so, which won't conflict even when we ship wiredtiger-a.b.d.so. - Refs #1458. - -commit e913b0811114d65b543cd78824e809eb487fd330 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Fri Dec 19 17:19:58 2014 +1100 - - Check that handles are not being walked by eviction before discarding. - - refs #1497 - -commit 0d21e437917bc7cf08393852a3074957431ea30e -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Fri Dec 19 15:44:15 2014 +1100 - - Use the eviction server to write pages with READGEN_OLDEST set. - - Even before the eviction trigger has been reached. This should mean that we clear those pages out of cache earlier, and hopefully will save application threads from doing the evictions (at least sometimes). - -commit e0adfba3c4011c49b73ff3e4a165a4a938f69cb3 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Fri Dec 19 12:23:21 2014 +1100 - - Don't try to write leaves from the sweep server. Previously, this was done without locking the handle, and so could race with LSM discarding a handle. - - We know the handle has been idle, so there is a good chance that a checkpoint has run since the last update and the write leaves was wasted effort. If not, this change will keep the handle locked for longer preventing new opens, but it has been idle for a while anyway. - - refs #1495, #1497 (maybe) - -commit 87328a8d5c1d4a201a1df604ba32a87863948bbb -Author: Mark Benvenuto <mark.benvenuto@mongodb.com> -Date: Thu Dec 18 14:51:51 2014 -0500 - - fix test_salvage on Windows - -commit 1953776ada137f3deae50169bf889d2063b353d3 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Thu Dec 18 16:45:06 2014 +1100 - - Don't try to set and clear session->split_gen in WT_INTL_FOREACH_BEGIN: there are too many of those loops to ensure that none of them skip clearing it. Instead, make sure all calls are wrapped in WT_WITH_APAGE_INDEX. - - refs SERVER-16546 - -commit 235f747e2df80d9899497595a2b649e7d6df8601 -Author: Mark Benvenuto <mark.benvenuto@mongodb.com> -Date: Wed Dec 17 14:11:41 2014 -0500 - - snprintf - Implement a custom version of snprintf match the truncation behavior of C99 standard snprintf until MSVC supports snprintf. - -commit 857a6fd0c4b6b001c78cbbc507674e2129029dff -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Dec 17 15:51:41 2014 -0500 - - Check the block header checksum before we clear it, it should be the same as the passed-in checksum, and if just those 4 bytes are corrupted, we wouldn't noticed. SERVER-16457. - -commit dfa706056c4a359f7f894047bc9e5399efcec776 -Author: Don Anderson <dda@ddanderson.com> -Date: Tue Dec 16 15:58:34 2014 -0500 - - Some refactoring of python packing. More checks for standalone unit tests. - Refs #1429. - -2.8-RC3 Dec 17 2014 -------------------- - -commit bb064847e1c45f2b396d3f65f4e08cd10f33ed6e -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Wed Dec 17 15:45:23 2014 +1100 - - Detect write-write conflicts before no-overwrite cursors decide to skip an update. - - refs SERVER-16351 - -commit 91abf8e35d5246a653bd9615ffd9723d87999c38 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Wed Dec 17 13:36:02 2014 +1100 - - Add support for none configuration string to log compressor. - - To be consistent with block compression configuration. - -commit 5438fee4942b4dbf484799dad6e12e042d253e99 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Wed Dec 17 12:09:33 2014 +1100 - - Return an error if a shared cache configuration is set, but not enabled. - - Check for a configuration via shared_cache=(size=). - - Refs #1487 - -commit 390a5b71b25492dc3030e908a65a11a04401852b -Author: Keith Bostic <keith@wiredtiger.com> -Date: Tue Dec 16 15:32:09 2014 -0500 - - We documented that huffman_key and huffman_value took "none" as an argument, but they didn't. - - Worse, if key was set but not value, we'd set value anyway, free of charge, and if value was set but not key, we'd fail. I doubt this is a problem (it's pretty surprising if anyone would set key but not also set value). - - Use __wt_config_gets_none() to support the "none" setting, re-work the logic to ignore keys or values that are zero-length. - - Reference #1417. - -commit 662e26eeb31f76f2c4aeebf6690c9056612de32e -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Tue Dec 16 14:20:01 2014 -0500 - - Atomically create all log files and move them into place. #1482 - -commit 5c30d62dbf7ec0976d6ec4d2aed4ba272aadd499 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Dec 15 19:41:05 2014 -0500 - - If we race with the logging thread and get to __wt_logmgr_destroy() while __log_archive_once() is still using conn->log_path, we can free it out from under the running server. If there's a logging thread running, don't free conn->log_path until we've joined that thread. - Reference #1480. - -commit d77d35db407fd74c266bdb728b12c74fdab26ba2 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Mon Dec 15 18:33:54 2014 -0500 - - Even if we don't track any overflow pages during our read of the file, we still have to process the list of pages looking for leaf pages that reference unavailable overflow pages, no overflow pages doesn't imply there are no references to overflow pages. - -commit cedf8cfe69bf964629aab498feb20a0b1ab77bc0 -Author: Don Anderson <dda@ddanderson.com> -Date: Mon Dec 15 11:54:15 2014 -0500 - - Fix use of 'compressed' flag for printlog. - Added printlog call to test case for log compression. - Refs #1472. - -commit 3210b11cf7bfb79f3ed52cd1c17a13c644a82e7a -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Mon Dec 15 11:02:10 2014 -0500 - - Fix memory leak. Always free log_path. #1473 - -commit 097c61e5f3326bc71f2d645b3f539c5c6d1ae3fb -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Mon Dec 15 05:58:06 2014 +0000 - - Allow printlog to work without recovery. - - It now works even if compression or a custom path are setup. - -commit b8921272755ce66d09ab2a001745573420bd41ac -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Dec 10 15:40:06 2014 -0500 - - We can't use the corrected page size to calculate the buffer's space available, but we don't have a page size either. We do know how much space we added, so use that to increment the space available. - - The raw compression handler can no longer pass a page size buffer to the underlying compression function, because a single key/value object could be larger than the page size, instead, pass a buffer of the same length as the source to compress, whatever that is. - -commit 402041727de02931be1dd385f3c970f31a53341c -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Wed Dec 10 10:53:27 2014 +1100 - - Remove the min / max bounds on overflow sizes. - -commit 5088ee53fce569915e8de8c168da50cff7991ec1 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Tue Dec 9 16:22:20 2014 -0500 - - Separate the btree maximum key/value sizes from the underlying page size, reference #1282. - - Deprecate the internal_item_max and leaf_item_max configuration strings, replace with internal_key_max, leaf_key_max and leaf_value_max. - - Remove examples/c/ex_file.c (there's no real need for a "file" URI example, and it's easy to replace the one place the documentation used it). - -commit e1e187e8fdfb48526f2a62e3f0f48072c30db53e -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Tue Dec 16 10:29:08 2014 +1100 - - Update the swept handles statistic any time we close the underlying handle, regardless of whether a session still references the data handle. - - reds #1460, #1466 - -commit 080b34fde5de97459c383c67ba93d9fdc88090a8 -Author: Don Anderson <dda@ddanderson.com> -Date: Fri Dec 12 11:13:22 2014 -0500 - - Allow pruning scenarios with different limits for default vs. long runs. The pruned scenario list now matches the original ordering. Tests now print with both the scenario number and the scenario args. This should make it easier to diagnose and debug problems that effect only certain scenarios. Refs #1461. - -commit d5b88e08e2f6e39d098cfff3c013f4aa035c88bc -Author: Don Anderson <dda@ddanderson.com> -Date: Thu Dec 11 15:13:11 2014 -0500 - - Changed python test suite to allow for shorter runs by default, with a --long (or -l) option for the complete runs. Txn02 in particular now runs only a small number of scenarios by default. Also added a @longtest("description") decorator for individual tests that can be marked to be run only under --long. Refs #1461. - -commit 2f37332e5bbd14823f0c78ad38672dbce074e87f -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Fri Dec 12 17:21:50 2014 +1100 - - Ensure metadata table is open at start of checkpoints. - -commit 2cb10882f4f7189a3c2de4d7e187117873fded32 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Fri Dec 12 17:07:11 2014 +1100 - - Switch to stashing the metadata dhandle, not the btree. - - Also update checkpoint to use the saved handle instead of - searching for it again. - -commit 42c05161cf8cc74606b259ceeeb41dd38ea7fb4e -Author: Susan LoVerso <sue@wiredtiger.com> -Date: Thu Dec 11 15:24:14 2014 -0500 - - Use dhandle hash lists in more places. Adjust sweep timings. #1460 - -commit 7fb6315e45e74f0bef0a04505018e1ab0b68d144 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Thu Dec 11 23:06:19 2014 +1100 - - If LSM search_near finds a matching tombstone, step the whole LSM cursor next to find the closest key. We can't step individual chunk cursors, or we could return a record that is deleted in a more recent chunk. - - MongoDB BF-694, BF-700 - -commit 5f6bbc898564aefb312255555abd34202cb98815 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Wed Dec 10 14:39:25 2014 +1100 - - Track whether eviction is making progress regardless of whether the cache is 100% full. Otherwise we can get into a tight loop. Use the count of pages evicted rather than a flag, now that there are multiple eviction threads. - -commit cbe9e9bdbc508f95076b8097d41bb4cc799eab1c -Author: Don Anderson <dda@ddanderson.com> -Date: Tue Dec 9 19:11:52 2014 -0500 - - Change timing to allow archive thread to complete on tests that - do archive. Since this can make each test run substantially longer, - reduce the number of tests that are doing archive from ~4000 to - something under 100. Refs #1452. - -commit 62af85890179abb9fda17a619fcd5ae69fb369e0 -Merge: b83bf08 b24c7af -Author: Michael Cahill <mjc@wiredtiger.com> -Date: Wed Dec 10 11:03:48 2014 +1100 - - Merge pull request #1449 from wiredtiger/lsm-switch-simplify - - Improve and simplify the LSM switch logic - -commit 7e0f7d7b803f9af04ad10b2bec6ef5073aa79248 -Author: Don Anderson <dda@ddanderson.com> -Date: Tue Dec 9 16:04:48 2014 -0500 - - SESSION->drop with "force" of nonexistant index/colgroup should be silent. - Refs #1436. - -commit be364821d75c0c42169d79c486fa582c777f7082 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Tue Dec 9 15:58:28 2014 +1100 - - Sweep old handles more aggressively: - - 1. don't have checkpoint or other periodic operations like statistics logging keep old handles alive; - 2. don't wait for all sessions to empty the file from their cache before closing; - 3. only update the time of death from the sweep thread. - -commit c96a4c954ccc73744f8a1fbcf2fea6debdfca018 -Merge: cc8eb0b 2e332b9 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Tue Dec 9 14:41:36 2014 +1100 - - Merge pull request #1443 from wiredtiger/cursor-open-optimize - - Cursor open optimize - -commit 993c8ede8ff64eac9e87d1adcd39f8575039222b -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Tue Dec 9 13:21:13 2014 +1100 - - Improve and simplify the LSM switch logic: it is fine to keep writing into a chunk while it is being switched, until either it hits the hard chunk size limit or a switch transaction ID is chosen that is larger than the writer's. Fixes an assertion failure introduced by #1432 that could write to an old chunk after the switch completed. - - refs #1432, #1418 - -commit 5551461cd5f26249e4330c9f87b4945d7ec2bb34 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Mon Dec 8 18:01:31 2014 +1100 - - If there are only two LSM worker threads, don't let the cache get full. - - Allow the first thread to do flushes as well as switches and drops if there are only two threads. - - Refs #1441, but this is a hang seen from that test/format configuration, not a segfault. - -commit 8f06d6b79dabed54ad1e05515bbdb31e23c4e991 -Author: Don Anderson <dda@ddanderson.com> -Date: Fri Dec 5 14:22:41 2014 -0500 - - Modify printlog output so that arbitrary strings are shown as using the JSON Unicode standard. refs #1438. - -commit 68090796dea07e7b2d3d5bee8d69aafcd8febe16 -Author: Alex Gorrod <alexg@wiredtiger.com> -Date: Fri Dec 5 04:36:25 2014 +0000 - - Avoid string comparisons when looking up tables. - - Save a hash value in the table so we can do integer comparisons rather than string comparisons. - -commit 4de5e3a71bfad1c2a9ef1eccccdd45ec02fecba9 -Author: Michael Cahill <michael.cahill@wiredtiger.com> -Date: Fri Dec 5 13:01:51 2014 +1100 - - Force eviction if we see many consecutive deletes when scanning through a page. This fixes quadratic behavior in find-first+delete workloads. - -commit bbced52c939e16ad5662b3a177cef3e52abddd6e -Author: Keith Bostic <keith@wiredtiger.com> -Date: Thu Dec 4 07:57:13 2014 -0500 - - In the final close, continue and remove the handle no matter what errors we see, otherwise the handle-close code in __wt_conn_dhandle_discard() can become infinite loops, where we repeatedly attempt to close the same file handles. Reference #1434. - -commit 46fa7f0b6397fe765c5e8c2853f9cd0b067bc808 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Dec 3 13:46:01 2014 -0500 - - Changes in #1204, #1288 mistakenly changed the values for some error defines, which breaks backward compatibility. (The WT_DEADLOCK error name sorted differently from WT_ROLLBACK, and we were assigning error values based on the sort order in a script.) Revert the change, and make sure it doesn't happen again. - -commit 249e88485c75951a0584a7c7a8dd4b8f8b6a3382 -Author: Keith Bostic <keith@wiredtiger.com> -Date: Wed Dec 3 06:25:16 2014 -0500 - - Support "none" in all configuration strings as an alternative to an empty string. Reference #1417. - -commit 63d7c7869f8c2ab5a3e6ee935d1e37f21d40755f -Author: Don Anderson <dda@ddanderson.com> -Date: Tue Dec 2 14:00:11 2014 -0500 - - Added log compression. When configured, we attempt to compress each log record. Added printlog output to show before/after compression sizes. Refs #1359. - diff --git a/src/third_party/wiredtiger/bench/wtperf/idle_table_cycle.c b/src/third_party/wiredtiger/bench/wtperf/idle_table_cycle.c index b699b5b9dd1..3c079bb560f 100644 --- a/src/third_party/wiredtiger/bench/wtperf/idle_table_cycle.c +++ b/src/third_party/wiredtiger/bench/wtperf/idle_table_cycle.c @@ -129,7 +129,8 @@ cycle_idle_tables(void *arg) * Drop the table. Keep retrying on EBUSY failure - it is an * expected return when checkpoints are happening. */ - while ((ret = session->drop(session, uri, "force")) == EBUSY) + while ((ret = session->drop( + session, uri, "force,checkpoint_wait=false")) == EBUSY) __wt_sleep(1, 0); if (ret != 0 && ret != EBUSY) { diff --git a/src/third_party/wiredtiger/bench/wtperf/runners/checkpoint_schema_race.wtperf b/src/third_party/wiredtiger/bench/wtperf/runners/checkpoint_schema_race.wtperf new file mode 100644 index 00000000000..ade8e88ee9b --- /dev/null +++ b/src/third_party/wiredtiger/bench/wtperf/runners/checkpoint_schema_race.wtperf @@ -0,0 +1,20 @@ +# Check create and drop behavior concurrent with checkpoints (WT-2798). +# Setup a multiple tables and a cache size large enough that checkpoints can +# take a long time. +conn_config="cache_size=8GB,log=(enabled=false),checkpoint=(wait=30)" +table_config="leaf_page_max=4k,internal_page_max=16k,type=file" +icount=10000000 +table_count=100 +table_count_idle=100 +# Turn on create/drop of idle tables, but don't worry if individual operations +# take a long time. +idle_table_cycle=120 +populate_threads=5 +checkpoint_threads=0 +report_interval=5 +# 100 million +random_range=10000000 +run_time=300 +# Setup a workload that dirties a lot of the cache +threads=((count=2,reads=1),(count=2,inserts=1),(count=2,updates=1)) +value_sz=500 diff --git a/src/third_party/wiredtiger/build_win/filelist.win b/src/third_party/wiredtiger/build_win/filelist.win deleted file mode 100644 index c370303d5f8..00000000000 --- a/src/third_party/wiredtiger/build_win/filelist.win +++ /dev/null @@ -1,172 +0,0 @@ - -# List of source files for WiredTiger library. -# filelist -- -src/async/async_api.c -src/async/async_op.c -src/async/async_worker.c -src/block/block_addr.c -src/block/block_ckpt.c -src/block/block_compact.c -src/block/block_ext.c -src/block/block_map.c -src/block/block_mgr.c -src/block/block_open.c -src/block/block_read.c -src/block/block_session.c -src/block/block_slvg.c -src/block/block_vrfy.c -src/block/block_write.c -src/bloom/bloom.c -src/btree/bt_compact.c -src/btree/bt_curnext.c -src/btree/bt_curprev.c -src/btree/bt_cursor.c -src/btree/bt_debug.c -src/btree/bt_delete.c -src/btree/bt_discard.c -src/btree/bt_handle.c -src/btree/bt_huffman.c -src/btree/bt_io.c -src/btree/bt_misc.c -src/btree/bt_ovfl.c -src/btree/bt_page.c -src/btree/bt_read.c -src/btree/bt_rebalance.c -src/btree/bt_ret.c -src/btree/bt_slvg.c -src/btree/bt_split.c -src/btree/bt_stat.c -src/btree/bt_sync.c -src/btree/bt_upgrade.c -src/btree/bt_vrfy.c -src/btree/bt_vrfy_dsk.c -src/btree/bt_walk.c -src/btree/col_modify.c -src/btree/col_srch.c -src/btree/row_key.c -src/btree/row_modify.c -src/btree/row_srch.c -src/cache/cache_las.c -src/config/config.c -src/config/config_api.c -src/config/config_check.c -src/config/config_collapse.c -src/config/config_def.c -src/config/config_ext.c -src/config/config_upgrade.c -src/conn/api_strerror.c -src/conn/api_version.c -src/conn/conn_api.c -src/conn/conn_cache.c -src/conn/conn_cache_pool.c -src/conn/conn_ckpt.c -src/conn/conn_dhandle.c -src/conn/conn_handle.c -src/conn/conn_log.c -src/conn/conn_open.c -src/conn/conn_stat.c -src/conn/conn_sweep.c -src/cursor/cur_backup.c -src/cursor/cur_bulk.c -src/cursor/cur_config.c -src/cursor/cur_ds.c -src/cursor/cur_dump.c -src/cursor/cur_file.c -src/cursor/cur_index.c -src/cursor/cur_join.c -src/cursor/cur_json.c -src/cursor/cur_log.c -src/cursor/cur_metadata.c -src/cursor/cur_stat.c -src/cursor/cur_std.c -src/cursor/cur_table.c -src/evict/evict_file.c -src/evict/evict_lru.c -src/evict/evict_page.c -src/log/log.c -src/log/log_auto.c -src/log/log_slot.c -src/lsm/lsm_cursor.c -src/lsm/lsm_cursor_bulk.c -src/lsm/lsm_manager.c -src/lsm/lsm_merge.c -src/lsm/lsm_meta.c -src/lsm/lsm_stat.c -src/lsm/lsm_tree.c -src/lsm/lsm_work_unit.c -src/lsm/lsm_worker.c -src/meta/meta_apply.c -src/meta/meta_ckpt.c -src/meta/meta_ext.c -src/meta/meta_table.c -src/meta/meta_track.c -src/meta/meta_turtle.c -src/os_common/filename.c -src/os_common/os_abort.c -src/os_common/os_alloc.c -src/os_common/os_fhandle.c -src/os_common/os_fs_inmemory.c -src/os_common/os_fs_stdio.c -src/os_common/os_getline.c -src/os_common/os_getopt.c -src/os_common/os_init.c -src/os_common/os_strtouq.c -src/os_win/os_dir.c -src/os_win/os_dlopen.c -src/os_win/os_errno.c -src/os_win/os_fs.c -src/os_win/os_getenv.c -src/os_win/os_map.c -src/os_win/os_mtx_cond.c -src/os_win/os_once.c -src/os_win/os_pagesize.c -src/os_win/os_path.c -src/os_win/os_priv.c -src/os_win/os_setvbuf.c -src/os_win/os_sleep.c -src/os_win/os_snprintf.c -src/os_win/os_thread.c -src/os_win/os_time.c -src/os_win/os_vsnprintf.c -src/os_win/os_yield.c -src/packing/pack_api.c -src/packing/pack_impl.c -src/packing/pack_stream.c -src/reconcile/rec_track.c -src/reconcile/rec_write.c -src/schema/schema_create.c -src/schema/schema_drop.c -src/schema/schema_list.c -src/schema/schema_open.c -src/schema/schema_plan.c -src/schema/schema_project.c -src/schema/schema_rename.c -src/schema/schema_stat.c -src/schema/schema_truncate.c -src/schema/schema_util.c -src/schema/schema_worker.c -src/session/session_api.c -src/session/session_compact.c -src/session/session_dhandle.c -src/session/session_salvage.c -src/support/cksum.c -src/support/cond_auto.c -src/support/crypto.c -src/support/err.c -src/support/global.c -src/support/hash_city.c -src/support/hash_fnv.c -src/support/hazard.c -src/support/hex.c -src/support/huffman.c -src/support/mtx_rw.c -src/support/pow.c -src/support/rand.c -src/support/scratch.c -src/support/stat.c -src/txn/txn.c -src/txn/txn_ckpt.c -src/txn/txn_ext.c -src/txn/txn_log.c -src/txn/txn_nsnap.c -src/txn/txn_recover.c diff --git a/src/third_party/wiredtiger/dist/api_data.py b/src/third_party/wiredtiger/dist/api_data.py index 90b1c8378a2..0d995a3e2e2 100644 --- a/src/third_party/wiredtiger/dist/api_data.py +++ b/src/third_party/wiredtiger/dist/api_data.py @@ -420,40 +420,6 @@ connection_runtime_config = [ interval in seconds at which to check for files that are inactive and close them''', min=1, max=100000), ]), - Config('log', '', r''' - enable logging. Enabling logging uses three sessions from the - configured session_max''', - type='category', subconfig=[ - Config('archive', 'true', r''' - automatically archive unneeded log files''', - type='boolean'), - Config('compressor', 'none', r''' - configure a compressor for log records. Permitted values are - \c "none" or custom compression engine name created with - WT_CONNECTION::add_compressor. If WiredTiger has builtin support - for \c "snappy", \c "lz4" or \c "zlib" compression, these names - are also available. See @ref compression for more information'''), - Config('enabled', 'false', r''' - enable logging subsystem''', - type='boolean'), - Config('file_max', '100MB', r''' - the maximum size of log files''', - min='100KB', max='2GB'), - Config('path', '"."', r''' - the path to a directory into which the log files are written. - If the value is not an absolute path name, the files are created - relative to the database home'''), - Config('prealloc', 'true', r''' - pre-allocate log files.''', - type='boolean'), - Config('recover', 'on', r''' - run recovery or error if recovery needs to run after an - unclean shutdown.''', - choices=['error','on']), - Config('zero_fill', 'false', r''' - manually write zeroes into log files''', - type='boolean'), - ]), Config('lsm_manager', '', r''' configure database wide options for LSM tree management. The LSM manager is started automatically the first time an LSM tree is opened. @@ -525,38 +491,6 @@ connection_runtime_config = [ are logged using the \c statistics_log configuration. See @ref statistics for more information''', type='list', choices=['all', 'fast', 'none', 'clear']), - Config('statistics_log', '', r''' - log any statistics the database is configured to maintain, - to a file. See @ref statistics for more information. Enabling - the statistics log server uses a session from the configured - session_max''', - type='category', subconfig=[ - Config('json', 'false', r''' - encode statistics in JSON format''', - type='boolean'), - Config('on_close', 'false', r'''log statistics on database close''', - type='boolean'), - Config('path', '"WiredTigerStat.%d.%H"', r''' - the pathname to a file into which the log records are written, - may contain ISO C standard strftime conversion specifications. - If the value is not an absolute path name, the file is created - relative to the database home'''), - Config('sources', '', r''' - if non-empty, include statistics for the list of data source - URIs, if they are open at the time of the statistics logging. - The list may include URIs matching a single data source - ("table:mytable"), or a URI matching all data sources of a - particular type ("table:")''', - type='list'), - Config('timestamp', '"%b %d %H:%M:%S"', r''' - a timestamp prepended to each log record, may contain strftime - conversion specifications, when \c json is configured, defaults - to \c "%FT%Y.000Z"'''), - Config('wait', '0', r''' - seconds to wait between each write of the log records; setting - this value above 0 configures statistics logging''', - min='0', max='100000'), - ]), Config('verbose', '', r''' enable messages for various events. Only available if WiredTiger is configured with --enable-verbose. Options are given as a @@ -590,13 +524,113 @@ connection_runtime_config = [ 'write']), ] +# wiredtiger_open and WT_CONNECTION.reconfigure log configurations. +log_configuration_common = [ + Config('archive', 'true', r''' + automatically archive unneeded log files''', + type='boolean'), + Config('prealloc', 'true', r''' + pre-allocate log files.''', + type='boolean'), + Config('zero_fill', 'false', r''' + manually write zeroes into log files''', + type='boolean') +] +connection_reconfigure_log_configuration = [ + Config('log', '', r''' + enable logging. Enabling logging uses three sessions from the + configured session_max''', + type='category', subconfig= + log_configuration_common) +] +wiredtiger_open_log_configuration = [ + Config('log', '', r''' + enable logging. Enabling logging uses three sessions from the + configured session_max''', + type='category', subconfig= + log_configuration_common + [ + Config('enabled', 'false', r''' + enable logging subsystem''', + type='boolean'), + Config('compressor', 'none', r''' + configure a compressor for log records. Permitted values are + \c "none" or custom compression engine name created with + WT_CONNECTION::add_compressor. If WiredTiger has builtin support + for \c "snappy", \c "lz4" or \c "zlib" compression, these names + are also available. See @ref compression for more information'''), + Config('file_max', '100MB', r''' + the maximum size of log files''', + min='100KB', max='2GB'), + Config('path', '"."', r''' + the name of a directory into which log files are written. The + directory must already exist. If the value is not an absolute + path, the path is relative to the database home (see @ref + absolute_path for more information)'''), + Config('recover', 'on', r''' + run recovery or error if recovery needs to run after an + unclean shutdown''', + choices=['error','on']) + ]), +] + +# wiredtiger_open and WT_CONNECTION.reconfigure statistics log configurations. +statistics_log_configuration_common = [ + Config('json', 'false', r''' + encode statistics in JSON format''', + type='boolean'), + Config('on_close', 'false', r'''log statistics on database close''', + type='boolean'), + Config('sources', '', r''' + if non-empty, include statistics for the list of data source + URIs, if they are open at the time of the statistics logging. + The list may include URIs matching a single data source + ("table:mytable"), or a URI matching all data sources of a + particular type ("table:")''', + type='list'), + Config('timestamp', '"%b %d %H:%M:%S"', r''' + a timestamp prepended to each log record, may contain strftime + conversion specifications, when \c json is configured, defaults + to \c "%FT%Y.000Z"'''), + Config('wait', '0', r''' + seconds to wait between each write of the log records; setting + this value above 0 configures statistics logging''', + min='0', max='100000'), +] +connection_reconfigure_statistics_log_configuration = [ + Config('statistics_log', '', r''' + log any statistics the database is configured to maintain, + to a file. See @ref statistics for more information. Enabling + the statistics log server uses a session from the configured + session_max''', + type='category', subconfig= + statistics_log_configuration_common) +] +wiredtiger_open_statistics_log_configuration = [ + Config('statistics_log', '', r''' + log any statistics the database is configured to maintain, + to a file. See @ref statistics for more information. Enabling + the statistics log server uses a session from the configured + session_max''', + type='category', subconfig= + statistics_log_configuration_common + [ + Config('path', '"."', r''' + the name of a directory into which statistics files are written. + The directory must already exist. If the value is not an absolute + path, the path is relative to the database home (see @ref + absolute_path for more information)''') + ]) +] + session_config = [ Config('isolation', 'read-committed', r''' the default isolation level for operations in this session''', choices=['read-uncommitted', 'read-committed', 'snapshot']), ] -wiredtiger_open_common = connection_runtime_config + [ +wiredtiger_open_common =\ + connection_runtime_config +\ + wiredtiger_open_log_configuration +\ + wiredtiger_open_statistics_log_configuration + [ Config('buffer_alignment', '-1', r''' in-memory alignment (in bytes) for buffers used for I/O. The default value of -1 indicates a platform-specific alignment value @@ -1084,7 +1118,11 @@ methods = { don't free memory during close''', type='boolean'), ]), -'WT_CONNECTION.reconfigure' : Method(connection_runtime_config), +'WT_CONNECTION.reconfigure' : Method( + connection_reconfigure_log_configuration +\ + connection_reconfigure_statistics_log_configuration +\ + connection_runtime_config +), 'WT_CONNECTION.set_file_system' : Method([]), 'WT_CONNECTION.load_extension' : Method([ diff --git a/src/third_party/wiredtiger/dist/s_string.ok b/src/third_party/wiredtiger/dist/s_string.ok index 7966ff2cf2e..0694c90e5d3 100644 --- a/src/third_party/wiredtiger/dist/s_string.ok +++ b/src/third_party/wiredtiger/dist/s_string.ok @@ -86,6 +86,7 @@ DbEnv Decrement Decrypt DeleteFileA +EACCES EAGAIN EB EBUSY @@ -117,6 +118,7 @@ FLv FNV FORALL FOREACH +FS FULLFSYNC FindClose FindFirstFile @@ -326,6 +328,7 @@ UID UIDs UINT ULINE +UNC URI URIs UTF @@ -528,6 +531,7 @@ cust customp cv cxa +dT data's database's datalen @@ -557,6 +561,7 @@ dequeued der dereference desc +designator dest destSize dev @@ -1065,6 +1070,7 @@ tV tablename tcbench td +tempdir testutil th tid @@ -1091,6 +1097,7 @@ txn txnc txnid txnmin +txt typedef uB uS diff --git a/src/third_party/wiredtiger/examples/c/ex_all.c b/src/third_party/wiredtiger/examples/c/ex_all.c index dd807922c10..e8727df3f60 100644 --- a/src/third_party/wiredtiger/examples/c/ex_all.c +++ b/src/third_party/wiredtiger/examples/c/ex_all.c @@ -1160,34 +1160,27 @@ main(void) if (ret == 0) (void)conn->close(conn, NULL); +#ifdef MIGHT_NOT_RUN + /* + * Don't run this code, statistics logging doesn't yet support tables. + */ /*! [Statistics logging with a table] */ ret = wiredtiger_open(home, NULL, "create, statistics_log=(" - "sources=(\"lsm:table1\",\"lsm:table2\"), wait=5)", + "sources=(\"table:table1\",\"table:table2\"), wait=5)", &conn); /*! [Statistics logging with a table] */ if (ret == 0) (void)conn->close(conn, NULL); - /*! [Statistics logging with all tables] */ - ret = wiredtiger_open(home, NULL, - "create, statistics_log=(sources=(\"lsm:\"), wait=5)", - &conn); - /*! [Statistics logging with all tables] */ - if (ret == 0) - (void)conn->close(conn, NULL); - -#ifdef MIGHT_NOT_RUN /* - * This example code gets run, and a non-existent log file path might - * cause the open to fail. The documentation requires code snippets, - * use #ifdef's to avoid running it. + * Don't run this code, statistics logging doesn't yet support indexes. */ - /*! [Statistics logging with path] */ + /*! [Statistics logging with a source type] */ ret = wiredtiger_open(home, NULL, - "create," - "statistics_log=(wait=120,path=/log/log.%m.%d.%y)", &conn); - /*! [Statistics logging with path] */ + "create, statistics_log=(sources=(\"index:\"), wait=5)", + &conn); + /*! [Statistics logging with a source type] */ if (ret == 0) (void)conn->close(conn, NULL); diff --git a/src/third_party/wiredtiger/examples/c/ex_config.c b/src/third_party/wiredtiger/examples/c/ex_config.c deleted file mode 100644 index 2ac8198176c..00000000000 --- a/src/third_party/wiredtiger/examples/c/ex_config.c +++ /dev/null @@ -1,91 +0,0 @@ -/*- - * Public Domain 2014-2016 MongoDB, Inc. - * Public Domain 2008-2014 WiredTiger, Inc. - * - * This is free and unencumbered software released into the public domain. - * - * Anyone is free to copy, modify, publish, use, compile, sell, or - * distribute this software, either in source code form or as a compiled - * binary, for any purpose, commercial or non-commercial, and by any - * means. - * - * In jurisdictions that recognize copyright laws, the author or authors - * of this software dedicate any and all copyright interest in the - * software to the public domain. We make this dedication for the benefit - * of the public at large and to the detriment of our heirs and - * successors. We intend this dedication to be an overt act of - * relinquishment in perpetuity of all present and future rights to this - * software under copyright law. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, - * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - * IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR - * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - * OTHER DEALINGS IN THE SOFTWARE. - * - * ex_config.c - * This is an example demonstrating how to configure various database and - * table properties. - */ - -#include <stdio.h> -#include <stdlib.h> -#include <string.h> - -#include <wiredtiger.h> - -static const char *home; - -int -main(void) -{ - int ret; - WT_CONNECTION *conn; - WT_SESSION *session; - WT_CURSOR *cursor; - const char *key, *value; - - /* - * Create a clean test directory for this run of the test program if the - * environment variable isn't already set (as is done by make check). - */ - if (getenv("WIREDTIGER_HOME") == NULL) { - home = "WT_HOME"; - ret = system("rm -rf WT_HOME && mkdir WT_HOME"); - } else - home = NULL; - - /*! [configure cache size] */ - if ((ret = wiredtiger_open(home, NULL, - "create,cache_size=500M", &conn)) != 0) - fprintf(stderr, "Error connecting to %s: %s\n", - home, wiredtiger_strerror(ret)); - /*! [configure cache size] */ - - /*! [create a table] */ - ret = conn->open_session(conn, NULL, NULL, &session); - - ret = session->create(session, - "table:access", "key_format=S,value_format=S"); - /*! [create a table] */ - - /*! [transaction] */ - ret = session->begin_transaction(session, "priority=100,name=mytxn"); - - ret = session->open_cursor(session, "config:", NULL, NULL, &cursor); - - while ((ret = cursor->next(cursor)) == 0) { - ret = cursor->get_key(cursor, &key); - ret = cursor->get_value(cursor, &value); - printf("configuration value: %s = %s\n", key, value); - } - - ret = session->commit_transaction(session, NULL); - /*! [transaction] */ - - ret = conn->close(conn, NULL); - - return (ret); -} diff --git a/src/third_party/wiredtiger/examples/c/ex_file_system.c b/src/third_party/wiredtiger/examples/c/ex_file_system.c index 77e8f40480b..55ee20e9331 100644 --- a/src/third_party/wiredtiger/examples/c/ex_file_system.c +++ b/src/third_party/wiredtiger/examples/c/ex_file_system.c @@ -118,18 +118,17 @@ int demo_file_system_create(WT_CONNECTION *, WT_CONFIG_ARG *); /* * Forward function declarations for file system API implementation */ -static int demo_fs_open(WT_FILE_SYSTEM *, - WT_SESSION *, const char *, WT_OPEN_FILE_TYPE, uint32_t, WT_FILE_HANDLE **); +static int demo_fs_open(WT_FILE_SYSTEM *, WT_SESSION *, + const char *, WT_FS_OPEN_FILE_TYPE, uint32_t, WT_FILE_HANDLE **); static int demo_fs_directory_list(WT_FILE_SYSTEM *, WT_SESSION *, const char *, const char *, char ***, uint32_t *); static int demo_fs_directory_list_free( WT_FILE_SYSTEM *, WT_SESSION *, char **, uint32_t); -static int demo_fs_directory_sync(WT_FILE_SYSTEM *file_system, - WT_SESSION *session, const char *directory); static int demo_fs_exist(WT_FILE_SYSTEM *, WT_SESSION *, const char *, bool *); -static int demo_fs_remove(WT_FILE_SYSTEM *, WT_SESSION *, const char *); +static int demo_fs_remove( + WT_FILE_SYSTEM *, WT_SESSION *, const char *, uint32_t); static int demo_fs_rename( - WT_FILE_SYSTEM *, WT_SESSION *, const char *, const char *); + WT_FILE_SYSTEM *, WT_SESSION *, const char *, const char *, uint32_t); static int demo_fs_size( WT_FILE_SYSTEM *, WT_SESSION *, const char *, wt_off_t *); static int demo_fs_terminate(WT_FILE_SYSTEM *, WT_SESSION *); @@ -255,7 +254,6 @@ demo_file_system_create(WT_CONNECTION *conn, WT_CONFIG_ARG *config) /* Initialize the in-memory jump table. */ file_system->fs_directory_list = demo_fs_directory_list; file_system->fs_directory_list_free = demo_fs_directory_list_free; - file_system->fs_directory_sync = demo_fs_directory_sync; file_system->fs_exist = demo_fs_exist; file_system->fs_open_file = demo_fs_open; file_system->fs_remove = demo_fs_remove; @@ -282,7 +280,7 @@ err: free(demo_fs); */ static int demo_fs_open(WT_FILE_SYSTEM *file_system, WT_SESSION *session, - const char *name, WT_OPEN_FILE_TYPE file_type, uint32_t flags, + const char *name, WT_FS_OPEN_FILE_TYPE file_type, uint32_t flags, WT_FILE_HANDLE **file_handlep) { DEMO_FILE_HANDLE *demo_fh; @@ -469,21 +467,6 @@ demo_fs_directory_list_free(WT_FILE_SYSTEM *file_system, } /* - * demo_fs_directory_sync -- - * Directory sync for our demo file system, which is a no-op. - */ -static int -demo_fs_directory_sync(WT_FILE_SYSTEM *file_system, - WT_SESSION *session, const char *directory) -{ - (void)file_system; /* Unused */ - (void)session; /* Unused */ - (void)directory; /* Unused */ - - return (0); -} - -/* * demo_fs_exist -- * Return if the file exists. */ @@ -507,13 +490,15 @@ demo_fs_exist(WT_FILE_SYSTEM *file_system, * POSIX remove. */ static int -demo_fs_remove( - WT_FILE_SYSTEM *file_system, WT_SESSION *session, const char *name) +demo_fs_remove(WT_FILE_SYSTEM *file_system, + WT_SESSION *session, const char *name, uint32_t flags) { DEMO_FILE_SYSTEM *demo_fs; DEMO_FILE_HANDLE *demo_fh; int ret = 0; + (void)flags; /* Unused */ + demo_fs = (DEMO_FILE_SYSTEM *)file_system; ret = ENOENT; @@ -531,13 +516,15 @@ demo_fs_remove( */ static int demo_fs_rename(WT_FILE_SYSTEM *file_system, - WT_SESSION *session, const char *from, const char *to) + WT_SESSION *session, const char *from, const char *to, uint32_t flags) { DEMO_FILE_HANDLE *demo_fh; DEMO_FILE_SYSTEM *demo_fs; char *copy; int ret = 0; + (void)flags; /* Unused */ + demo_fs = (DEMO_FILE_SYSTEM *)file_system; LOCK_FILE_SYSTEM(session, demo_fs); diff --git a/src/third_party/wiredtiger/examples/java/com/wiredtiger/examples/ex_all.java b/src/third_party/wiredtiger/examples/java/com/wiredtiger/examples/ex_all.java index 48e85c9fade..83a37e9a6a5 100644 --- a/src/third_party/wiredtiger/examples/java/com/wiredtiger/examples/ex_all.java +++ b/src/third_party/wiredtiger/examples/java/com/wiredtiger/examples/ex_all.java @@ -988,6 +988,10 @@ allExample() /*! [Statistics logging] */ conn.close(null); + if (false) { // MIGHT_NOT_RUN + /* + * Don't run this code, statistics logging doesn't yet support tables. + */ /*! [Statistics logging with a table] */ conn = wiredtiger.open(home, "create," + @@ -995,23 +999,13 @@ allExample() /*! [Statistics logging with a table] */ conn.close(null); - /*! [Statistics logging with all tables] */ - conn = wiredtiger.open(home, - "create,statistics_log=(sources=(\"table:\"))"); - /*! [Statistics logging with all tables] */ - conn.close(null); - - if (false) { // MIGHT_NOT_RUN /* - * This example code gets run, and a non-existent log file path might - * cause the open to fail. The documentation requires code snippets, - * use if (false) to avoid running it. + * Don't run this code, statistics logging doesn't yet support indexes. */ - /*! [Statistics logging with path] */ + /*! [Statistics logging with a source type] */ conn = wiredtiger.open(home, - "create," + - "statistics_log=(wait=120,path=/log/log.%m.%d.%y)"); - /*! [Statistics logging with path] */ + "create,statistics_log=(sources=(\"index:\"))"); + /*! [Statistics logging with a source type] */ conn.close(null); /* diff --git a/src/third_party/wiredtiger/src/block/block_open.c b/src/third_party/wiredtiger/src/block/block_open.c index 1603b1574e7..7cff7eab629 100644 --- a/src/third_party/wiredtiger/src/block/block_open.c +++ b/src/third_party/wiredtiger/src/block/block_open.c @@ -15,9 +15,10 @@ static int __desc_read(WT_SESSION_IMPL *, WT_BLOCK *); * Drop a file. */ int -__wt_block_manager_drop(WT_SESSION_IMPL *session, const char *filename) +__wt_block_manager_drop( + WT_SESSION_IMPL *session, const char *filename, bool durable) { - return (__wt_remove_if_exists(session, filename)); + return (__wt_remove_if_exists(session, filename, durable)); } /* @@ -43,8 +44,9 @@ __wt_block_manager_create( * in our space. Move any existing files out of the way and complain. */ for (;;) { - if ((ret = __wt_open(session, filename, WT_OPEN_FILE_TYPE_DATA, - WT_OPEN_CREATE | WT_OPEN_EXCLUSIVE, &fh)) == 0) + if ((ret = __wt_open(session, filename, + WT_FS_OPEN_FILE_TYPE_DATA, WT_FS_OPEN_CREATE | + WT_FS_OPEN_DURABLE | WT_FS_OPEN_EXCLUSIVE, &fh)) == 0) break; WT_ERR_TEST(ret != EEXIST, ret); @@ -56,7 +58,7 @@ __wt_block_manager_create( WT_ERR(__wt_fs_exist(session, tmp->data, &exists)); if (!exists) { WT_ERR(__wt_fs_rename( - session, filename, tmp->data)); + session, filename, tmp->data, false)); WT_ERR(__wt_msg(session, "unexpected file %s found, renamed to %s", filename, (const char *)tmp->data)); @@ -77,16 +79,9 @@ __wt_block_manager_create( /* Close the file handle. */ WT_TRET(__wt_close(session, &fh)); - /* - * Some filesystems require that we sync the directory to be confident - * that the file will appear. - */ - if (ret == 0) - WT_TRET(__wt_fs_directory_sync(session, filename)); - /* Undo any create on error. */ if (ret != 0) - WT_TRET(__wt_fs_remove(session, filename)); + WT_TRET(__wt_fs_remove(session, filename, false)); err: __wt_scr_free(session, &tmp); @@ -207,11 +202,11 @@ __wt_block_open(WT_SESSION_IMPL *session, */ flags = 0; if (readonly && FLD_ISSET(conn->direct_io, WT_DIRECT_IO_CHECKPOINT)) - LF_SET(WT_OPEN_DIRECTIO); + LF_SET(WT_FS_OPEN_DIRECTIO); if (!readonly && FLD_ISSET(conn->direct_io, WT_DIRECT_IO_DATA)) - LF_SET(WT_OPEN_DIRECTIO); + LF_SET(WT_FS_OPEN_DIRECTIO); WT_ERR(__wt_open( - session, filename, WT_OPEN_FILE_TYPE_DATA, flags, &block->fh)); + session, filename, WT_FS_OPEN_FILE_TYPE_DATA, flags, &block->fh)); /* Set the file's size. */ WT_ERR(__wt_filesize(session, block->fh, &block->size)); diff --git a/src/third_party/wiredtiger/src/btree/bt_curnext.c b/src/third_party/wiredtiger/src/btree/bt_curnext.c index 70b3ba56e31..e6f4728a695 100644 --- a/src/third_party/wiredtiger/src/btree/bt_curnext.c +++ b/src/third_party/wiredtiger/src/btree/bt_curnext.c @@ -183,6 +183,7 @@ __cursor_var_next(WT_CURSOR_BTREE *cbt, bool newpage) if (cbt->last_standard_recno == 0) return (WT_NOTFOUND); __cursor_set_recno(cbt, cbt->ref->ref_recno); + cbt->cip_saved = NULL; goto new_page; } @@ -301,12 +302,13 @@ __cursor_row_next(WT_CURSOR_BTREE *cbt, bool newpage) * WT_INSERT_HEAD[0], and so on. This means WT_INSERT lists are * odd-numbered slots, and WT_ROW array slots are even-numbered slots. * - * New page configuration. + * Initialize for each new page. */ if (newpage) { cbt->ins_head = WT_ROW_INSERT_SMALLEST(page); cbt->ins = WT_SKIP_FIRST(cbt->ins_head); cbt->row_iteration_slot = 1; + cbt->rip_saved = NULL; goto new_insert; } @@ -517,11 +519,13 @@ __wt_btcur_iterate_setup(WT_CURSOR_BTREE *cbt) */ F_SET(cbt, WT_CBT_ITERATE_NEXT | WT_CBT_ITERATE_PREV); - /* - * Clear the count of deleted items on the page. - */ + /* Clear the count of deleted items on the page. */ cbt->page_deleted_count = 0; + /* Clear saved iteration cursor position information. */ + cbt->cip_saved = NULL; + cbt->rip_saved = NULL; + /* * If we don't have a search page, then we're done, we're starting at * the beginning or end of the tree, not as a result of a search. diff --git a/src/third_party/wiredtiger/src/btree/bt_curprev.c b/src/third_party/wiredtiger/src/btree/bt_curprev.c index 872f648446c..cba0c053b90 100644 --- a/src/third_party/wiredtiger/src/btree/bt_curprev.c +++ b/src/third_party/wiredtiger/src/btree/bt_curprev.c @@ -329,6 +329,7 @@ __cursor_var_prev(WT_CURSOR_BTREE *cbt, bool newpage) if (cbt->last_standard_recno == 0) return (WT_NOTFOUND); __cursor_set_recno(cbt, cbt->last_standard_recno); + cbt->cip_saved = NULL; goto new_page; } @@ -447,7 +448,7 @@ __cursor_row_prev(WT_CURSOR_BTREE *cbt, bool newpage) * WT_INSERT_HEAD[0], and so on. This means WT_INSERT lists are * odd-numbered slots, and WT_ROW array slots are even-numbered slots. * - * New page configuration. + * Initialize for each new page. */ if (newpage) { /* @@ -464,6 +465,7 @@ __cursor_row_prev(WT_CURSOR_BTREE *cbt, bool newpage) WT_ROW_INSERT_SLOT(page, page->pg_row_entries - 1); cbt->ins = WT_SKIP_LAST(cbt->ins_head); cbt->row_iteration_slot = page->pg_row_entries * 2 + 1; + cbt->rip_saved = NULL; goto new_insert; } diff --git a/src/third_party/wiredtiger/src/btree/bt_huffman.c b/src/third_party/wiredtiger/src/btree/bt_huffman.c index 9e9d69c342e..918791d9c6e 100644 --- a/src/third_party/wiredtiger/src/btree/bt_huffman.c +++ b/src/third_party/wiredtiger/src/btree/bt_huffman.c @@ -157,7 +157,8 @@ __huffman_confchk_file(WT_SESSION_IMPL *session, /* Check the file exists. */ WT_RET(__wt_strndup(session, v->str + len, v->len - len, &fname)); - WT_ERR(__wt_fopen(session, fname, WT_OPEN_FIXED, WT_STREAM_READ, &fs)); + WT_ERR(__wt_fopen( + session, fname, WT_FS_OPEN_FIXED, WT_STREAM_READ, &fs)); /* Optionally return the file handle. */ if (fsp == NULL) diff --git a/src/third_party/wiredtiger/src/config/config_def.c b/src/third_party/wiredtiger/src/config/config_def.c index 1b656c5a0aa..352464ee92a 100644 --- a/src/third_party/wiredtiger/src/config/config_def.c +++ b/src/third_party/wiredtiger/src/config/config_def.c @@ -67,16 +67,9 @@ static const WT_CONFIG_CHECK }; static const WT_CONFIG_CHECK - confchk_wiredtiger_open_log_subconfigs[] = { + confchk_WT_CONNECTION_reconfigure_log_subconfigs[] = { { "archive", "boolean", NULL, NULL, NULL, 0 }, - { "compressor", "string", NULL, NULL, NULL, 0 }, - { "enabled", "boolean", NULL, NULL, NULL, 0 }, - { "file_max", "int", NULL, "min=100KB,max=2GB", NULL, 0 }, - { "path", "string", NULL, NULL, NULL, 0 }, { "prealloc", "boolean", NULL, NULL, NULL, 0 }, - { "recover", "string", - NULL, "choices=[\"error\",\"on\"]", - NULL, 0 }, { "zero_fill", "boolean", NULL, NULL, NULL, 0 }, { NULL, NULL, NULL, NULL, NULL, 0 } }; @@ -99,10 +92,9 @@ static const WT_CONFIG_CHECK }; static const WT_CONFIG_CHECK - confchk_wiredtiger_open_statistics_log_subconfigs[] = { + confchk_WT_CONNECTION_reconfigure_statistics_log_subconfigs[] = { { "json", "boolean", NULL, NULL, NULL, 0 }, { "on_close", "boolean", NULL, NULL, NULL, 0 }, - { "path", "string", NULL, NULL, NULL, 0 }, { "sources", "list", NULL, NULL, NULL, 0 }, { "timestamp", "string", NULL, NULL, NULL, 0 }, { "wait", "int", NULL, "min=0,max=100000", NULL, 0 }, @@ -135,7 +127,7 @@ static const WT_CONFIG_CHECK confchk_WT_CONNECTION_reconfigure[] = { confchk_wiredtiger_open_file_manager_subconfigs, 3 }, { "log", "category", NULL, NULL, - confchk_wiredtiger_open_log_subconfigs, 8 }, + confchk_WT_CONNECTION_reconfigure_log_subconfigs, 3 }, { "lsm_manager", "category", NULL, NULL, confchk_wiredtiger_open_lsm_manager_subconfigs, 2 }, @@ -148,7 +140,7 @@ static const WT_CONFIG_CHECK confchk_WT_CONNECTION_reconfigure[] = { NULL, 0 }, { "statistics_log", "category", NULL, NULL, - confchk_wiredtiger_open_statistics_log_subconfigs, 6 }, + confchk_WT_CONNECTION_reconfigure_statistics_log_subconfigs, 5 }, { "verbose", "list", NULL, "choices=[\"api\",\"block\",\"checkpoint\",\"compact\"," "\"evict\",\"evictserver\",\"fileops\",\"handleops\",\"log\"," @@ -608,6 +600,32 @@ static const WT_CONFIG_CHECK }; static const WT_CONFIG_CHECK + confchk_wiredtiger_open_log_subconfigs[] = { + { "archive", "boolean", NULL, NULL, NULL, 0 }, + { "compressor", "string", NULL, NULL, NULL, 0 }, + { "enabled", "boolean", NULL, NULL, NULL, 0 }, + { "file_max", "int", NULL, "min=100KB,max=2GB", NULL, 0 }, + { "path", "string", NULL, NULL, NULL, 0 }, + { "prealloc", "boolean", NULL, NULL, NULL, 0 }, + { "recover", "string", + NULL, "choices=[\"error\",\"on\"]", + NULL, 0 }, + { "zero_fill", "boolean", NULL, NULL, NULL, 0 }, + { NULL, NULL, NULL, NULL, NULL, 0 } +}; + +static const WT_CONFIG_CHECK + confchk_wiredtiger_open_statistics_log_subconfigs[] = { + { "json", "boolean", NULL, NULL, NULL, 0 }, + { "on_close", "boolean", NULL, NULL, NULL, 0 }, + { "path", "string", NULL, NULL, NULL, 0 }, + { "sources", "list", NULL, NULL, NULL, 0 }, + { "timestamp", "string", NULL, NULL, NULL, 0 }, + { "wait", "int", NULL, "min=0,max=100000", NULL, 0 }, + { NULL, NULL, NULL, NULL, NULL, 0 } +}; + +static const WT_CONFIG_CHECK confchk_wiredtiger_open_transaction_sync_subconfigs[] = { { "enabled", "boolean", NULL, NULL, NULL, 0 }, { "method", "string", @@ -975,12 +993,10 @@ static const WT_CONFIG_ENTRY config_entries[] = { "eviction=(threads_max=1,threads_min=1),eviction_dirty_target=80," "eviction_dirty_trigger=95,eviction_target=80,eviction_trigger=95" ",file_manager=(close_handle_minimum=250,close_idle_time=30," - "close_scan_interval=10),log=(archive=,compressor=,enabled=0," - "file_max=100MB,path=\".\",prealloc=,recover=on,zero_fill=0)," + "close_scan_interval=10),log=(archive=,prealloc=,zero_fill=0)," "lsm_manager=(merge=,worker_thread_max=4),lsm_merge=," "shared_cache=(chunk=10MB,name=,quota=0,reserve=0,size=500MB)," - "statistics=none,statistics_log=(json=0,on_close=0," - "path=\"WiredTigerStat.%d.%H\",sources=," + "statistics=none,statistics_log=(json=0,on_close=0,sources=," "timestamp=\"%b %d %H:%M:%S\",wait=0),verbose=", confchk_WT_CONNECTION_reconfigure, 18 }, @@ -1181,11 +1197,10 @@ static const WT_CONFIG_ENTRY config_entries[] = { ",worker_thread_max=4),lsm_merge=,mmap=,multiprocess=0,readonly=0" ",session_max=100,session_scratch_max=2MB," "shared_cache=(chunk=10MB,name=,quota=0,reserve=0,size=500MB)," - "statistics=none,statistics_log=(json=0,on_close=0," - "path=\"WiredTigerStat.%d.%H\",sources=," - "timestamp=\"%b %d %H:%M:%S\",wait=0),transaction_sync=(enabled=0" - ",method=fsync),use_environment=,use_environment_priv=0,verbose=," - "write_through=", + "statistics=none,statistics_log=(json=0,on_close=0,path=\".\"," + "sources=,timestamp=\"%b %d %H:%M:%S\",wait=0)," + "transaction_sync=(enabled=0,method=fsync),use_environment=," + "use_environment_priv=0,verbose=,write_through=", confchk_wiredtiger_open, 38 }, { "wiredtiger_open_all", @@ -1203,11 +1218,11 @@ static const WT_CONFIG_ENTRY config_entries[] = { ",worker_thread_max=4),lsm_merge=,mmap=,multiprocess=0,readonly=0" ",session_max=100,session_scratch_max=2MB," "shared_cache=(chunk=10MB,name=,quota=0,reserve=0,size=500MB)," - "statistics=none,statistics_log=(json=0,on_close=0," - "path=\"WiredTigerStat.%d.%H\",sources=," - "timestamp=\"%b %d %H:%M:%S\",wait=0),transaction_sync=(enabled=0" - ",method=fsync),use_environment=,use_environment_priv=0,verbose=," - "version=(major=0,minor=0),write_through=", + "statistics=none,statistics_log=(json=0,on_close=0,path=\".\"," + "sources=,timestamp=\"%b %d %H:%M:%S\",wait=0)," + "transaction_sync=(enabled=0,method=fsync),use_environment=," + "use_environment_priv=0,verbose=,version=(major=0,minor=0)," + "write_through=", confchk_wiredtiger_open_all, 39 }, { "wiredtiger_open_basecfg", @@ -1224,10 +1239,9 @@ static const WT_CONFIG_ENTRY config_entries[] = { "worker_thread_max=4),lsm_merge=,mmap=,multiprocess=0,readonly=0," "session_max=100,session_scratch_max=2MB,shared_cache=(chunk=10MB" ",name=,quota=0,reserve=0,size=500MB),statistics=none," - "statistics_log=(json=0,on_close=0,path=\"WiredTigerStat.%d.%H\"," - "sources=,timestamp=\"%b %d %H:%M:%S\",wait=0)," - "transaction_sync=(enabled=0,method=fsync),verbose=," - "version=(major=0,minor=0),write_through=", + "statistics_log=(json=0,on_close=0,path=\".\",sources=," + "timestamp=\"%b %d %H:%M:%S\",wait=0),transaction_sync=(enabled=0" + ",method=fsync),verbose=,version=(major=0,minor=0),write_through=", confchk_wiredtiger_open_basecfg, 33 }, { "wiredtiger_open_usercfg", @@ -1244,10 +1258,9 @@ static const WT_CONFIG_ENTRY config_entries[] = { "worker_thread_max=4),lsm_merge=,mmap=,multiprocess=0,readonly=0," "session_max=100,session_scratch_max=2MB,shared_cache=(chunk=10MB" ",name=,quota=0,reserve=0,size=500MB),statistics=none," - "statistics_log=(json=0,on_close=0,path=\"WiredTigerStat.%d.%H\"," - "sources=,timestamp=\"%b %d %H:%M:%S\",wait=0)," - "transaction_sync=(enabled=0,method=fsync),verbose=," - "write_through=", + "statistics_log=(json=0,on_close=0,path=\".\",sources=," + "timestamp=\"%b %d %H:%M:%S\",wait=0),transaction_sync=(enabled=0" + ",method=fsync),verbose=,write_through=", confchk_wiredtiger_open_usercfg, 32 }, { NULL, NULL, NULL, 0 } diff --git a/src/third_party/wiredtiger/src/conn/conn_api.c b/src/third_party/wiredtiger/src/conn/conn_api.c index 98267eeeb2c..b3092eb4828 100644 --- a/src/third_party/wiredtiger/src/conn/conn_api.c +++ b/src/third_party/wiredtiger/src/conn/conn_api.c @@ -1217,7 +1217,8 @@ __conn_config_file(WT_SESSION_IMPL *session, return (0); /* Open the configuration file. */ - WT_RET(__wt_open(session, filename, WT_OPEN_FILE_TYPE_REGULAR, 0, &fh)); + WT_RET(__wt_open( + session, filename, WT_FS_OPEN_FILE_TYPE_REGULAR, 0, &fh)); WT_ERR(__wt_filesize(session, fh, &size)); if (size == 0) goto err; @@ -1510,8 +1511,8 @@ __conn_single(WT_SESSION_IMPL *session, const char *cfg[]) exist = false; if (!is_create) WT_ERR(__wt_fs_exist(session, WT_WIREDTIGER, &exist)); - ret = __wt_open(session, WT_SINGLETHREAD, WT_OPEN_FILE_TYPE_REGULAR, - is_create || exist ? WT_OPEN_CREATE : 0, &conn->lock_fh); + ret = __wt_open(session, WT_SINGLETHREAD, WT_FS_OPEN_FILE_TYPE_REGULAR, + is_create || exist ? WT_FS_OPEN_CREATE : 0, &conn->lock_fh); /* * If this is a read-only connection and we cannot grab the lock @@ -1563,7 +1564,8 @@ __conn_single(WT_SESSION_IMPL *session, const char *cfg[]) /* We own the lock file, optionally create the WiredTiger file. */ ret = __wt_open(session, WT_WIREDTIGER, - WT_OPEN_FILE_TYPE_REGULAR, is_create ? WT_OPEN_CREATE : 0, &fh); + WT_FS_OPEN_FILE_TYPE_REGULAR, is_create ? WT_FS_OPEN_CREATE : 0, + &fh); /* * If we're read-only, check for handled errors. Even if able to open @@ -1784,7 +1786,7 @@ __conn_write_base_config(WT_SESSION_IMPL *session, const char *cfg[]) * runs. This doesn't matter for correctness, it's just cleaning up * random files. */ - WT_RET(__wt_remove_if_exists(session, WT_BASECONFIG_SET)); + WT_RET(__wt_remove_if_exists(session, WT_BASECONFIG_SET, false)); /* * The base configuration file is only written if creating the database, @@ -1809,7 +1811,7 @@ __conn_write_base_config(WT_SESSION_IMPL *session, const char *cfg[]) return (0); WT_RET(__wt_fopen(session, WT_BASECONFIG_SET, - WT_OPEN_CREATE | WT_OPEN_EXCLUSIVE, WT_STREAM_WRITE, &fs)); + WT_FS_OPEN_CREATE | WT_FS_OPEN_EXCLUSIVE, WT_STREAM_WRITE, &fs)); WT_ERR(__wt_fprintf(session, fs, "%s\n\n", "# Do not modify this file.\n" @@ -1870,7 +1872,8 @@ __conn_write_base_config(WT_SESSION_IMPL *session, const char *cfg[]) if (0) { /* Close open file handle, remove any temporary file. */ err: WT_TRET(__wt_fclose(session, &fs)); - WT_TRET(__wt_remove_if_exists(session, WT_BASECONFIG_SET)); + WT_TRET( + __wt_remove_if_exists(session, WT_BASECONFIG_SET, false)); } __wt_free(session, base_config); diff --git a/src/third_party/wiredtiger/src/conn/conn_ckpt.c b/src/third_party/wiredtiger/src/conn/conn_ckpt.c index a23350a5e46..b1d619dc1e8 100644 --- a/src/third_party/wiredtiger/src/conn/conn_ckpt.c +++ b/src/third_party/wiredtiger/src/conn/conn_ckpt.c @@ -23,37 +23,16 @@ __ckpt_server_config(WT_SESSION_IMPL *session, const char **cfg, bool *startp) WT_DECL_RET; char *p; + *startp = false; + conn = S2C(session); - /* - * The checkpoint configuration requires a wait time and/or a log - * size -- if one is not set, we're not running at all. - * Checkpoints based on log size also require logging be enabled. - */ WT_RET(__wt_config_gets(session, cfg, "checkpoint.wait", &cval)); conn->ckpt_usecs = (uint64_t)cval.val * WT_MILLION; WT_RET(__wt_config_gets(session, cfg, "checkpoint.log_size", &cval)); conn->ckpt_logsize = (wt_off_t)cval.val; - /* Checkpoints are incompatible with in-memory configuration */ - if (conn->ckpt_usecs != 0 || conn->ckpt_logsize != 0) { - WT_RET(__wt_config_gets(session, cfg, "in_memory", &cval)); - if (cval.val != 0) - WT_RET_MSG(session, EINVAL, - "In memory configuration incompatible with " - "checkpoints"); - } - - __wt_log_written_reset(session); - if ((conn->ckpt_usecs == 0 && conn->ckpt_logsize == 0) || - (conn->ckpt_logsize && conn->ckpt_usecs == 0 && - !FLD_ISSET(conn->log_flags, WT_CONN_LOG_ENABLED))) { - *startp = false; - return (0); - } - *startp = true; - /* * The application can specify a checkpoint name, which we ignore if * it's our default. @@ -72,6 +51,26 @@ __ckpt_server_config(WT_SESSION_IMPL *session, const char **cfg, bool *startp) conn->ckpt_config = p; } + /* + * The checkpoint configuration requires a wait time and/or a log size, + * if neither is set, we're not running at all. Checkpoints based on log + * size also require logging be enabled. + */ + if (conn->ckpt_usecs != 0 || + (conn->ckpt_logsize != 0 && + FLD_ISSET(conn->log_flags, WT_CONN_LOG_ENABLED))) { + /* Checkpoints are incompatible with in-memory configuration */ + WT_ERR(__wt_config_gets(session, cfg, "in_memory", &cval)); + if (cval.val != 0) + WT_ERR_MSG(session, EINVAL, + "checkpoint configuration incompatible with " + "in-memory configuration"); + + __wt_log_written_reset(session); + + *startp = true; + } + err: __wt_scr_free(session, &tmp); return (ret); } @@ -179,7 +178,16 @@ __wt_checkpoint_server_create(WT_SESSION_IMPL *session, const char *cfg[]) conn = S2C(session); start = false; - /* If there is already a server running, shut it down. */ + /* + * Stop any server that is already running. This means that each time + * reconfigure is called we'll bounce the server even if there are no + * configuration changes. This makes our life easier as the underlying + * configuration routine doesn't have to worry about freeing objects + * in the connection structure (it's guaranteed to always start with a + * blank slate), and we don't have to worry about races where a running + * server is reading configuration information that we're updating, and + * it's not expected that reconfiguration will happen a lot. + */ if (conn->ckpt_session != NULL) WT_RET(__wt_checkpoint_server_destroy(session)); diff --git a/src/third_party/wiredtiger/src/conn/conn_log.c b/src/third_party/wiredtiger/src/conn/conn_log.c index 1ae370ef2fa..18ed71e4688 100644 --- a/src/third_party/wiredtiger/src/conn/conn_log.c +++ b/src/third_party/wiredtiger/src/conn/conn_log.c @@ -51,6 +51,25 @@ __logmgr_config( WT_CONNECTION_IMPL *conn; bool enabled; + /* + * A note on reconfiguration: the standard "is this configuration string + * allowed" checks should fail if reconfiguration has invalid strings, + * for example, "log=(enabled)", or "statistics_log=(path=XXX)", because + * the connection reconfiguration method doesn't allow those strings. + * Additionally, the base configuration values during reconfiguration + * are the currently configured values (so we don't revert to default + * values when repeatedly reconfiguring), and configuration processing + * of a currently set value should not change the currently set value. + * + * In this code path, log server reconfiguration does not stop/restart + * the log server, so there's no point in re-evaluating configuration + * strings that cannot be reconfigured, risking bugs in configuration + * setup, and depending on evaluation of currently set values to always + * result in the currently set value. Skip tests for any configuration + * strings which don't make sense during reconfiguration, but don't + * worry about error reporting because it should never happen. + */ + conn = S2C(session); WT_RET(__wt_config_gets(session, cfg, "log.enabled", &cval)); @@ -62,6 +81,8 @@ __logmgr_config( * * If it is off and the user it turning it on, or it is on * and the user is turning it off, return an error. + * + * See above: should never happen. */ if (reconfig && ((enabled && !FLD_ISSET(conn->log_flags, WT_CONN_LOG_ENABLED)) || @@ -83,6 +104,8 @@ __logmgr_config( * Setup a log path and compression even if logging is disabled in case * we are going to print a log. Only do this on creation. Once a * compressor or log path are set they cannot be changed. + * + * See above: should never happen. */ if (!reconfig) { conn->log_compressor = NULL; @@ -95,6 +118,7 @@ __logmgr_config( WT_RET(__wt_strndup( session, cval.str, cval.len, &conn->log_path)); } + /* We are done if logging isn't enabled. */ if (!*runp) return (0); @@ -103,13 +127,14 @@ __logmgr_config( if (cval.val != 0) FLD_SET(conn->log_flags, WT_CONN_LOG_ARCHIVE); + /* + * The file size cannot be reconfigured. The amount of memory allocated + * to the log slots may be based on the log file size at creation and we + * don't want to re-allocate that memory while running. + * + * See above: should never happen. + */ if (!reconfig) { - /* - * Ignore if the user tries to change the file size. The - * amount of memory allocated to the log slots may be based - * on the log file size at creation and we don't want to - * re-allocate that memory while running. - */ WT_RET(__wt_config_gets(session, cfg, "log.file_max", &cval)); conn->log_file_max = (wt_off_t)cval.val; WT_STAT_FAST_CONN_SET(session, @@ -125,12 +150,17 @@ __logmgr_config( conn->log_prealloc = 1; /* - * Note that it is meaningless to reconfigure this value during - * runtime. It only matters on create before recovery runs. + * Note it's meaningless to reconfigure this value during runtime, it + * only matters on create before recovery runs. + * + * See above: should never happen. */ - WT_RET(__wt_config_gets_def(session, cfg, "log.recover", 0, &cval)); - if (cval.len != 0 && WT_STRING_MATCH("error", cval.str, cval.len)) - FLD_SET(conn->log_flags, WT_CONN_LOG_RECOVER_ERR); + if (!reconfig) { + WT_RET(__wt_config_gets_def( + session, cfg, "log.recover", 0, &cval)); + if (WT_STRING_MATCH("error", cval.str, cval.len)) + FLD_SET(conn->log_flags, WT_CONN_LOG_RECOVER_ERR); + } WT_RET(__wt_config_gets(session, cfg, "log.zero_fill", &cval)); if (cval.val != 0) { diff --git a/src/third_party/wiredtiger/src/conn/conn_stat.c b/src/third_party/wiredtiger/src/conn/conn_stat.c index 855ff57808e..4e7cac59c4a 100644 --- a/src/third_party/wiredtiger/src/conn/conn_stat.c +++ b/src/third_party/wiredtiger/src/conn/conn_stat.c @@ -36,6 +36,31 @@ __stat_sources_free(WT_SESSION_IMPL *session, char ***sources) } /* + * __stat_config_discard -- + * Discard all statistics-log configuration. + */ +static int +__stat_config_discard(WT_SESSION_IMPL *session) +{ + WT_CONNECTION_IMPL *conn; + WT_DECL_RET; + + conn = S2C(session); + + /* + * Discard all statistics-log configuration information, called when + * reconfiguring or destroying the statistics logging setup, + */ + __wt_free(session, conn->stat_format); + ret = __wt_fclose(session, &conn->stat_fs); + __wt_free(session, conn->stat_path); + __stat_sources_free(session, &conn->stat_sources); + conn->stat_stamp = NULL; + conn->stat_usecs = 0; + return (ret); +} + +/* * __wt_conn_stat_init -- * Initialize the per-connection statistics. */ @@ -73,20 +98,37 @@ __statlog_config(WT_SESSION_IMPL *session, const char **cfg, bool *runp) WT_CONFIG objectconf; WT_CONFIG_ITEM cval, k, v; WT_CONNECTION_IMPL *conn; + WT_DECL_ITEM(tmp); WT_DECL_RET; int cnt; char **sources; + /* + * A note on reconfiguration: the standard "is this configuration string + * allowed" checks should fail if reconfiguration has invalid strings, + * for example, "log=(enabled)", or "statistics_log=(path=XXX)", because + * the connection reconfiguration method doesn't allow those strings. + * Additionally, the base configuration values during reconfiguration + * are the currently configured values (so we don't revert to default + * values when repeatedly reconfiguring), and configuration processing + * of a currently set value should not change the currently set value. + * + * In this code path, a previous statistics log server reconfiguration + * may have stopped the server (and we're about to restart it). Because + * stopping the server discarded the configured information stored in + * the connection structure, we have to re-evaluate all configuration + * values, reconfiguration can't skip any of them. + */ + conn = S2C(session); sources = NULL; - WT_RET(__wt_config_gets(session, cfg, "statistics_log.wait", &cval)); /* Only start the server if wait time is non-zero */ + WT_RET(__wt_config_gets(session, cfg, "statistics_log.wait", &cval)); *runp = cval.val != 0; conn->stat_usecs = (uint64_t)cval.val * WT_MILLION; - WT_RET(__wt_config_gets( - session, cfg, "statistics_log.json", &cval)); + WT_RET(__wt_config_gets(session, cfg, "statistics_log.json", &cval)); if (cval.val != 0) FLD_SET(conn->stat_flags, WT_CONN_STAT_JSON); @@ -96,24 +138,30 @@ __statlog_config(WT_SESSION_IMPL *session, const char **cfg, bool *runp) FLD_SET(conn->stat_flags, WT_CONN_STAT_ON_CLOSE); /* - * Statistics logging configuration requires either a wait time or an - * on-close setting. - */ - if (!*runp && !FLD_ISSET(conn->stat_flags, WT_CONN_STAT_ON_CLOSE)) - return (0); - - /* - * If any statistics logging is done, this must not be a read-only - * connection. + * We don't allow the log path to be reconfigured for security reasons. + * (Applications passing input strings directly to reconfigure would + * expose themselves to a potential security problem, the utility of + * reconfiguring a statistics log path isn't worth the security risk.) + * + * See above for the details, but during reconfiguration we're loading + * the path value from the saved configuration information, and it's + * required during reconfiguration because we potentially stopped and + * are restarting, the server. */ - WT_RET(__wt_config_gets(session, cfg, "statistics_log.sources", &cval)); - WT_RET(__wt_config_subinit(session, &objectconf, &cval)); + WT_RET(__wt_config_gets(session, cfg, "statistics_log.path", &cval)); + WT_ERR(__wt_scr_alloc(session, 0, &tmp)); + WT_ERR(__wt_buf_fmt(session, + tmp, "%.*s/%s", (int)cval.len, cval.str, WT_STATLOG_FILENAME)); + WT_ERR(__wt_filename(session, tmp->data, &conn->stat_path)); + + WT_ERR(__wt_config_gets(session, cfg, "statistics_log.sources", &cval)); + WT_ERR(__wt_config_subinit(session, &objectconf, &cval)); for (cnt = 0; (ret = __wt_config_next(&objectconf, &k, &v)) == 0; ++cnt) ; - WT_RET_NOTFOUND_OK(ret); + WT_ERR_NOTFOUND_OK(ret); if (cnt != 0) { - WT_RET(__wt_calloc_def(session, cnt + 1, &sources)); - WT_RET(__wt_config_subinit(session, &objectconf, &cval)); + WT_ERR(__wt_calloc_def(session, cnt + 1, &sources)); + WT_ERR(__wt_config_subinit(session, &objectconf, &cval)); for (cnt = 0; (ret = __wt_config_next(&objectconf, &k, &v)) == 0; ++cnt) { /* @@ -138,29 +186,37 @@ __statlog_config(WT_SESSION_IMPL *session, const char **cfg, bool *runp) sources = NULL; } - WT_ERR(__wt_config_gets(session, cfg, "statistics_log.path", &cval)); - WT_ERR(__wt_nfilename(session, cval.str, cval.len, &conn->stat_path)); - /* * When using JSON format, use the same timestamp format as MongoDB by - * default. + * default. This requires caution: the user might have set the timestamp + * in a previous reconfigure call and we don't want to override that, so + * compare the retrieved value with the default value to decide if we + * should use the JSON default. + * + * (This still implies if the user explicitly sets the timestamp to the + * default value, then sets the JSON flag in a separate reconfigure + * call, or vice-versa, we will incorrectly switch to the JSON default + * timestamp. But there's no way to detect that, and this is all a low + * probability path.) + * + * !!! + * Don't rewrite in the compressed "%FT%T.000Z" form, MSVC13 segfaults. */ - if (FLD_ISSET(conn->stat_flags, WT_CONN_STAT_JSON)) { - ret = __wt_config_gets( - session, &cfg[1], "statistics_log.timestamp", &cval); - if (ret == WT_NOTFOUND) - WT_ERR(__wt_strdup( - session, "%FT%T.000Z", &conn->stat_format)); - WT_ERR_NOTFOUND_OK(ret); - } - if (conn->stat_format == NULL) { - WT_ERR(__wt_config_gets( - session, cfg, "statistics_log.timestamp", &cval)); +#define WT_TIMESTAMP_DEFAULT "%b %d %H:%M:%S" +#define WT_TIMESTAMP_JSON_DEFAULT "%Y-%m-%dT%H:%M:%S.000Z" + WT_ERR(__wt_config_gets( + session, cfg, "statistics_log.timestamp", &cval)); + if (FLD_ISSET(conn->stat_flags, WT_CONN_STAT_JSON) && + WT_STRING_MATCH(WT_TIMESTAMP_DEFAULT, cval.str, cval.len)) + WT_ERR(__wt_strdup( + session, WT_TIMESTAMP_JSON_DEFAULT, &conn->stat_format)); + else WT_ERR(__wt_strndup( session, cval.str, cval.len, &conn->stat_format)); - } err: __stat_sources_free(session, &sources); + __wt_scr_free(session, &tmp); + return (ret); } @@ -373,7 +429,7 @@ __statlog_log_one(WT_SESSION_IMPL *session, WT_ITEM *path, WT_ITEM *tmp) if (path != NULL) (void)strcpy(path->mem, tmp->mem); WT_RET(__wt_fopen(session, tmp->mem, - WT_OPEN_CREATE | WT_OPEN_FIXED, WT_STREAM_APPEND, + WT_FS_OPEN_CREATE | WT_FS_OPEN_FIXED, WT_STREAM_APPEND, &log_stream)); } conn->stat_fs = log_stream; @@ -538,14 +594,23 @@ __wt_statlog_create(WT_SESSION_IMPL *session, const char *cfg[]) bool start; conn = S2C(session); - start = false; /* * Stop any server that is already running. This means that each time * reconfigure is called we'll bounce the server even if there are no - * configuration changes - but that makes our lives easier. + * configuration changes. This makes our life easier as the underlying + * configuration routine doesn't have to worry about freeing objects + * in the connection structure (it's guaranteed to always start with a + * blank slate), and we don't have to worry about races where a running + * server is reading configuration information that we're updating, and + * it's not expected that reconfiguration will happen a lot. + * + * If there's no server running, discard any configuration information + * so we don't leak memory during reconfiguration. */ - if (conn->stat_session != NULL) + if (conn->stat_session == NULL) + WT_RET(__stat_config_discard(session)); + else WT_RET(__wt_statlog_destroy(session, false)); WT_RET(__statlog_config(session, cfg, &start)); @@ -568,38 +633,28 @@ __wt_statlog_destroy(WT_SESSION_IMPL *session, bool is_close) conn = S2C(session); + /* Stop the server thread. */ F_CLR(conn, WT_CONN_SERVER_STATISTICS); if (conn->stat_tid_set) { WT_TRET(__wt_cond_signal(session, conn->stat_cond)); WT_TRET(__wt_thread_join(session, conn->stat_tid)); conn->stat_tid_set = false; } + WT_TRET(__wt_cond_destroy(session, &conn->stat_cond)); /* Log a set of statistics on shutdown if configured. */ if (is_close) WT_TRET(__wt_statlog_log_one(session)); - WT_TRET(__wt_cond_destroy(session, &conn->stat_cond)); - - __stat_sources_free(session, &conn->stat_sources); - __wt_free(session, conn->stat_path); - __wt_free(session, conn->stat_format); + /* Discard all configuration information. */ + WT_TRET(__stat_config_discard(session)); /* Close the server thread's session. */ if (conn->stat_session != NULL) { wt_session = &conn->stat_session->iface; WT_TRET(wt_session->close(wt_session, NULL)); + conn->stat_session = NULL; } - /* Clear connection settings so reconfigure is reliable. */ - conn->stat_session = NULL; - conn->stat_tid_set = false; - conn->stat_format = NULL; - WT_TRET(__wt_fclose(session, &conn->stat_fs)); - conn->stat_path = NULL; - conn->stat_sources = NULL; - conn->stat_stamp = NULL; - conn->stat_usecs = 0; - return (ret); } diff --git a/src/third_party/wiredtiger/src/cursor/cur_backup.c b/src/third_party/wiredtiger/src/cursor/cur_backup.c index 4ee23008687..f7565729cf9 100644 --- a/src/third_party/wiredtiger/src/cursor/cur_backup.c +++ b/src/third_party/wiredtiger/src/cursor/cur_backup.c @@ -235,7 +235,7 @@ __backup_start( * doesn't confuse restarting in the source database. */ WT_ERR(__wt_fopen(session, WT_BACKUP_TMP, - WT_OPEN_CREATE, WT_STREAM_WRITE, &cb->bfs)); + WT_FS_OPEN_CREATE, WT_STREAM_WRITE, &cb->bfs)); /* * If a list of targets was specified, work our way through them. * Else, generate a list of all database objects. @@ -261,7 +261,7 @@ __backup_start( */ dest = WT_INCREMENTAL_BACKUP; WT_ERR(__wt_fopen(session, WT_INCREMENTAL_SRC, - WT_OPEN_CREATE, WT_STREAM_WRITE, &srcfs)); + WT_FS_OPEN_CREATE, WT_STREAM_WRITE, &srcfs)); WT_ERR(__backup_list_append( session, cb, WT_INCREMENTAL_BACKUP)); } else { @@ -287,7 +287,7 @@ err: /* Close the hot backup file. */ WT_TRET(__backup_stop(session)); } else { WT_ASSERT(session, dest != NULL); - WT_TRET(__wt_fs_rename(session, WT_BACKUP_TMP, dest)); + WT_TRET(__wt_fs_rename(session, WT_BACKUP_TMP, dest, false)); } return (ret); @@ -443,10 +443,10 @@ __wt_backup_file_remove(WT_SESSION_IMPL *session) * always know we were a source directory while there's any chance of * an incremental backup file existing. */ - WT_TRET(__wt_remove_if_exists(session, WT_BACKUP_TMP)); - WT_TRET(__wt_remove_if_exists(session, WT_INCREMENTAL_BACKUP)); - WT_TRET(__wt_remove_if_exists(session, WT_INCREMENTAL_SRC)); - WT_TRET(__wt_remove_if_exists(session, WT_METADATA_BACKUP)); + WT_TRET(__wt_remove_if_exists(session, WT_BACKUP_TMP, true)); + WT_TRET(__wt_remove_if_exists(session, WT_INCREMENTAL_BACKUP, true)); + WT_TRET(__wt_remove_if_exists(session, WT_INCREMENTAL_SRC, true)); + WT_TRET(__wt_remove_if_exists(session, WT_METADATA_BACKUP, true)); return (ret); } diff --git a/src/third_party/wiredtiger/src/docs/security.dox b/src/third_party/wiredtiger/src/docs/security.dox index 331f74d969b..82e13ae7ad3 100644 --- a/src/third_party/wiredtiger/src/docs/security.dox +++ b/src/third_party/wiredtiger/src/docs/security.dox @@ -2,10 +2,23 @@ @section directory_permissions Database directory permissions -All WiredTiger files are stored in the database home directory, and the -WiredTiger database directory should have its permissions set to ensure -database objects are not accessible to users without appropriate -permissions. See @ref home for more information. +By default, WiredTiger files are stored beneath the database home directory. +The WiredTiger database directory should have its permissions set to ensure +database objects are not accessible to users without appropriate permissions. +See @ref home for more information. + +@section absolute_path Absolute paths + +WiredTiger prepends the name of the database home to file names which +do not appear to be absolute paths. (The absolute path test is +simplistic, matching a leading slash character on POSIX systems or a +leading alphabetic character and colon on Windows.) No file path +sanitization or validation is done by WiredTiger, for example, file +paths may match universal naming conventions (UNC), or include \c "../" +(dot dot slash) components. + +Applications are responsible for validating user-supplied file paths as +necessary to prevent directory traversal attacks. @section file_permissions File permissions diff --git a/src/third_party/wiredtiger/src/docs/spell.ok b/src/third_party/wiredtiger/src/docs/spell.ok index 96fe04d7426..e08eb7d1447 100644 --- a/src/third_party/wiredtiger/src/docs/spell.ok +++ b/src/third_party/wiredtiger/src/docs/spell.ok @@ -80,6 +80,7 @@ SCons Seward's SiH TXT +UNC URIs WILLNEED WiredTiger @@ -368,6 +369,7 @@ php png posix pre +prepends primary's printf printlog @@ -411,6 +413,7 @@ runtime rwlock sHQ sHq +sanitization scalable scanf schemas diff --git a/src/third_party/wiredtiger/src/docs/statistics.dox b/src/third_party/wiredtiger/src/docs/statistics.dox index 0a29e351e4e..36ce2711dc5 100644 --- a/src/third_party/wiredtiger/src/docs/statistics.dox +++ b/src/third_party/wiredtiger/src/docs/statistics.dox @@ -90,11 +90,20 @@ cursor. @section statistics_log Statistics logging -WiredTiger will optionally log database statistics into a file when the +WiredTiger will optionally log database statistics into files when the the ::wiredtiger_open \c statistics_log configuration is set. -The resulting statistics can be displayed using the \c wtstats visualization -tool. For more information, see @ref_single wtstats. +The log files are named \c WiredTiger.%%d.%%H, where \c %%d is replaced +with the day of the month as a decimal number (01-31), and \c %%H +is replaced by the hour (24-hour clock) as a decimal number (00-23). +Each log file contains the statistics for the hour specified in its name. + +The location of the log files may be changed with the \c statistics_log.path +configuration string. + +The resulting statistics can be displayed and interactively examined +using the \c wtstats visualization tool. For more information, see +@ref_single wtstats. The following example logs statistics every 30 seconds: @@ -120,7 +129,7 @@ Statistics for all underlying data sources of a particular type may be included by adding a partial data source URI to the \c statistics_log configuration string: -@snippet ex_all.c Statistics logging with all tables +@snippet ex_all.c Statistics logging with a source type When database statistics are logged, the database home will be the first space-separated entry for each record in the log file. For example: @@ -151,23 +160,9 @@ currently open in the database, nor will any statistics requiring the traversal of a tree (as if the \c statistics_fast configuration string were set). -The location of the log files may be changed with the \c statistics_log.path -configuration string. The \c path value value may contain ISO C90 standard -strftime conversion specifications. WiredTiger will not create non-existent -directories in the path, they must exist before ::wiredtiger_open is called. - -The following example logs statistics into files named with the month, -day and year: - -@snippet ex_all.c Statistics logging with path - A Python script that parses the default logging output and uses the <a href="http://www.gnuplot.info/">gnuplot</a>, utility to generate Portable Network Graphics (PNG) format graphs is included in the WiredTiger distribution in the file \c tools/statlog.py. -@m_if{c} -To interactively examine statistics results, see @ref wtstats. -@m_endif - */ diff --git a/src/third_party/wiredtiger/src/docs/upgrading.dox b/src/third_party/wiredtiger/src/docs/upgrading.dox index 5e824fee977..c5fbc0a86a2 100644 --- a/src/third_party/wiredtiger/src/docs/upgrading.dox +++ b/src/third_party/wiredtiger/src/docs/upgrading.dox @@ -1,5 +1,16 @@ /*! @page upgrading Upgrading WiredTiger applications +@section version_281 Upgrading to Version 2.8.1 +<dl> +<dt>Statistics logging path</dt> +<dd> +The statistics logging path configuration has been simplified to be only a +path to a directory, and the file name component of the path may no longer +be specified. Applications depending on the ability to set statistics log +file names will require modifications. +</dd> + +</dl><hr> @section version_280 Upgrading to Version 2.8.0 <dl> <dt>LSM metadata</dt> @@ -55,7 +66,6 @@ The WiredTiger public API used to define a structure that could encapsulate log sequence numbers. That structure is no longer exposed publicly. </dd> -<dt> </dl><hr> @section version_270 Upgrading to Version 2.7.0 diff --git a/src/third_party/wiredtiger/src/evict/evict_page.c b/src/third_party/wiredtiger/src/evict/evict_page.c index 305b81fe69e..0d26c1a2a84 100644 --- a/src/third_party/wiredtiger/src/evict/evict_page.c +++ b/src/third_party/wiredtiger/src/evict/evict_page.c @@ -145,6 +145,18 @@ done: if (((inmem_split && ret == 0) || (forced_eviction && ret == EBUSY)) && WT_TRET(__wt_evict_server_wake(session)); } + /* + * When application threads perform eviction, we don't want to cache + * reconciliation structures. + */ + if (!F_ISSET(session, WT_SESSION_INTERNAL)) { + if (session->block_manager_cleanup != NULL) + WT_TRET(session->block_manager_cleanup(session)); + + if (session->reconcile_cleanup != NULL) + WT_TRET(session->reconcile_cleanup(session)); + } + return (ret); } /* diff --git a/src/third_party/wiredtiger/src/include/connection.h b/src/third_party/wiredtiger/src/include/connection.h index 0e0c357279a..804498f6869 100644 --- a/src/third_party/wiredtiger/src/include/connection.h +++ b/src/third_party/wiredtiger/src/include/connection.h @@ -314,6 +314,7 @@ struct __wt_connection_impl { uint32_t evict_workers; /* Number of eviction workers */ WT_EVICT_WORKER *evict_workctx; /* Eviction worker context */ +#define WT_STATLOG_FILENAME "WiredTigerStat.%d.%H" WT_SESSION_IMPL *stat_session; /* Statistics log session */ wt_thread_t stat_tid; /* Statistics log thread */ bool stat_tid_set; /* Statistics log thread set */ diff --git a/src/third_party/wiredtiger/src/include/cursor.i b/src/third_party/wiredtiger/src/include/cursor.i index 553dd03f958..c1fb7bfcb9f 100644 --- a/src/third_party/wiredtiger/src/include/cursor.i +++ b/src/third_party/wiredtiger/src/include/cursor.i @@ -38,9 +38,6 @@ __cursor_pos_clear(WT_CURSOR_BTREE *cbt) cbt->ins_head = NULL; cbt->ins_stack[0] = NULL; - cbt->cip_saved = NULL; - cbt->rip_saved = NULL; - F_CLR(cbt, WT_CBT_POSITION_MASK); } diff --git a/src/third_party/wiredtiger/src/include/extern.h b/src/third_party/wiredtiger/src/include/extern.h index b0c0f6eccad..20329e9d9b7 100644 --- a/src/third_party/wiredtiger/src/include/extern.h +++ b/src/third_party/wiredtiger/src/include/extern.h @@ -44,7 +44,7 @@ extern void __wt_block_extlist_free(WT_SESSION_IMPL *session, WT_EXTLIST *el); extern int __wt_block_map(WT_SESSION_IMPL *session, WT_BLOCK *block, void *mapped_regionp, size_t *lengthp, void *mapped_cookiep); extern int __wt_block_unmap(WT_SESSION_IMPL *session, WT_BLOCK *block, void *mapped_region, size_t length, void *mapped_cookie); extern int __wt_block_manager_open(WT_SESSION_IMPL *session, const char *filename, const char *cfg[], bool forced_salvage, bool readonly, uint32_t allocsize, WT_BM **bmp); -extern int __wt_block_manager_drop(WT_SESSION_IMPL *session, const char *filename); +extern int __wt_block_manager_drop( WT_SESSION_IMPL *session, const char *filename, bool durable); extern int __wt_block_manager_create( WT_SESSION_IMPL *session, const char *filename, uint32_t allocsize); extern void __wt_block_configure_first_fit(WT_BLOCK *block, bool on); extern int __wt_block_open(WT_SESSION_IMPL *session, const char *filename, const char *cfg[], bool forced_salvage, bool readonly, uint32_t allocsize, WT_BLOCK **blockp); @@ -485,8 +485,7 @@ extern int __wt_turtle_read(WT_SESSION_IMPL *session, const char *key, char **va extern int __wt_turtle_update(WT_SESSION_IMPL *session, const char *key, const char *value); extern int __wt_filename(WT_SESSION_IMPL *session, const char *name, char **path); extern int __wt_nfilename( WT_SESSION_IMPL *session, const char *name, size_t namelen, char **path); -extern int __wt_remove_if_exists(WT_SESSION_IMPL *session, const char *name); -extern int __wt_rename_and_sync_directory( WT_SESSION_IMPL *session, const char *from, const char *to); +extern int __wt_remove_if_exists(WT_SESSION_IMPL *session, const char *name, bool durable); extern int __wt_copy_and_sync(WT_SESSION *wt_session, const char *from, const char *to); extern void __wt_abort(WT_SESSION_IMPL *session) WT_GCC_FUNC_DECL_ATTRIBUTE((noreturn)); extern int __wt_calloc(WT_SESSION_IMPL *session, size_t number, size_t size, void *retp); @@ -500,7 +499,7 @@ extern int __wt_errno(void); extern const char *__wt_strerror(WT_SESSION_IMPL *session, int error, char *errbuf, size_t errlen); extern int __wt_ext_map_windows_error( WT_EXTENSION_API *wt_api, WT_SESSION *wt_session, uint32_t windows_error); extern bool __wt_handle_is_open(WT_SESSION_IMPL *session, const char *name); -extern int __wt_open(WT_SESSION_IMPL *session, const char *name, WT_OPEN_FILE_TYPE file_type, u_int flags, WT_FH **fhp); +extern int __wt_open(WT_SESSION_IMPL *session, const char *name, WT_FS_OPEN_FILE_TYPE file_type, u_int flags, WT_FH **fhp); extern int __wt_close(WT_SESSION_IMPL *session, WT_FH **fhp); extern int __wt_close_connection_close(WT_SESSION_IMPL *session); extern int __wt_os_inmemory(WT_SESSION_IMPL *session); diff --git a/src/third_party/wiredtiger/src/include/os_fs.i b/src/third_party/wiredtiger/src/include/os_fs.i index 88ee71d953a..a3a2fe29b65 100644 --- a/src/third_party/wiredtiger/src/include/os_fs.i +++ b/src/third_party/wiredtiger/src/include/os_fs.i @@ -8,7 +8,7 @@ /* * __wt_fs_directory_list -- - * Get a list of files from a directory. + * Return a list of files from a directory. */ static inline int __wt_fs_directory_list(WT_SESSION_IMPL *session, @@ -61,61 +61,6 @@ __wt_fs_directory_list_free( } /* - * __wt_fs_directory_sync -- - * Flush a directory to ensure file creation is durable. - */ -static inline int -__wt_fs_directory_sync(WT_SESSION_IMPL *session, const char *name) -{ - WT_DECL_RET; - WT_FILE_SYSTEM *file_system; - WT_SESSION *wt_session; - char *copy, *dir; - - WT_ASSERT(session, !F_ISSET(S2C(session), WT_CONN_READONLY)); - - WT_RET(__wt_verbose( - session, WT_VERB_FILEOPS, "%s: directory-sync", name)); - - /* - * POSIX 1003.1 does not require that fsync of a file handle ensures the - * entry in the directory containing the file has also reached disk (and - * there are historic Linux filesystems requiring it). If the underlying - * filesystem method is set, do an explicit fsync on a file descriptor - * for the directory to be sure. - * - * directory-sync is not a required call, no method means the call isn't - * needed. - */ - file_system = S2C(session)->file_system; - if (file_system->fs_directory_sync == NULL) - return (0); - - copy = NULL; - if (name == NULL || strchr(name, '/') == NULL) - name = S2C(session)->home; - else { - /* - * File name construction should not return a path without any - * slash separator, but caution isn't unreasonable. - */ - WT_RET(__wt_filename(session, name, ©)); - if ((dir = strrchr(copy, '/')) == NULL) - name = S2C(session)->home; - else { - dir[1] = '\0'; - name = copy; - } - } - - wt_session = (WT_SESSION *)session; - ret = file_system->fs_directory_sync(file_system, wt_session, name); - - __wt_free(session, copy); - return (ret); -} - -/* * __wt_fs_exist -- * Return if the file exists. */ @@ -141,10 +86,10 @@ __wt_fs_exist(WT_SESSION_IMPL *session, const char *name, bool *existp) /* * __wt_fs_remove -- - * POSIX remove. + * Remove the file. */ static inline int -__wt_fs_remove(WT_SESSION_IMPL *session, const char *name) +__wt_fs_remove(WT_SESSION_IMPL *session, const char *name, bool durable) { WT_DECL_RET; WT_FILE_SYSTEM *file_system; @@ -169,7 +114,8 @@ __wt_fs_remove(WT_SESSION_IMPL *session, const char *name) file_system = S2C(session)->file_system; wt_session = (WT_SESSION *)session; - ret = file_system->fs_remove(file_system, wt_session, path); + ret = file_system->fs_remove( + file_system, wt_session, path, durable ? WT_FS_DURABLE : 0); __wt_free(session, path); return (ret); @@ -177,10 +123,11 @@ __wt_fs_remove(WT_SESSION_IMPL *session, const char *name) /* * __wt_fs_rename -- - * POSIX rename. + * Rename the file. */ static inline int -__wt_fs_rename(WT_SESSION_IMPL *session, const char *from, const char *to) +__wt_fs_rename( + WT_SESSION_IMPL *session, const char *from, const char *to, bool durable) { WT_DECL_RET; WT_FILE_SYSTEM *file_system; @@ -211,8 +158,8 @@ __wt_fs_rename(WT_SESSION_IMPL *session, const char *from, const char *to) file_system = S2C(session)->file_system; wt_session = (WT_SESSION *)session; - ret = file_system->fs_rename( - file_system, wt_session, from_path, to_path); + ret = file_system->fs_rename(file_system, + wt_session, from_path, to_path, durable ? WT_FS_DURABLE : 0); err: __wt_free(session, from_path); __wt_free(session, to_path); @@ -221,7 +168,7 @@ err: __wt_free(session, from_path); /* * __wt_fs_size -- - * Get the size of a file in bytes, by file name. + * Return the size of a file in bytes, by file name. */ static inline int __wt_fs_size(WT_SESSION_IMPL *session, const char *name, wt_off_t *sizep) diff --git a/src/third_party/wiredtiger/src/include/os_fstream.i b/src/third_party/wiredtiger/src/include/os_fstream.i index 8c0fdadbdb0..92274431011 100644 --- a/src/third_party/wiredtiger/src/include/os_fstream.i +++ b/src/third_party/wiredtiger/src/include/os_fstream.i @@ -93,5 +93,5 @@ __wt_sync_and_rename(WT_SESSION_IMPL *session, WT_TRET(__wt_fclose(session, &fstr)); WT_RET(ret); - return (__wt_rename_and_sync_directory(session, from, to)); + return (__wt_fs_rename(session, from, to, true)); } diff --git a/src/third_party/wiredtiger/src/include/txn.h b/src/third_party/wiredtiger/src/include/txn.h index d10738cc670..2e41ae8620d 100644 --- a/src/third_party/wiredtiger/src/include/txn.h +++ b/src/third_party/wiredtiger/src/include/txn.h @@ -98,6 +98,7 @@ struct __wt_txn_global { volatile uint32_t checkpoint_id; /* Checkpoint's session ID */ volatile uint64_t checkpoint_gen; volatile uint64_t checkpoint_pinned; + volatile uint64_t checkpoint_txnid; /* Checkpoint's txn ID */ /* Named snapshot state. */ WT_RWLOCK *nsnap_rwlock; diff --git a/src/third_party/wiredtiger/src/include/wiredtiger.in b/src/third_party/wiredtiger/src/include/wiredtiger.in index f578f4e6c08..c27fd174030 100644 --- a/src/third_party/wiredtiger/src/include/wiredtiger.in +++ b/src/third_party/wiredtiger/src/include/wiredtiger.in @@ -1836,25 +1836,8 @@ struct __wt_connection { * configuration options defined below.} * @config{ archive, automatically archive * unneeded log files., a boolean flag; default \c true.} - * @config{ compressor, configure a compressor - * for log records. Permitted values are \c "none" or custom - * compression engine name created with WT_CONNECTION::add_compressor. - * If WiredTiger has builtin support for \c "snappy"\, \c "lz4" or \c - * "zlib" compression\, these names are also available. See @ref - * compression for more information., a string; default \c none.} - * @config{ enabled, enable logging subsystem., a - * boolean flag; default \c false.} - * @config{ file_max, the maximum size of log - * files., an integer between 100KB and 2GB; default \c 100MB.} - * @config{ path, the path to a directory into - * which the log files are written. If the value is not an absolute - * path name\, the files are created relative to the database home., a - * string; default \c ".".} - * @config{ prealloc, - * pre-allocate log files., a boolean flag; default \c true.} - * @config{ recover, run recovery or error if - * recovery needs to run after an unclean shutdown., a string\, chosen - * from the following options: \c "error"\, \c "on"; default \c on.} + * @config{ prealloc, pre-allocate log files., a + * boolean flag; default \c true.} * @config{ zero_fill, manually write zeroes into * log files., a boolean flag; default \c false.} * @config{ ),,} @@ -1914,11 +1897,6 @@ struct __wt_connection { * statistics in JSON format., a boolean flag; default \c false.} * @config{ on_close, log statistics on database * close., a boolean flag; default \c false.} - * @config{ path, the pathname to a file into - * which the log records are written\, may contain ISO C standard - * strftime conversion specifications. If the value is not an absolute - * path name\, the file is created relative to the database home., a - * string; default \c "WiredTigerStat.%d.%H".} * @config{ sources, if non-empty\, include * statistics for the list of data source URIs\, if they are open at the * time of the statistics logging. The list may include URIs matching a @@ -2343,9 +2321,10 @@ struct __wt_connection { * subsystem., a boolean flag; default \c false.} * @config{ file_max, the maximum size of log files., an * integer between 100KB and 2GB; default \c 100MB.} - * @config{ path, the path to a directory into which the - * log files are written. If the value is not an absolute path name\, the files - * are created relative to the database home., a string; default \c ".".} + * @config{ path, the name of a directory into which log + * files are written. The directory must already exist. If the value is not an + * absolute path\, the path is relative to the database home (see @ref + * absolute_path for more information)., a string; default \c ".".} * @config{ prealloc, pre-allocate log files., a boolean * flag; default \c true.} * @config{ recover, run recovery @@ -2415,16 +2394,15 @@ struct __wt_connection { * boolean flag; default \c false.} * @config{ on_close, * log statistics on database close., a boolean flag; default \c false.} - * @config{ path, the pathname to a file into which the - * log records are written\, may contain ISO C standard strftime conversion - * specifications. If the value is not an absolute path name\, the file is - * created relative to the database home., a string; default \c - * "WiredTigerStat.%d.%H".} - * @config{ sources, if - * non-empty\, include statistics for the list of data source URIs\, if they are - * open at the time of the statistics logging. The list may include URIs - * matching a single data source ("table:mytable")\, or a URI matching all data - * sources of a particular type ("table:")., a list of strings; default empty.} + * @config{ path, the name of a directory into which + * statistics files are written. The directory must already exist. If the + * value is not an absolute path\, the path is relative to the database home + * (see @ref absolute_path for more information)., a string; default \c ".".} + * @config{ sources, if non-empty\, include statistics + * for the list of data source URIs\, if they are open at the time of the + * statistics logging. The list may include URIs matching a single data source + * ("table:mytable")\, or a URI matching all data sources of a particular type + * ("table:")., a list of strings; default empty.} * @config{ timestamp, a timestamp prepended to each log * record\, may contain strftime conversion specifications\, when \c json is * configured\, defaults to \c "%FT%Y.000Z"., a string; default \c "%b %d @@ -3701,24 +3679,34 @@ struct __wt_extractor { #if !defined(SWIG) /*! WT_FILE_SYSTEM::open_file file types */ typedef enum { - WT_OPEN_FILE_TYPE_CHECKPOINT, /*!< open a data file checkpoint */ - WT_OPEN_FILE_TYPE_DATA, /*!< open a data file */ - WT_OPEN_FILE_TYPE_DIRECTORY, /*!< open a directory */ - WT_OPEN_FILE_TYPE_LOG, /*!< open a log file */ - WT_OPEN_FILE_TYPE_REGULAR /*!< open a regular file */ -} WT_OPEN_FILE_TYPE; + WT_FS_OPEN_FILE_TYPE_CHECKPOINT,/*!< open a data file checkpoint */ + WT_FS_OPEN_FILE_TYPE_DATA, /*!< open a data file */ + WT_FS_OPEN_FILE_TYPE_DIRECTORY, /*!< open a directory */ + WT_FS_OPEN_FILE_TYPE_LOG, /*!< open a log file */ + WT_FS_OPEN_FILE_TYPE_REGULAR /*!< open a regular file */ +} WT_FS_OPEN_FILE_TYPE; /*! WT_FILE_SYSTEM::open_file flags: create if does not exist */ -#define WT_OPEN_CREATE 0x001 +#define WT_FS_OPEN_CREATE 0x001 /*! WT_FILE_SYSTEM::open_file flags: direct I/O requested */ -#define WT_OPEN_DIRECTIO 0x002 -/*! WT_FILE_SYSTEM::open_file flags: error if exclusive use not available */ -#define WT_OPEN_EXCLUSIVE 0x004 +#define WT_FS_OPEN_DIRECTIO 0x002 +/*! WT_FILE_SYSTEM::open_file flags: file creation must be durable */ +#define WT_FS_OPEN_DURABLE 0x004 +/*! + * WT_FILE_SYSTEM::open_file flags: return EBUSY if exclusive use not available + */ +#define WT_FS_OPEN_EXCLUSIVE 0x008 #ifndef DOXYGEN -#define WT_OPEN_FIXED 0x008 /* Path not home relative (internal) */ +#define WT_FS_OPEN_FIXED 0x010 /* Path not home relative (internal) */ #endif /*! WT_FILE_SYSTEM::open_file flags: open is read-only */ -#define WT_OPEN_READONLY 0x010 +#define WT_FS_OPEN_READONLY 0x020 + +/*! + * WT_FILE_SYSTEM::remove or WT_FILE_SYSTEM::rename flags: the remove or rename + * operation must be durable + */ +#define WT_FS_DURABLE 0x001 /*! * The interface implemented by applications to provide a custom file system @@ -3768,23 +3756,6 @@ struct __wt_file_system { WT_SESSION *session, char **dirlist, uint32_t count); /*! - * Flush the named directory. - * - * This method is not required for readonly file systems or file systems - * where it is not necessary to flush a file's directory to ensure the - * durability of file system operations, and should be set to NULL when - * not required by the file system. - * - * @errors - * - * @param file_system the WT_FILE_SYSTEM - * @param session the current WiredTiger session - * @param directory the name of the directory - */ - int (*fs_directory_sync)(WT_FILE_SYSTEM *file_system, - WT_SESSION *session, const char *directory); - - /*! * Return if the named file system object exists. * * @errors @@ -3800,6 +3771,16 @@ struct __wt_file_system { /*! * Open a handle for a named file system object * + * The method should return ENOENT if the file is not being created and + * does not exist. + * + * The method should return EACCES if the file cannot be opened in the + * requested mode (for example, a file opened for writing in a readonly + * file system). + * + * The method should return EBUSY if ::WT_FS_OPEN_EXCLUSIVE is set and + * the file is in use. + * * @errors * * @param file_system the WT_FILE_SYSTEM @@ -3809,8 +3790,8 @@ struct __wt_file_system { * The file type is provided to allow optimization for different file * access patterns. * @param flags flags indicating how to open the file, one or more of - * ::WT_OPEN_CREATE, ::WT_OPEN_DIRECTIO, ::WT_OPEN_EXCLUSIVE or - * ::WT_OPEN_READONLY. + * ::WT_FS_OPEN_CREATE, ::WT_FS_OPEN_DIRECTIO, ::WT_FS_OPEN_DURABLE, + * ::WT_FS_OPEN_EXCLUSIVE or ::WT_FS_OPEN_READONLY. * @param[out] file_handlep the handle to the newly opened file. File * system implementations must allocate memory for the handle and * the WT_FILE_HANDLE::name field, and fill in the WT_FILE_HANDLE:: @@ -3819,7 +3800,7 @@ struct __wt_file_system { * their own structure as a superset of a WT_FILE_HANDLE:: structure. */ int (*fs_open_file)(WT_FILE_SYSTEM *file_system, WT_SESSION *session, - const char *name, WT_OPEN_FILE_TYPE file_type, uint32_t flags, + const char *name, WT_FS_OPEN_FILE_TYPE file_type, uint32_t flags, WT_FILE_HANDLE **file_handlep); /*! @@ -3833,9 +3814,11 @@ struct __wt_file_system { * @param file_system the WT_FILE_SYSTEM * @param session the current WiredTiger session * @param name the name of the file system object + * @param durable if the operation requires durability + * @param flags 0 or ::WT_FS_DURABLE */ - int (*fs_remove)( - WT_FILE_SYSTEM *file_system, WT_SESSION *session, const char *name); + int (*fs_remove)(WT_FILE_SYSTEM *file_system, + WT_SESSION *session, const char *name, uint32_t flags); /*! * Rename a named file system object @@ -3849,9 +3832,10 @@ struct __wt_file_system { * @param session the current WiredTiger session * @param from the original name of the object * @param to the new name for the object + * @param flags 0 or ::WT_FS_DURABLE */ - int (*fs_rename)(WT_FILE_SYSTEM *file_system, - WT_SESSION *session, const char *from, const char *to); + int (*fs_rename)(WT_FILE_SYSTEM *file_system, WT_SESSION *session, + const char *from, const char *to, uint32_t flags); /*! * Return the size of a named file system object diff --git a/src/third_party/wiredtiger/src/log/log.c b/src/third_party/wiredtiger/src/log/log.c index bf83c280d8d..75405deb512 100644 --- a/src/third_party/wiredtiger/src/log/log.c +++ b/src/third_party/wiredtiger/src/log/log.c @@ -683,7 +683,7 @@ __log_openfile(WT_SESSION_IMPL *session, WT_LOG_DESC *desc; WT_LOG_RECORD *logrec; uint32_t allocsize; - u_int flags; + u_int wtopen_flags; conn = S2C(session); log = conn->log; @@ -695,13 +695,13 @@ __log_openfile(WT_SESSION_IMPL *session, WT_ERR(__log_filename(session, id, file_prefix, buf)); WT_ERR(__wt_verbose(session, WT_VERB_LOG, "opening log %s", (const char *)buf->data)); - flags = 0; + wtopen_flags = 0; if (ok_create) - LF_SET(WT_OPEN_CREATE); + FLD_SET(wtopen_flags, WT_FS_OPEN_CREATE); if (FLD_ISSET(conn->direct_io, WT_DIRECT_IO_LOG)) - LF_SET(WT_OPEN_DIRECTIO); + FLD_SET(wtopen_flags, WT_FS_OPEN_DIRECTIO); WT_ERR(__wt_open( - session, buf->data, WT_OPEN_FILE_TYPE_LOG, flags, fhp)); + session, buf->data, WT_FS_OPEN_FILE_TYPE_LOG, wtopen_flags, fhp)); /* * If we are not creating the log file but opening it for reading, @@ -773,7 +773,7 @@ __log_alloc_prealloc(WT_SESSION_IMPL *session, uint32_t to_num) * All file setup, writing the header and pre-allocation was done * before. We only need to rename it. */ - WT_ERR(__wt_fs_rename(session, from_path->data, to_path->data)); + WT_ERR(__wt_fs_rename(session, from_path->data, to_path->data, false)); err: __wt_scr_free(session, &from_path); __wt_scr_free(session, &to_path); @@ -1058,7 +1058,7 @@ __wt_log_allocfile( /* * Rename it into place and make it available. */ - WT_ERR(__wt_fs_rename(session, from_path->data, to_path->data)); + WT_ERR(__wt_fs_rename(session, from_path->data, to_path->data, false)); err: __wt_scr_free(session, &from_path); __wt_scr_free(session, &to_path); @@ -1081,7 +1081,7 @@ __wt_log_remove(WT_SESSION_IMPL *session, WT_ERR(__log_filename(session, lognum, file_prefix, path)); WT_ERR(__wt_verbose(session, WT_VERB_LOG, "log_remove: remove log %s", (char *)path->data)); - WT_ERR(__wt_fs_remove(session, path->data)); + WT_ERR(__wt_fs_remove(session, path->data, false)); err: __wt_scr_free(session, &path); return (ret); } @@ -1117,7 +1117,7 @@ __wt_log_open(WT_SESSION_IMPL *session) WT_RET(__wt_verbose(session, WT_VERB_LOG, "log_open: open fh to directory %s", conn->log_path)); WT_RET(__wt_open(session, conn->log_path, - WT_OPEN_FILE_TYPE_DIRECTORY, 0, &log->log_dir_fh)); + WT_FS_OPEN_FILE_TYPE_DIRECTORY, 0, &log->log_dir_fh)); } if (!F_ISSET(conn, WT_CONN_READONLY)) { diff --git a/src/third_party/wiredtiger/src/lsm/lsm_cursor.c b/src/third_party/wiredtiger/src/lsm/lsm_cursor.c index 78235fb6a92..55e311fd273 100644 --- a/src/third_party/wiredtiger/src/lsm/lsm_cursor.c +++ b/src/third_party/wiredtiger/src/lsm/lsm_cursor.c @@ -219,13 +219,20 @@ __clsm_enter(WT_CURSOR_LSM *clsm, bool reset, bool update) * transaction ID in each chunk: any transaction ID * that overlaps with our snapshot is a potential * conflict. + * + * Note that the global snap_min is correct here: it + * tracks concurrent transactions excluding special + * transactions such as checkpoint (which we can't + * conflict with because checkpoint only writes the + * metadata, which is not an LSM tree). */ clsm->nupdates = 1; if (txn->isolation == WT_ISO_SNAPSHOT && F_ISSET(clsm, WT_CLSM_OPEN_SNAPSHOT)) { WT_ASSERT(session, F_ISSET(txn, WT_TXN_HAS_SNAPSHOT)); - snap_min = txn->snap_min; + snap_min = + WT_SESSION_TXN_STATE(session)->snap_min; for (switch_txnp = &clsm->switch_txn[clsm->nchunks - 2]; clsm->nupdates < clsm->nchunks; diff --git a/src/third_party/wiredtiger/src/lsm/lsm_work_unit.c b/src/third_party/wiredtiger/src/lsm/lsm_work_unit.c index c19f42327be..0f2a407c70d 100644 --- a/src/third_party/wiredtiger/src/lsm/lsm_work_unit.c +++ b/src/third_party/wiredtiger/src/lsm/lsm_work_unit.c @@ -526,7 +526,7 @@ __lsm_drop_file(WT_SESSION_IMPL *session, const char *uri) ret = __wt_schema_drop(session, uri, drop_cfg)); if (ret == 0) - ret = __wt_fs_remove(session, uri + strlen("file:")); + ret = __wt_fs_remove(session, uri + strlen("file:"), false); WT_RET(__wt_verbose(session, WT_VERB_LSM, "Dropped %s", uri)); if (ret == EBUSY || ret == ENOENT) diff --git a/src/third_party/wiredtiger/src/meta/meta_track.c b/src/third_party/wiredtiger/src/meta/meta_track.c index eb06b2bed66..3d8b7c46500 100644 --- a/src/third_party/wiredtiger/src/meta/meta_track.c +++ b/src/third_party/wiredtiger/src/meta/meta_track.c @@ -141,7 +141,8 @@ __meta_track_apply(WT_SESSION_IMPL *session, WT_META_TRACK *trk) ret = bm->checkpoint_resolve(bm, session)); break; case WT_ST_DROP_COMMIT: - if ((ret = __wt_block_manager_drop(session, trk->a)) != 0) + if ((ret = + __wt_block_manager_drop(session, trk->a, false)) != 0) __wt_err(session, ret, "metadata remove dropped file %s", trk->a); break; @@ -188,13 +189,15 @@ __meta_track_unroll(WT_SESSION_IMPL *session, WT_META_TRACK *trk) * For removes, b is NULL. */ if (trk->a != NULL && trk->b != NULL && - (ret = __wt_rename_and_sync_directory(session, - trk->b + strlen("file:"), trk->a + strlen("file:"))) != 0) + (ret = __wt_fs_rename(session, + trk->b + strlen("file:"), trk->a + strlen("file:"), + true)) != 0) __wt_err(session, ret, "metadata unroll rename %s to %s", trk->b, trk->a); - if (trk->a == NULL && (ret = - __wt_fs_remove(session, trk->b + strlen("file:"))) != 0) + if (trk->a == NULL && + (ret = __wt_fs_remove(session, + trk->b + strlen("file:"), false)) != 0) __wt_err(session, ret, "metadata unroll create %s", trk->b); diff --git a/src/third_party/wiredtiger/src/meta/meta_turtle.c b/src/third_party/wiredtiger/src/meta/meta_turtle.c index 4d2b359bbed..ace0fabab48 100644 --- a/src/third_party/wiredtiger/src/meta/meta_turtle.c +++ b/src/third_party/wiredtiger/src/meta/meta_turtle.c @@ -158,7 +158,7 @@ __wt_turtle_init(WT_SESSION_IMPL *session) * Discard any turtle setup file left-over from previous runs. This * doesn't matter for correctness, it's just cleaning up random files. */ - WT_RET(__wt_remove_if_exists(session, WT_METADATA_TURTLE_SET)); + WT_RET(__wt_remove_if_exists(session, WT_METADATA_TURTLE_SET, false)); /* * We could die after creating the turtle file and before creating the @@ -197,9 +197,10 @@ __wt_turtle_init(WT_SESSION_IMPL *session) "Both %s and %s exist; recreating metadata from " "backup", WT_METADATA_TURTLE, WT_METADATA_BACKUP)); - WT_RET(__wt_remove_if_exists(session, WT_METAFILE)); + WT_RET( + __wt_remove_if_exists(session, WT_METAFILE, false)); WT_RET(__wt_remove_if_exists( - session, WT_METADATA_TURTLE)); + session, WT_METADATA_TURTLE, false)); load = true; } } else @@ -305,7 +306,7 @@ __wt_turtle_update(WT_SESSION_IMPL *session, const char *key, const char *value) * every time. */ WT_RET(__wt_fopen(session, WT_METADATA_TURTLE_SET, - WT_OPEN_CREATE | WT_OPEN_EXCLUSIVE, WT_STREAM_WRITE, &fs)); + WT_FS_OPEN_CREATE | WT_FS_OPEN_EXCLUSIVE, WT_STREAM_WRITE, &fs)); version = wiredtiger_version(&vmajor, &vminor, &vpatch); WT_ERR(__wt_fprintf(session, fs, @@ -320,7 +321,7 @@ __wt_turtle_update(WT_SESSION_IMPL *session, const char *key, const char *value) /* Close any file handle left open, remove any temporary file. */ err: WT_TRET(__wt_fclose(session, &fs)); - WT_TRET(__wt_remove_if_exists(session, WT_METADATA_TURTLE_SET)); + WT_TRET(__wt_remove_if_exists(session, WT_METADATA_TURTLE_SET, false)); return (ret); } diff --git a/src/third_party/wiredtiger/src/os_common/filename.c b/src/third_party/wiredtiger/src/os_common/filename.c index 5f174288350..8b6c1269829 100644 --- a/src/third_party/wiredtiger/src/os_common/filename.c +++ b/src/third_party/wiredtiger/src/os_common/filename.c @@ -56,55 +56,17 @@ __wt_nfilename( * Remove a file if it exists. */ int -__wt_remove_if_exists(WT_SESSION_IMPL *session, const char *name) +__wt_remove_if_exists(WT_SESSION_IMPL *session, const char *name, bool durable) { bool exist; WT_RET(__wt_fs_exist(session, name, &exist)); if (exist) - WT_RET(__wt_fs_remove(session, name)); + WT_RET(__wt_fs_remove(session, name, durable)); return (0); } /* - * __wt_rename_and_sync_directory -- - * Rename a file and sync the enclosing directory. - */ -int -__wt_rename_and_sync_directory( - WT_SESSION_IMPL *session, const char *from, const char *to) -{ - const char *fp, *tp; - bool same_directory; - - /* Rename the source file to the target. */ - WT_RET(__wt_fs_rename(session, from, to)); - - /* - * Flush the backing directory to guarantee the rename. My reading of - * POSIX 1003.1 is there's no guarantee flushing only one of the from - * or to directories, or flushing a common parent, is sufficient, and - * even if POSIX were to make that guarantee, existing filesystems are - * known to not provide the guarantee or only provide the guarantee - * with specific mount options. Flush both of the from/to directories - * until it's a performance problem. - */ - WT_RET(__wt_fs_directory_sync(session, from)); - - /* - * In almost all cases, we're going to be renaming files in the same - * directory, we can at least fast-path that. - */ - fp = strrchr(from, '/'); - tp = strrchr(to, '/'); - same_directory = (fp == NULL && tp == NULL) || - (fp != NULL && tp != NULL && - fp - from == tp - to && memcmp(from, to, (size_t)(fp - from)) == 0); - - return (same_directory ? 0 : __wt_fs_directory_sync(session, to)); -} - -/* * __wt_copy_and_sync -- * Copy a file safely; here to support the wt utility. */ @@ -134,13 +96,13 @@ __wt_copy_and_sync(WT_SESSION *wt_session, const char *from, const char *to) WT_ERR(__wt_scr_alloc(session, 0, &tmp)); WT_ERR(__wt_buf_fmt(session, tmp, "%s.copy", to)); - WT_ERR(__wt_remove_if_exists(session, to)); - WT_ERR(__wt_remove_if_exists(session, tmp->data)); + WT_ERR(__wt_remove_if_exists(session, to, false)); + WT_ERR(__wt_remove_if_exists(session, tmp->data, false)); /* Open the from and temporary file handles. */ - WT_ERR(__wt_open(session, from, WT_OPEN_FILE_TYPE_REGULAR, 0, &ffh)); - WT_ERR(__wt_open(session, tmp->data, WT_OPEN_FILE_TYPE_REGULAR, - WT_OPEN_CREATE | WT_OPEN_EXCLUSIVE, &tfh)); + WT_ERR(__wt_open(session, from, WT_FS_OPEN_FILE_TYPE_REGULAR, 0, &ffh)); + WT_ERR(__wt_open(session, tmp->data, WT_FS_OPEN_FILE_TYPE_REGULAR, + WT_FS_OPEN_CREATE | WT_FS_OPEN_EXCLUSIVE, &tfh)); /* * Allocate a copy buffer. Don't use a scratch buffer, this thing is @@ -162,7 +124,7 @@ __wt_copy_and_sync(WT_SESSION *wt_session, const char *from, const char *to) WT_ERR(__wt_fsync(session, tfh, true)); WT_ERR(__wt_close(session, &tfh)); - ret = __wt_rename_and_sync_directory(session, tmp->data, to); + ret = __wt_fs_rename(session, tmp->data, to, true); err: WT_TRET(__wt_close(session, &ffh)); WT_TRET(__wt_close(session, &tfh)); diff --git a/src/third_party/wiredtiger/src/os_common/os_fhandle.c b/src/third_party/wiredtiger/src/os_common/os_fhandle.c index 81e4cc14ccb..184a9df0e72 100644 --- a/src/third_party/wiredtiger/src/os_common/os_fhandle.c +++ b/src/third_party/wiredtiger/src/os_common/os_fhandle.c @@ -150,19 +150,19 @@ __open_verbose( */ switch (file_type) { - case WT_OPEN_FILE_TYPE_CHECKPOINT: + case WT_FS_OPEN_FILE_TYPE_CHECKPOINT: file_type_tag = "checkpoint"; break; - case WT_OPEN_FILE_TYPE_DATA: + case WT_FS_OPEN_FILE_TYPE_DATA: file_type_tag = "data"; break; - case WT_OPEN_FILE_TYPE_DIRECTORY: + case WT_FS_OPEN_FILE_TYPE_DIRECTORY: file_type_tag = "directory"; break; - case WT_OPEN_FILE_TYPE_LOG: + case WT_FS_OPEN_FILE_TYPE_LOG: file_type_tag = "log"; break; - case WT_OPEN_FILE_TYPE_REGULAR: + case WT_FS_OPEN_FILE_TYPE_REGULAR: file_type_tag = "regular"; break; default: @@ -172,18 +172,18 @@ __open_verbose( WT_RET(__wt_scr_alloc(session, 0, &tmp)); sep = " ("; -#define WT_OPEN_VERBOSE_FLAG(f, name) \ +#define WT_FS_OPEN_VERBOSE_FLAG(f, name) \ if (LF_ISSET(f)) { \ WT_ERR(__wt_buf_catfmt( \ session, tmp, "%s%s", sep, name)); \ sep = ", "; \ } - WT_OPEN_VERBOSE_FLAG(WT_OPEN_CREATE, "create"); - WT_OPEN_VERBOSE_FLAG(WT_OPEN_DIRECTIO, "direct-IO"); - WT_OPEN_VERBOSE_FLAG(WT_OPEN_EXCLUSIVE, "exclusive"); - WT_OPEN_VERBOSE_FLAG(WT_OPEN_FIXED, "fixed"); - WT_OPEN_VERBOSE_FLAG(WT_OPEN_READONLY, "readonly"); + WT_FS_OPEN_VERBOSE_FLAG(WT_FS_OPEN_CREATE, "create"); + WT_FS_OPEN_VERBOSE_FLAG(WT_FS_OPEN_DIRECTIO, "direct-IO"); + WT_FS_OPEN_VERBOSE_FLAG(WT_FS_OPEN_EXCLUSIVE, "exclusive"); + WT_FS_OPEN_VERBOSE_FLAG(WT_FS_OPEN_FIXED, "fixed"); + WT_FS_OPEN_VERBOSE_FLAG(WT_FS_OPEN_READONLY, "readonly"); if (tmp->size != 0) WT_ERR(__wt_buf_catfmt(session, tmp, ")")); @@ -209,7 +209,7 @@ err: __wt_scr_free(session, &tmp); */ int __wt_open(WT_SESSION_IMPL *session, - const char *name, WT_OPEN_FILE_TYPE file_type, u_int flags, WT_FH **fhp) + const char *name, WT_FS_OPEN_FILE_TYPE file_type, u_int flags, WT_FH **fhp) { WT_CONNECTION_IMPL *conn; WT_DECL_RET; @@ -247,12 +247,12 @@ __wt_open(WT_SESSION_IMPL *session, if (F_ISSET(conn, WT_CONN_READONLY)) { lock_file = strcmp(name, WT_SINGLETHREAD) == 0; if (!lock_file) - LF_SET(WT_OPEN_READONLY); - WT_ASSERT(session, lock_file || !LF_ISSET(WT_OPEN_CREATE)); + LF_SET(WT_FS_OPEN_READONLY); + WT_ASSERT(session, lock_file || !LF_ISSET(WT_FS_OPEN_CREATE)); } /* Create the path to the file. */ - if (!LF_ISSET(WT_OPEN_FIXED)) + if (!LF_ISSET(WT_FS_OPEN_FIXED)) WT_ERR(__wt_filename(session, name, &path)); /* Call the underlying open function. */ @@ -261,7 +261,7 @@ __wt_open(WT_SESSION_IMPL *session, open_called = true; WT_ERR(__fhandle_method_finalize( - session, fh->handle, LF_ISSET(WT_OPEN_READONLY))); + session, fh->handle, LF_ISSET(WT_FS_OPEN_READONLY))); /* * Repeat the check for a match: if there's no match, link our newly diff --git a/src/third_party/wiredtiger/src/os_common/os_fs_inmemory.c b/src/third_party/wiredtiger/src/os_common/os_fs_inmemory.c index 09c2e08db83..178adc1dac8 100644 --- a/src/third_party/wiredtiger/src/os_common/os_fs_inmemory.c +++ b/src/third_party/wiredtiger/src/os_common/os_fs_inmemory.c @@ -188,14 +188,16 @@ __im_fs_exist(WT_FILE_SYSTEM *file_system, * POSIX remove. */ static int -__im_fs_remove( - WT_FILE_SYSTEM *file_system, WT_SESSION *wt_session, const char *name) +__im_fs_remove(WT_FILE_SYSTEM *file_system, + WT_SESSION *wt_session, const char *name, uint32_t flags) { WT_DECL_RET; WT_FILE_HANDLE_INMEM *im_fh; WT_FILE_SYSTEM_INMEM *im_fs; WT_SESSION_IMPL *session; + WT_UNUSED(flags); + im_fs = (WT_FILE_SYSTEM_INMEM *)file_system; session = (WT_SESSION_IMPL *)wt_session; @@ -215,7 +217,7 @@ __im_fs_remove( */ static int __im_fs_rename(WT_FILE_SYSTEM *file_system, - WT_SESSION *wt_session, const char *from, const char *to) + WT_SESSION *wt_session, const char *from, const char *to, uint32_t flags) { WT_DECL_RET; WT_FILE_HANDLE_INMEM *im_fh; @@ -224,6 +226,8 @@ __im_fs_rename(WT_FILE_SYSTEM *file_system, uint64_t bucket; char *copy; + WT_UNUSED(flags); + im_fs = (WT_FILE_SYSTEM_INMEM *)file_system; session = (WT_SESSION_IMPL *)wt_session; @@ -463,7 +467,7 @@ err: __wt_spin_unlock(session, &im_fs->lock); */ static int __im_file_open(WT_FILE_SYSTEM *file_system, WT_SESSION *wt_session, - const char *name, WT_OPEN_FILE_TYPE file_type, uint32_t flags, + const char *name, WT_FS_OPEN_FILE_TYPE file_type, uint32_t flags, WT_FILE_HANDLE **file_handlep) { WT_DECL_RET; diff --git a/src/third_party/wiredtiger/src/os_common/os_fs_stdio.c b/src/third_party/wiredtiger/src/os_common/os_fs_stdio.c deleted file mode 100644 index 9baba9b6945..00000000000 --- a/src/third_party/wiredtiger/src/os_common/os_fs_stdio.c +++ /dev/null @@ -1,239 +0,0 @@ -/*- - * Copyright (c) 2014-2016 MongoDB, Inc. - * Copyright (c) 2008-2014 WiredTiger, Inc. - * All rights reserved. - * - * See the file LICENSE for redistribution information. - */ - -#include "wt_internal.h" - -/* - * __stdio_handle_advise -- - * POSIX fadvise. - */ -static int -__stdio_handle_advise(WT_SESSION_IMPL *session, - WT_FH *fh, wt_off_t offset, wt_off_t len, int advice) -{ - WT_UNUSED(offset); - WT_UNUSED(len); - WT_UNUSED(advice); - WT_RET_MSG(session, ENOTSUP, "%s: handle-advise", fh->name); -} - -/* - * __stdio_handle_allocate -- - * POSIX fallocate. - */ -static int -__stdio_handle_allocate( - WT_SESSION_IMPL *session, WT_FH *fh, wt_off_t offset, wt_off_t len) -{ - WT_UNUSED(offset); - WT_UNUSED(len); - WT_RET_MSG(session, ENOTSUP, "%s: handle-allocate", fh->name); -} - -/* - * __stdio_handle_close -- - * ANSI C close/fclose. - */ -static int -__stdio_handle_close(WT_SESSION_IMPL *session, WT_FH *fh) -{ - WT_RET_MSG(session, ENOTSUP, "%s: handle-close", fh->name); -} - -/* - * __stdio_handle_getc -- - * ANSI C fgetc. - */ -static int -__stdio_handle_getc(WT_SESSION_IMPL *session, WT_FH *fh, int *chp) -{ - WT_UNUSED(chp); - WT_RET_MSG(session, ENOTSUP, "%s: handle-getc", fh->name); -} - -/* - * __stdio_handle_lock -- - * Lock/unlock a file. - */ -static int -__stdio_handle_lock(WT_SESSION_IMPL *session, WT_FH *fh, bool lock) -{ - WT_UNUSED(lock); - WT_RET_MSG(session, ENOTSUP, "%s: handle-lock", fh->name); -} - -/* - * __stdio_handle_map -- - * Map a file. - */ -static int -__stdio_handle_map(WT_SESSION_IMPL *session, - WT_FH *fh, void *p, size_t *lenp, void **mappingcookie) -{ - WT_UNUSED(p); - WT_UNUSED(lenp); - WT_UNUSED(mappingcookie); - WT_RET_MSG(session, ENOTSUP, "%s: handle-map", fh->name); -} - -/* - * __stdio_handle_map_discard -- - * Discard a section of a mapped region. - */ -static int -__stdio_handle_map_discard( - WT_SESSION_IMPL *session, WT_FH *fh, void *p, size_t len) -{ - WT_UNUSED(p); - WT_UNUSED(len); - WT_RET_MSG(session, ENOTSUP, "%s: handle-map-discard", fh->name); -} - -/* - * __stdio_handle_map_preload -- - * Preload a section of a mapped region. - */ -static int -__stdio_handle_map_preload( - WT_SESSION_IMPL *session, WT_FH *fh, const void *p, size_t len) -{ - WT_UNUSED(p); - WT_UNUSED(len); - WT_RET_MSG(session, ENOTSUP, "%s: handle-map-preload", fh->name); -} - -/* - * __stdio_handle_map_unmap -- - * Unmap a file. - */ -static int -__stdio_handle_map_unmap(WT_SESSION_IMPL *session, - WT_FH *fh, void *p, size_t len, void **mappingcookie) -{ - WT_UNUSED(p); - WT_UNUSED(len); - WT_UNUSED(mappingcookie); - WT_RET_MSG(session, ENOTSUP, "%s: handle-map-unmap", fh->name); -} - -/* - * __stdio_handle_printf -- - * ANSI C vfprintf. - */ -static int -__stdio_handle_printf( - WT_SESSION_IMPL *session, WT_FH *fh, const char *fmt, va_list ap) -{ - if (vfprintf(fh->fp, fmt, ap) >= 0) - return (0); - WT_RET_MSG(session, EIO, "%s: handle-printf: vfprintf", fh->name); -} - -/* - * __stdio_handle_read -- - * POSIX pread. - */ -static int -__stdio_handle_read( - WT_SESSION_IMPL *session, WT_FH *fh, wt_off_t offset, size_t len, void *buf) -{ - WT_UNUSED(offset); - WT_UNUSED(len); - WT_UNUSED(buf); - WT_RET_MSG(session, ENOTSUP, "%s: handle-read", fh->name); -} - -/* - * __stdio_handle_size -- - * Get the size of a file in bytes, by file handle. - */ -static int -__stdio_handle_size(WT_SESSION_IMPL *session, WT_FH *fh, wt_off_t *sizep) -{ - WT_UNUSED(sizep); - WT_RET_MSG(session, ENOTSUP, "%s: handle-size", fh->name); -} - -/* - * __stdio_handle_sync -- - * POSIX fflush/fsync. - */ -static int -__stdio_handle_sync(WT_SESSION_IMPL *session, WT_FH *fh, bool block) -{ - WT_UNUSED(block); - - if (fflush(fh->fp) == 0) - return (0); - WT_RET_MSG(session, __wt_errno(), "%s: handle-sync: fflush", fh->name); -} - -/* - * __stdio_handle_truncate -- - * POSIX ftruncate. - */ -static int -__stdio_handle_truncate(WT_SESSION_IMPL *session, WT_FH *fh, wt_off_t len) -{ - WT_UNUSED(len); - WT_RET_MSG(session, ENOTSUP, "%s: handle-truncate", fh->name); -} - -/* - * __stdio_handle_write -- - * POSIX pwrite. - */ -static int -__stdio_handle_write(WT_SESSION_IMPL *session, - WT_FH *fh, wt_off_t offset, size_t len, const void *buf) -{ - WT_UNUSED(offset); - WT_UNUSED(len); - WT_UNUSED(buf); - WT_RET_MSG(session, ENOTSUP, "%s: handle-write", fh->name); -} - -/* - * __stdio_func_init -- - * Initialize stdio functions. - */ -static void -__stdio_func_init(WT_FH *fh, const char *name, FILE *fp) -{ - fh->name = name; - fh->fp = fp; - - fh->fh_advise = __stdio_handle_advise; - fh->fh_allocate = __stdio_handle_allocate; - fh->fh_close = __stdio_handle_close; - fh->fh_getc = __stdio_handle_getc; - fh->fh_lock = __stdio_handle_lock; - fh->fh_map = __stdio_handle_map; - fh->fh_map_discard = __stdio_handle_map_discard; - fh->fh_map_preload = __stdio_handle_map_preload; - fh->fh_map_unmap = __stdio_handle_map_unmap; - fh->fh_printf = __stdio_handle_printf; - fh->fh_read = __stdio_handle_read; - fh->fh_size = __stdio_handle_size; - fh->fh_sync = __stdio_handle_sync; - fh->fh_truncate = __stdio_handle_truncate; - fh->fh_write = __stdio_handle_write; -} - -/* - * __wt_os_stdio -- - * Initialize the stdio configuration. - */ -int -__wt_os_stdio(WT_SESSION_IMPL *session) -{ - __stdio_func_init(WT_STDERR(session), "stderr", stderr); - __stdio_func_init(WT_STDOUT(session), "stdout", stdout); - - return (0); -} diff --git a/src/third_party/wiredtiger/src/os_common/os_fstream.c b/src/third_party/wiredtiger/src/os_common/os_fstream.c index 0b199529e19..5a368ea75e6 100644 --- a/src/third_party/wiredtiger/src/os_common/os_fstream.c +++ b/src/third_party/wiredtiger/src/os_common/os_fstream.c @@ -187,7 +187,7 @@ __wt_fopen(WT_SESSION_IMPL *session, fstr = NULL; WT_RET(__wt_open( - session, name, WT_OPEN_FILE_TYPE_REGULAR, open_flags, &fh)); + session, name, WT_FS_OPEN_FILE_TYPE_REGULAR, open_flags, &fh)); WT_ERR(__wt_calloc_one(session, &fstr)); fstr->fh = fh; diff --git a/src/third_party/wiredtiger/src/os_common/os_getline.c b/src/third_party/wiredtiger/src/os_common/os_getline.c deleted file mode 100644 index 01e11581edf..00000000000 --- a/src/third_party/wiredtiger/src/os_common/os_getline.c +++ /dev/null @@ -1,51 +0,0 @@ -/*- - * Copyright (c) 2014-2016 MongoDB, Inc. - * Copyright (c) 2008-2014 WiredTiger, Inc. - * All rights reserved. - * - * See the file LICENSE for redistribution information. - */ - -#include "wt_internal.h" - -/* - * __wt_getline -- - * Get a line from a stream. - * - * Implementation of the POSIX getline or BSD fgetln functions (finding the - * function in a portable way is hard, it's simple enough to write it instead). - * - * Note: Unlike the standard getline calls, this function doesn't include the - * trailing newline character in the returned buffer and discards empty lines - * (so the caller's EOF marker is a returned line length of 0). - */ -int -__wt_getline(WT_SESSION_IMPL *session, WT_ITEM *buf, WT_FH *fh) -{ - int c; - - /* - * We always NUL-terminate the returned string (even if it's empty), - * make sure there's buffer space for a trailing NUL in all cases. - */ - WT_RET(__wt_buf_init(session, buf, 100)); - - for (;;) { - WT_RET(fh->fh_getc(session, fh, &c)); - if (c == EOF) - break; - - /* Leave space for a trailing NUL. */ - WT_RET(__wt_buf_extend(session, buf, buf->size + 2)); - if (c == '\n') { - if (buf->size == 0) - continue; - break; - } - ((char *)buf->mem)[buf->size++] = (char)c; - } - - ((char *)buf->mem)[buf->size] = '\0'; - - return (0); -} diff --git a/src/third_party/wiredtiger/src/os_common/os_init.c b/src/third_party/wiredtiger/src/os_common/os_init.c deleted file mode 100644 index 512216c52a5..00000000000 --- a/src/third_party/wiredtiger/src/os_common/os_init.c +++ /dev/null @@ -1,41 +0,0 @@ -/*- - * Copyright (c) 2014-2016 MongoDB, Inc. - * Copyright (c) 2008-2014 WiredTiger, Inc. - * All rights reserved. - * - * See the file LICENSE for redistribution information. - */ - -#include "wt_internal.h" - -/* - * __wt_os_init -- - * Initialize the OS layer. - */ -int -__wt_os_init(WT_SESSION_IMPL *session) -{ - return (F_ISSET(S2C(session), WT_CONN_IN_MEMORY) ? - __wt_os_inmemory(session) : -#if defined(_MSC_VER) - __wt_os_win(session)); -#else - __wt_os_posix(session)); -#endif -} - -/* - * __wt_os_cleanup -- - * Clean up the OS layer. - */ -int -__wt_os_cleanup(WT_SESSION_IMPL *session) -{ - return (F_ISSET(S2C(session), WT_CONN_IN_MEMORY) ? - __wt_os_inmemory_cleanup(session) : -#if defined(_MSC_VER) - __wt_os_win_cleanup(session)); -#else - __wt_os_posix_cleanup(session)); -#endif -} diff --git a/src/third_party/wiredtiger/src/os_posix/os_errno.c b/src/third_party/wiredtiger/src/os_posix/os_errno.c deleted file mode 100644 index a0f1202c6ef..00000000000 --- a/src/third_party/wiredtiger/src/os_posix/os_errno.c +++ /dev/null @@ -1,71 +0,0 @@ -/*- - * Copyright (c) 2014-2016 MongoDB, Inc. - * Copyright (c) 2008-2014 WiredTiger, Inc. - * All rights reserved. - * - * See the file LICENSE for redistribution information. - */ - -#include "wt_internal.h" - -/* - * __wt_errno -- - * Return errno, or WT_ERROR if errno not set. - */ -int -__wt_errno(void) -{ - /* - * Called when we know an error occurred, and we want the system - * error code, but there's some chance it's not set. - */ - return (errno == 0 ? WT_ERROR : errno); -} - -/* - * __wt_map_error_rdonly -- - * Map an error into a WiredTiger error code specific for - * read-only operation which intercepts based on certain types - * of failures. - */ -int -__wt_map_error_rdonly(int error) -{ - if (error == ENOENT) - return (WT_NOTFOUND); - else if (error == EACCES) - return (WT_PERM_DENIED); - return (error); -} - -/* - * __wt_strerror -- - * POSIX implementation of WT_SESSION.strerror and wiredtiger_strerror. - */ -const char * -__wt_strerror(WT_SESSION_IMPL *session, int error, char *errbuf, size_t errlen) -{ - const char *p; - - /* - * Check for a WiredTiger or POSIX constant string, no buffer needed. - */ - if ((p = __wt_wiredtiger_error(error)) != NULL) - return (p); - - /* - * When called from wiredtiger_strerror, write a passed-in buffer. - * When called from WT_SESSION.strerror, write the session's buffer. - * - * Fallback to a generic message. - */ - if (session == NULL && - snprintf(errbuf, errlen, "error return: %d", error) > 0) - return (errbuf); - if (session != NULL && __wt_buf_fmt( - session, &session->err, "error return: %d", error) == 0) - return (session->err.data); - - /* Defeated. */ - return ("Unable to return error string"); -} diff --git a/src/third_party/wiredtiger/src/os_posix/os_fs.c b/src/third_party/wiredtiger/src/os_posix/os_fs.c index 86fa2e8f117..11f38ec063b 100644 --- a/src/third_party/wiredtiger/src/os_posix/os_fs.c +++ b/src/third_party/wiredtiger/src/os_posix/os_fs.c @@ -30,7 +30,7 @@ /* * __posix_sync -- - * Underlying support function to flush a file handle. + * Underlying support function to flush a file descriptor. */ static int __posix_sync( @@ -77,33 +77,42 @@ __posix_sync( #ifdef __linux__ /* * __posix_directory_sync -- - * Flush a directory to ensure file creation is durable. + * Flush a directory to ensure file creation, remove or rename is durable. */ static int -__posix_directory_sync( - WT_FILE_SYSTEM *file_system, WT_SESSION *wt_session, const char *path) +__posix_directory_sync(WT_SESSION_IMPL *session, const char *path) { + WT_DECL_ITEM(tmp); WT_DECL_RET; - WT_SESSION_IMPL *session; int fd, tret; + char *dir; - WT_UNUSED(file_system); + WT_RET(__wt_scr_alloc(session, 0, &tmp)); + WT_ERR(__wt_buf_setstr(session, tmp, path)); - session = (WT_SESSION_IMPL *)wt_session; + /* + * This layer should never see a path that doesn't include a trailing + * path separator, this code asserts that fact. + */ + dir = tmp->mem; + strrchr(dir, '/')[1] = '\0'; + fd = -1; /* -Wconditional-uninitialized */ WT_SYSCALL_RETRY(( - (fd = open(path, O_RDONLY, 0444)) == -1 ? -1 : 0), ret); + (fd = open(dir, O_RDONLY, 0444)) == -1 ? -1 : 0), ret); if (ret != 0) - WT_RET_MSG(session, ret, "%s: directory-sync: open", path); + WT_ERR_MSG(session, ret, "%s: directory-sync: open", dir); - ret = __posix_sync(session, fd, path, "directory-sync"); + ret = __posix_sync(session, fd, dir, "directory-sync"); WT_SYSCALL(close(fd), tret); if (tret != 0) { - __wt_err(session, tret, "%s: directory-sync: close", path); + __wt_err(session, tret, "%s: directory-sync: close", dir); if (ret == 0) ret = tret; } + +err: __wt_scr_free(session, &tmp); return (ret); } #endif @@ -141,8 +150,8 @@ __posix_fs_exist(WT_FILE_SYSTEM *file_system, * Remove a file. */ static int -__posix_fs_remove( - WT_FILE_SYSTEM *file_system, WT_SESSION *wt_session, const char *name) +__posix_fs_remove(WT_FILE_SYSTEM *file_system, + WT_SESSION *wt_session, const char *name, uint32_t flags) { WT_DECL_RET; WT_SESSION_IMPL *session; @@ -159,9 +168,17 @@ __posix_fs_remove( * using unlink may be marginally safer. */ WT_SYSCALL(unlink(name), ret); - if (ret == 0) + if (ret != 0) + WT_RET_MSG(session, ret, "%s: file-remove: unlink", name); + + if (!LF_ISSET(WT_FS_DURABLE)) return (0); - WT_RET_MSG(session, ret, "%s: file-remove: unlink", name); + +#ifdef __linux__ + /* Flush the backing directory to guarantee the remove. */ + WT_RET (__posix_directory_sync(session, name)); +#endif + return (0); } /* @@ -170,7 +187,7 @@ __posix_fs_remove( */ static int __posix_fs_rename(WT_FILE_SYSTEM *file_system, - WT_SESSION *wt_session, const char *from, const char *to) + WT_SESSION *wt_session, const char *from, const char *to, uint32_t flags) { WT_DECL_RET; WT_SESSION_IMPL *session; @@ -187,9 +204,43 @@ __posix_fs_rename(WT_FILE_SYSTEM *file_system, * return (if errno is 0), but we've done the best we can. */ WT_SYSCALL(rename(from, to) != 0 ? -1 : 0, ret); - if (ret == 0) + if (ret != 0) + WT_RET_MSG( + session, ret, "%s to %s: file-rename: rename", from, to); + + if (!LF_ISSET(WT_FS_DURABLE)) return (0); - WT_RET_MSG(session, ret, "%s to %s: file-rename: rename", from, to); +#ifdef __linux__ + /* + * Flush the backing directory to guarantee the rename. My reading of + * POSIX 1003.1 is there's no guarantee flushing only one of the from + * or to directories, or flushing a common parent, is sufficient, and + * even if POSIX were to make that guarantee, existing filesystems are + * known to not provide the guarantee or only provide the guarantee + * with specific mount options. Flush both of the from/to directories + * until it's a performance problem. + */ + WT_RET(__posix_directory_sync(session, from)); + + /* + * In almost all cases, we're going to be renaming files in the same + * directory, we can at least fast-path that. + */ + { + bool same_directory; + const char *fp, *tp; + + fp = strrchr(from, '/'); + tp = strrchr(to, '/'); + same_directory = (fp == NULL && tp == NULL) || + (fp != NULL && tp != NULL && + fp - from == tp - to && memcmp(from, to, (size_t)(fp - from)) == 0); + + if (!same_directory) + WT_RET(__posix_directory_sync(session, to)); + } +#endif + return (0); } /* @@ -513,7 +564,7 @@ __posix_open_file_cloexec(WT_SESSION_IMPL *session, int fd, const char *name) */ static int __posix_open_file(WT_FILE_SYSTEM *file_system, WT_SESSION *wt_session, - const char *name, WT_OPEN_FILE_TYPE file_type, uint32_t flags, + const char *name, WT_FS_OPEN_FILE_TYPE file_type, uint32_t flags, WT_FILE_HANDLE **file_handlep) { WT_CONNECTION_IMPL *conn; @@ -536,7 +587,7 @@ __posix_open_file(WT_FILE_SYSTEM *file_system, WT_SESSION *wt_session, /* Set up error handling. */ pfh->fd = -1; - if (file_type == WT_OPEN_FILE_TYPE_DIRECTORY) { + if (file_type == WT_FS_OPEN_FILE_TYPE_DIRECTORY) { f = O_RDONLY; #ifdef O_CLOEXEC /* @@ -554,10 +605,10 @@ __posix_open_file(WT_FILE_SYSTEM *file_system, WT_SESSION *wt_session, goto directory_open; } - f = LF_ISSET(WT_OPEN_READONLY) ? O_RDONLY : O_RDWR; - if (LF_ISSET(WT_OPEN_CREATE)) { + f = LF_ISSET(WT_FS_OPEN_READONLY) ? O_RDONLY : O_RDWR; + if (LF_ISSET(WT_FS_OPEN_CREATE)) { f |= O_CREAT; - if (LF_ISSET(WT_OPEN_EXCLUSIVE)) + if (LF_ISSET(WT_FS_OPEN_EXCLUSIVE)) f |= O_EXCL; mode = 0666; } else @@ -577,7 +628,7 @@ __posix_open_file(WT_FILE_SYSTEM *file_system, WT_SESSION *wt_session, #endif #ifdef O_DIRECT /* Direct I/O. */ - if (LF_ISSET(WT_OPEN_DIRECTIO)) { + if (LF_ISSET(WT_FS_OPEN_DIRECTIO)) { f |= O_DIRECT; pfh->direct_io = true; } else @@ -585,11 +636,11 @@ __posix_open_file(WT_FILE_SYSTEM *file_system, WT_SESSION *wt_session, #endif #ifdef O_NOATIME /* Avoid updating metadata for read-only workloads. */ - if (file_type == WT_OPEN_FILE_TYPE_DATA) + if (file_type == WT_FS_OPEN_FILE_TYPE_DATA) f |= O_NOATIME; #endif - if (file_type == WT_OPEN_FILE_TYPE_LOG && + if (file_type == WT_FS_OPEN_FILE_TYPE_LOG && FLD_ISSET(conn->txn_logsync, WT_LOG_DSYNC)) { #ifdef O_DSYNC f |= O_DSYNC; @@ -601,6 +652,7 @@ __posix_open_file(WT_FILE_SYSTEM *file_system, WT_SESSION *wt_session, #endif } + /* Create/Open the file. */ WT_SYSCALL_RETRY(((pfh->fd = open(name, f, mode)) == -1 ? -1 : 0), ret); if (ret != 0) WT_ERR_MSG(session, ret, @@ -608,6 +660,16 @@ __posix_open_file(WT_FILE_SYSTEM *file_system, WT_SESSION *wt_session, "%s: handle-open: open: failed with direct I/O configured, " "some filesystem types do not support direct I/O" : "%s: handle-open: open", name); + +#ifdef __linux__ + /* + * Durability: some filesystems require a directory sync to be confident + * the file will appear. + */ + if (LF_ISSET(WT_FS_OPEN_DURABLE)) + WT_ERR(__posix_directory_sync(session, name)); +#endif + WT_ERR(__posix_open_file_cloexec(session, pfh->fd, name)); #if defined(HAVE_POSIX_FADVISE) @@ -616,7 +678,7 @@ __posix_open_file(WT_FILE_SYSTEM *file_system, WT_SESSION *wt_session, * Ignore fadvise when doing direct I/O, the kernel cache isn't * interesting. */ - if (!pfh->direct_io && file_type == WT_OPEN_FILE_TYPE_DATA) { + if (!pfh->direct_io && file_type == WT_FS_OPEN_FILE_TYPE_DATA) { WT_SYSCALL( posix_fadvise(pfh->fd, 0, 0, POSIX_FADV_RANDOM), ret); if (ret != 0) @@ -705,9 +767,6 @@ __wt_os_posix(WT_SESSION_IMPL *session) /* Initialize the POSIX jump table. */ file_system->fs_directory_list = __wt_posix_directory_list; file_system->fs_directory_list_free = __wt_posix_directory_list_free; -#ifdef __linux__ - file_system->fs_directory_sync = __posix_directory_sync; -#endif file_system->fs_exist = __posix_fs_exist; file_system->fs_open_file = __posix_open_file; file_system->fs_remove = __posix_fs_remove; diff --git a/src/third_party/wiredtiger/src/os_win/os_errno.c b/src/third_party/wiredtiger/src/os_win/os_errno.c deleted file mode 100644 index f3fffd5ef42..00000000000 --- a/src/third_party/wiredtiger/src/os_win/os_errno.c +++ /dev/null @@ -1,151 +0,0 @@ -/*- - * Copyright (c) 2014-2016 MongoDB, Inc. - * Copyright (c) 2008-2014 WiredTiger, Inc. - * All rights reserved. - * - * See the file LICENSE for redistribution information. - */ - -#include "wt_internal.h" - -static const int windows_error_offset = -29000; - -/* - * __wt_map_error_to_windows_error -- - * Return a negative integer, an encoded Windows error - * Standard C errors are positive integers from 0 - ~200 - * Windows errors are from 0 - 15999 according to the documentation - */ -static DWORD -__wt_map_error_to_windows_error(int error) -{ - /* - * Ensure we do not exceed the error range - * Also validate we do not get any COM errors - * (which are negative integers) - */ - WT_ASSERT(NULL, error < 0); - - return (error + -(windows_error_offset)); -} - -/* - * __wt_map_windows_error_to_error -- - * Return a positive integer, a decoded Windows error - */ -static int -__wt_map_windows_error_to_error(DWORD winerr) -{ - return (winerr + windows_error_offset); -} - -/* - * __wt_map_error_rdonly -- - * Map an error into a WiredTiger error code specific for - * read-only operation which intercepts based on certain types - * of failures. - */ -int -__wt_map_error_rdonly(int error) -{ - if (error == ERROR_FILE_NOT_FOUND) - return (WT_NOTFOUND); - else if (error == ERROR_ACCESS_DENIED) - return (WT_PERM_DENIED); - return (error); -} - -/* - * __wt_errno -- - * Return errno, or WT_ERROR if errno not set. - */ -int -__wt_errno(void) -{ - /* - * Check for 0: - * It's easy to introduce a problem by calling the wrong error function, - * for example, this function when the MSVC function set the C runtime - * error value. Handle gracefully and always return an error. - */ - return (errno == 0 ? WT_ERROR : errno); -} - -/* - * __wt_getlasterror -- - * Return GetLastError, or WT_ERROR if error not set. - */ -int -__wt_getlasterror(void) -{ - /* - * Called when we know an error occurred, and we want the system - * error code. - */ - DWORD err = GetLastError(); - - /* - * Check for ERROR_SUCCESS: - * It's easy to introduce a problem by calling the wrong error function, - * for example, this function when the MSVC function set the C runtime - * error value. Handle gracefully and always return an error. - */ - return (err == ERROR_SUCCESS ? - WT_ERROR : __wt_map_windows_error_to_error(err)); -} - -/* - * __wt_strerror -- - * Windows implementation of WT_SESSION.strerror and wiredtiger_strerror. - */ -const char * -__wt_strerror(WT_SESSION_IMPL *session, int error, char *errbuf, size_t errlen) -{ - DWORD lasterror; - const char *p; - char buf[512]; - - /* - * Check for a WiredTiger or POSIX constant string, no buffer needed. - */ - if ((p = __wt_wiredtiger_error(error)) != NULL) - return (p); - - /* - * When called from wiredtiger_strerror, write a passed-in buffer. - * When called from WT_SESSION.strerror, write the session's buffer. - * - * Check for Windows errors. - */ - if (error < 0) { - error = __wt_map_error_to_windows_error(error); - - lasterror = FormatMessageA( - FORMAT_MESSAGE_FROM_SYSTEM | - FORMAT_MESSAGE_IGNORE_INSERTS, - NULL, - error, - 0, /* let system choose the correct LANGID */ - buf, - sizeof(buf), - NULL); - - if (lasterror != 0 && session == NULL && - snprintf(errbuf, errlen, "%s", buf) > 0) - return (errbuf); - if (lasterror != 0 && session != NULL && - __wt_buf_fmt(session, &session->err, "%s", buf) == 0) - return (session->err.data); - } - - /* Fallback to a generic message. */ - if (session == NULL && - snprintf(errbuf, errlen, "error return: %d", error) > 0) - return (errbuf); - if (session != NULL && __wt_buf_fmt( - session, &session->err, "error return: %d", error) == 0) - return (session->err.data); - - /* Defeated. */ - return ("Unable to return error string"); -} diff --git a/src/third_party/wiredtiger/src/os_win/os_fs.c b/src/third_party/wiredtiger/src/os_win/os_fs.c index 5daba124e90..fc03e0a2595 100644 --- a/src/third_party/wiredtiger/src/os_win/os_fs.c +++ b/src/third_party/wiredtiger/src/os_win/os_fs.c @@ -36,13 +36,14 @@ __win_fs_exist(WT_FILE_SYSTEM *file_system, * Remove a file. */ static int -__win_fs_remove( - WT_FILE_SYSTEM *file_system, WT_SESSION *wt_session, const char *name) +__win_fs_remove(WT_FILE_SYSTEM *file_system, + WT_SESSION *wt_session, const char *name, uint32_t flags) { DWORD windows_error; WT_SESSION_IMPL *session; WT_UNUSED(file_system); + WT_UNUSED(flags); session = (WT_SESSION_IMPL *)wt_session; @@ -62,12 +63,13 @@ __win_fs_remove( */ static int __win_fs_rename(WT_FILE_SYSTEM *file_system, - WT_SESSION *wt_session, const char *from, const char *to) + WT_SESSION *wt_session, const char *from, const char *to, uint32_t flags) { DWORD windows_error; WT_SESSION_IMPL *session; WT_UNUSED(file_system); + WT_UNUSED(flags); session = (WT_SESSION_IMPL *)wt_session; @@ -426,7 +428,7 @@ __win_file_write(WT_FILE_HANDLE *file_handle, */ static int __win_open_file(WT_FILE_SYSTEM *file_system, WT_SESSION *wt_session, - const char *name, WT_OPEN_FILE_TYPE file_type, uint32_t flags, + const char *name, WT_FS_OPEN_FILE_TYPE file_type, uint32_t flags, WT_FILE_HANDLE **file_handlep) { DWORD dwCreationDisposition, windows_error; @@ -458,11 +460,11 @@ __win_open_file(WT_FILE_SYSTEM *file_system, WT_SESSION *wt_session, * require that functionality: create an empty WT_FH structure with * invalid handles. */ - if (file_type == WT_OPEN_FILE_TYPE_DIRECTORY) + if (file_type == WT_FS_OPEN_FILE_TYPE_DIRECTORY) goto directory_open; desired_access = GENERIC_READ; - if (!LF_ISSET(WT_OPEN_READONLY)) + if (!LF_ISSET(WT_FS_OPEN_READONLY)) desired_access |= GENERIC_WRITE; /* @@ -476,15 +478,15 @@ __win_open_file(WT_FILE_SYSTEM *file_system, WT_SESSION *wt_session, f = FILE_ATTRIBUTE_NORMAL; dwCreationDisposition = 0; - if (LF_ISSET(WT_OPEN_CREATE)) { + if (LF_ISSET(WT_FS_OPEN_CREATE)) { dwCreationDisposition = CREATE_NEW; - if (LF_ISSET(WT_OPEN_EXCLUSIVE)) + if (LF_ISSET(WT_FS_OPEN_EXCLUSIVE)) dwCreationDisposition = CREATE_ALWAYS; } else dwCreationDisposition = OPEN_EXISTING; /* Direct I/O. */ - if (LF_ISSET(WT_OPEN_DIRECTIO)) { + if (LF_ISSET(WT_FS_OPEN_DIRECTIO)) { f |= FILE_FLAG_NO_BUFFERING; win_fh->direct_io = true; } @@ -493,19 +495,19 @@ __win_open_file(WT_FILE_SYSTEM *file_system, WT_SESSION *wt_session, if (FLD_ISSET(conn->write_through, file_type)) f |= FILE_FLAG_WRITE_THROUGH; - if (file_type == WT_OPEN_FILE_TYPE_LOG && + if (file_type == WT_FS_OPEN_FILE_TYPE_LOG && FLD_ISSET(conn->txn_logsync, WT_LOG_DSYNC)) f |= FILE_FLAG_WRITE_THROUGH; /* Disable read-ahead on trees: it slows down random read workloads. */ - if (file_type == WT_OPEN_FILE_TYPE_DATA) + if (file_type == WT_FS_OPEN_FILE_TYPE_DATA) f |= FILE_FLAG_RANDOM_ACCESS; win_fh->filehandle = CreateFileA(name, desired_access, FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, dwCreationDisposition, f, NULL); if (win_fh->filehandle == INVALID_HANDLE_VALUE) { - if (LF_ISSET(WT_OPEN_CREATE) && + if (LF_ISSET(WT_FS_OPEN_CREATE) && GetLastError() == ERROR_FILE_EXISTS) win_fh->filehandle = CreateFileA(name, desired_access, FILE_SHARE_READ | FILE_SHARE_WRITE, @@ -528,7 +530,7 @@ __win_open_file(WT_FILE_SYSTEM *file_system, WT_SESSION *wt_session, * concurrently with reads on the file. Writes would also move the file * pointer. */ - if (!LF_ISSET(WT_OPEN_READONLY)) { + if (!LF_ISSET(WT_FS_OPEN_READONLY)) { win_fh->filehandle_secondary = CreateFileA(name, desired_access, FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, OPEN_EXISTING, f, NULL); diff --git a/src/third_party/wiredtiger/src/os_win/os_path.c b/src/third_party/wiredtiger/src/os_win/os_path.c index 220752ce7a1..74050600417 100644 --- a/src/third_party/wiredtiger/src/os_win/os_path.c +++ b/src/third_party/wiredtiger/src/os_win/os_path.c @@ -16,8 +16,30 @@ bool __wt_absolute_path(const char *path) { /* - * Check for a drive name (for example, "D:"), allow both forward and - * backward slashes. + * https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247 + * + * For Windows API functions that manipulate files, file names can often + * be relative to the current directory, while some APIs require a fully + * qualified path. A file name is relative to the current directory if + * it does not begin with one of the following: + * + * -- A UNC name of any format, which always start with two backslash + * characters ("\\"). + * -- A disk designator with a backslash, for example "C:\" or "d:\". + * -- A single backslash, for example, "\directory" or "\file.txt". This + * is also referred to as an absolute path. + * + * If a file name begins with only a disk designator but not the + * backslash after the colon, it is interpreted as a relative path to + * the current directory on the drive with the specified letter. Note + * that the current directory may or may not be the root directory + * depending on what it was set to during the most recent "change + * directory" operation on that disk. + * + * -- "C:tmp.txt" refers to a file named "tmp.txt" in the current + * directory on drive C. + * -- "C:tempdir\tmp.txt" refers to a file in a subdirectory to the + * current directory on drive C. */ if (strlen(path) >= 3 && __wt_isalpha(path[0]) && path[1] == ':') path += 2; diff --git a/src/third_party/wiredtiger/src/reconcile/rec_write.c b/src/third_party/wiredtiger/src/reconcile/rec_write.c index b49946bb10e..6bcb5457385 100644 --- a/src/third_party/wiredtiger/src/reconcile/rec_write.c +++ b/src/third_party/wiredtiger/src/reconcile/rec_write.c @@ -1135,8 +1135,18 @@ __rec_txn_read(WT_SESSION_IMPL *session, WT_RECONCILE *r, */ if (!skipped && (F_ISSET(btree, WT_BTREE_LOOKASIDE) || - __wt_txn_visible_all(session, max_txn))) + __wt_txn_visible_all(session, max_txn))) { + /* + * The checkpoint transaction is special. Make sure we never + * write (metadata) updates from a checkpoint in a concurrent + * session. + */ + WT_ASSERT(session, *updp == NULL || + (txnid = (*updp)->txnid) == WT_TXN_NONE || + txnid != S2C(session)->txn_global.checkpoint_txnid || + WT_SESSION_IS_CHECKPOINT(session)); return (0); + } /* * In some cases, there had better not be skipped updates or updates not diff --git a/src/third_party/wiredtiger/src/schema/schema_rename.c b/src/third_party/wiredtiger/src/schema/schema_rename.c index 8f4d374fd22..bc92c882117 100644 --- a/src/third_party/wiredtiger/src/schema/schema_rename.c +++ b/src/third_party/wiredtiger/src/schema/schema_rename.c @@ -64,7 +64,7 @@ __rename_file( WT_ERR(__wt_metadata_insert(session, newuri, oldvalue)); /* Rename the underlying file. */ - WT_ERR(__wt_fs_rename(session, filename, newfile)); + WT_ERR(__wt_fs_rename(session, filename, newfile, false)); if (WT_META_TRACKING(session)) WT_ERR(__wt_meta_track_fileop(session, uri, newuri)); diff --git a/src/third_party/wiredtiger/src/session/session_api.c b/src/third_party/wiredtiger/src/session/session_api.c index 77d1dc74c84..85d0fb1151c 100644 --- a/src/third_party/wiredtiger/src/session/session_api.c +++ b/src/third_party/wiredtiger/src/session/session_api.c @@ -66,6 +66,11 @@ __wt_session_copy_values(WT_SESSION_IMPL *session) TAILQ_FOREACH(cursor, &session->cursors, q) if (F_ISSET(cursor, WT_CURSTD_VALUE_INT)) { + /* We have to do this with a transaction ID pinned. */ + WT_ASSERT(session, + WT_SESSION_TXN_STATE(session)->snap_min != + WT_TXN_NONE); + F_CLR(cursor, WT_CURSTD_VALUE_INT); WT_RET(__wt_buf_set(session, &cursor->value, cursor->value.data, cursor->value.size)); diff --git a/src/third_party/wiredtiger/src/support/cksum.c b/src/third_party/wiredtiger/src/support/cksum.c deleted file mode 100644 index 0b086753406..00000000000 --- a/src/third_party/wiredtiger/src/support/cksum.c +++ /dev/null @@ -1,1327 +0,0 @@ -/*- - * Public Domain 2014-2016 MongoDB, Inc. - * Public Domain 2008-2014 WiredTiger, Inc. - * - * This is free and unencumbered software released into the public domain. - * - * Anyone is free to copy, modify, publish, use, compile, sell, or - * distribute this software, either in source code form or as a compiled - * binary, for any purpose, commercial or non-commercial, and by any - * means. - * - * In jurisdictions that recognize copyright laws, the author or authors - * of this software dedicate any and all copyright interest in the - * software to the public domain. We make this dedication for the benefit - * of the public at large and to the detriment of our heirs and - * successors. We intend this dedication to be an overt act of - * relinquishment in perpetuity of all present and future rights to this - * software under copyright law. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, - * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - * IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR - * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - * OTHER DEALINGS IN THE SOFTWARE. - */ -/* - * Slicing-by-8 algorithm by Michael E. Kounavis and Frank L. Berry, described - * in "Novel Table Lookup-Based Algorithms for High-Performance CRC Generation", - * IEEE Transactions on Computers, Volume 57 Issue 11, November 2008. - * - * See also Peter Kanowski's posting: - * http://www.strchr.com/crc32_popcnt - * - * The big endian version calculates the same result at each step, except the - * value of the crc is byte reversed from what it would be at that step for - * little endian. - */ - -#include "wt_internal.h" - -/* - * This file contains two implementations for computing CRC: one that uses - * hardware CRC instructions, available on newer x86_64/amd64, and one that uses - * a fast software algorithm. __wt_cksum() provides a common entry point that - * indirects to one of these two methods. - */ -static uint32_t (*__wt_cksum_func)(const void *chunk, size_t len); - -/* - * The CRC slicing tables are used by __wt_cksum_sw. - */ -static const uint32_t g_crc_slicing[8][256] = { -#ifdef WORDS_BIGENDIAN - /* - * Big endian tables have entries that are byte reversed from little - * endian tables. - */ - { - 0x00000000, 0x03836bf2, 0xf7703be1, 0xf4f35013, - 0x1f979ac7, 0x1c14f135, 0xe8e7a126, 0xeb64cad4, - 0xcf58d98a, 0xccdbb278, 0x3828e26b, 0x3bab8999, - 0xd0cf434d, 0xd34c28bf, 0x27bf78ac, 0x243c135e, - 0x6fc75e10, 0x6c4435e2, 0x98b765f1, 0x9b340e03, - 0x7050c4d7, 0x73d3af25, 0x8720ff36, 0x84a394c4, - 0xa09f879a, 0xa31cec68, 0x57efbc7b, 0x546cd789, - 0xbf081d5d, 0xbc8b76af, 0x487826bc, 0x4bfb4d4e, - 0xde8ebd20, 0xdd0dd6d2, 0x29fe86c1, 0x2a7ded33, - 0xc11927e7, 0xc29a4c15, 0x36691c06, 0x35ea77f4, - 0x11d664aa, 0x12550f58, 0xe6a65f4b, 0xe52534b9, - 0x0e41fe6d, 0x0dc2959f, 0xf931c58c, 0xfab2ae7e, - 0xb149e330, 0xb2ca88c2, 0x4639d8d1, 0x45bab323, - 0xaede79f7, 0xad5d1205, 0x59ae4216, 0x5a2d29e4, - 0x7e113aba, 0x7d925148, 0x8961015b, 0x8ae26aa9, - 0x6186a07d, 0x6205cb8f, 0x96f69b9c, 0x9575f06e, - 0xbc1d7b41, 0xbf9e10b3, 0x4b6d40a0, 0x48ee2b52, - 0xa38ae186, 0xa0098a74, 0x54fada67, 0x5779b195, - 0x7345a2cb, 0x70c6c939, 0x8435992a, 0x87b6f2d8, - 0x6cd2380c, 0x6f5153fe, 0x9ba203ed, 0x9821681f, - 0xd3da2551, 0xd0594ea3, 0x24aa1eb0, 0x27297542, - 0xcc4dbf96, 0xcfced464, 0x3b3d8477, 0x38beef85, - 0x1c82fcdb, 0x1f019729, 0xebf2c73a, 0xe871acc8, - 0x0315661c, 0x00960dee, 0xf4655dfd, 0xf7e6360f, - 0x6293c661, 0x6110ad93, 0x95e3fd80, 0x96609672, - 0x7d045ca6, 0x7e873754, 0x8a746747, 0x89f70cb5, - 0xadcb1feb, 0xae487419, 0x5abb240a, 0x59384ff8, - 0xb25c852c, 0xb1dfeede, 0x452cbecd, 0x46afd53f, - 0x0d549871, 0x0ed7f383, 0xfa24a390, 0xf9a7c862, - 0x12c302b6, 0x11406944, 0xe5b33957, 0xe63052a5, - 0xc20c41fb, 0xc18f2a09, 0x357c7a1a, 0x36ff11e8, - 0xdd9bdb3c, 0xde18b0ce, 0x2aebe0dd, 0x29688b2f, - 0x783bf682, 0x7bb89d70, 0x8f4bcd63, 0x8cc8a691, - 0x67ac6c45, 0x642f07b7, 0x90dc57a4, 0x935f3c56, - 0xb7632f08, 0xb4e044fa, 0x401314e9, 0x43907f1b, - 0xa8f4b5cf, 0xab77de3d, 0x5f848e2e, 0x5c07e5dc, - 0x17fca892, 0x147fc360, 0xe08c9373, 0xe30ff881, - 0x086b3255, 0x0be859a7, 0xff1b09b4, 0xfc986246, - 0xd8a47118, 0xdb271aea, 0x2fd44af9, 0x2c57210b, - 0xc733ebdf, 0xc4b0802d, 0x3043d03e, 0x33c0bbcc, - 0xa6b54ba2, 0xa5362050, 0x51c57043, 0x52461bb1, - 0xb922d165, 0xbaa1ba97, 0x4e52ea84, 0x4dd18176, - 0x69ed9228, 0x6a6ef9da, 0x9e9da9c9, 0x9d1ec23b, - 0x767a08ef, 0x75f9631d, 0x810a330e, 0x828958fc, - 0xc97215b2, 0xcaf17e40, 0x3e022e53, 0x3d8145a1, - 0xd6e58f75, 0xd566e487, 0x2195b494, 0x2216df66, - 0x062acc38, 0x05a9a7ca, 0xf15af7d9, 0xf2d99c2b, - 0x19bd56ff, 0x1a3e3d0d, 0xeecd6d1e, 0xed4e06ec, - 0xc4268dc3, 0xc7a5e631, 0x3356b622, 0x30d5ddd0, - 0xdbb11704, 0xd8327cf6, 0x2cc12ce5, 0x2f424717, - 0x0b7e5449, 0x08fd3fbb, 0xfc0e6fa8, 0xff8d045a, - 0x14e9ce8e, 0x176aa57c, 0xe399f56f, 0xe01a9e9d, - 0xabe1d3d3, 0xa862b821, 0x5c91e832, 0x5f1283c0, - 0xb4764914, 0xb7f522e6, 0x430672f5, 0x40851907, - 0x64b90a59, 0x673a61ab, 0x93c931b8, 0x904a5a4a, - 0x7b2e909e, 0x78adfb6c, 0x8c5eab7f, 0x8fddc08d, - 0x1aa830e3, 0x192b5b11, 0xedd80b02, 0xee5b60f0, - 0x053faa24, 0x06bcc1d6, 0xf24f91c5, 0xf1ccfa37, - 0xd5f0e969, 0xd673829b, 0x2280d288, 0x2103b97a, - 0xca6773ae, 0xc9e4185c, 0x3d17484f, 0x3e9423bd, - 0x756f6ef3, 0x76ec0501, 0x821f5512, 0x819c3ee0, - 0x6af8f434, 0x697b9fc6, 0x9d88cfd5, 0x9e0ba427, - 0xba37b779, 0xb9b4dc8b, 0x4d478c98, 0x4ec4e76a, - 0xa5a02dbe, 0xa623464c, 0x52d0165f, 0x51537dad - },{ - 0x00000000, 0x7798a213, 0xee304527, 0x99a8e734, - 0xdc618a4e, 0xabf9285d, 0x3251cf69, 0x45c96d7a, - 0xb8c3149d, 0xcf5bb68e, 0x56f351ba, 0x216bf3a9, - 0x64a29ed3, 0x133a3cc0, 0x8a92dbf4, 0xfd0a79e7, - 0x81f1c53f, 0xf669672c, 0x6fc18018, 0x1859220b, - 0x5d904f71, 0x2a08ed62, 0xb3a00a56, 0xc438a845, - 0x3932d1a2, 0x4eaa73b1, 0xd7029485, 0xa09a3696, - 0xe5535bec, 0x92cbf9ff, 0x0b631ecb, 0x7cfbbcd8, - 0x02e38b7f, 0x757b296c, 0xecd3ce58, 0x9b4b6c4b, - 0xde820131, 0xa91aa322, 0x30b24416, 0x472ae605, - 0xba209fe2, 0xcdb83df1, 0x5410dac5, 0x238878d6, - 0x664115ac, 0x11d9b7bf, 0x8871508b, 0xffe9f298, - 0x83124e40, 0xf48aec53, 0x6d220b67, 0x1abaa974, - 0x5f73c40e, 0x28eb661d, 0xb1438129, 0xc6db233a, - 0x3bd15add, 0x4c49f8ce, 0xd5e11ffa, 0xa279bde9, - 0xe7b0d093, 0x90287280, 0x098095b4, 0x7e1837a7, - 0x04c617ff, 0x735eb5ec, 0xeaf652d8, 0x9d6ef0cb, - 0xd8a79db1, 0xaf3f3fa2, 0x3697d896, 0x410f7a85, - 0xbc050362, 0xcb9da171, 0x52354645, 0x25ade456, - 0x6064892c, 0x17fc2b3f, 0x8e54cc0b, 0xf9cc6e18, - 0x8537d2c0, 0xf2af70d3, 0x6b0797e7, 0x1c9f35f4, - 0x5956588e, 0x2ecefa9d, 0xb7661da9, 0xc0febfba, - 0x3df4c65d, 0x4a6c644e, 0xd3c4837a, 0xa45c2169, - 0xe1954c13, 0x960dee00, 0x0fa50934, 0x783dab27, - 0x06259c80, 0x71bd3e93, 0xe815d9a7, 0x9f8d7bb4, - 0xda4416ce, 0xaddcb4dd, 0x347453e9, 0x43ecf1fa, - 0xbee6881d, 0xc97e2a0e, 0x50d6cd3a, 0x274e6f29, - 0x62870253, 0x151fa040, 0x8cb74774, 0xfb2fe567, - 0x87d459bf, 0xf04cfbac, 0x69e41c98, 0x1e7cbe8b, - 0x5bb5d3f1, 0x2c2d71e2, 0xb58596d6, 0xc21d34c5, - 0x3f174d22, 0x488fef31, 0xd1270805, 0xa6bfaa16, - 0xe376c76c, 0x94ee657f, 0x0d46824b, 0x7ade2058, - 0xf9fac3fb, 0x8e6261e8, 0x17ca86dc, 0x605224cf, - 0x259b49b5, 0x5203eba6, 0xcbab0c92, 0xbc33ae81, - 0x4139d766, 0x36a17575, 0xaf099241, 0xd8913052, - 0x9d585d28, 0xeac0ff3b, 0x7368180f, 0x04f0ba1c, - 0x780b06c4, 0x0f93a4d7, 0x963b43e3, 0xe1a3e1f0, - 0xa46a8c8a, 0xd3f22e99, 0x4a5ac9ad, 0x3dc26bbe, - 0xc0c81259, 0xb750b04a, 0x2ef8577e, 0x5960f56d, - 0x1ca99817, 0x6b313a04, 0xf299dd30, 0x85017f23, - 0xfb194884, 0x8c81ea97, 0x15290da3, 0x62b1afb0, - 0x2778c2ca, 0x50e060d9, 0xc94887ed, 0xbed025fe, - 0x43da5c19, 0x3442fe0a, 0xadea193e, 0xda72bb2d, - 0x9fbbd657, 0xe8237444, 0x718b9370, 0x06133163, - 0x7ae88dbb, 0x0d702fa8, 0x94d8c89c, 0xe3406a8f, - 0xa68907f5, 0xd111a5e6, 0x48b942d2, 0x3f21e0c1, - 0xc22b9926, 0xb5b33b35, 0x2c1bdc01, 0x5b837e12, - 0x1e4a1368, 0x69d2b17b, 0xf07a564f, 0x87e2f45c, - 0xfd3cd404, 0x8aa47617, 0x130c9123, 0x64943330, - 0x215d5e4a, 0x56c5fc59, 0xcf6d1b6d, 0xb8f5b97e, - 0x45ffc099, 0x3267628a, 0xabcf85be, 0xdc5727ad, - 0x999e4ad7, 0xee06e8c4, 0x77ae0ff0, 0x0036ade3, - 0x7ccd113b, 0x0b55b328, 0x92fd541c, 0xe565f60f, - 0xa0ac9b75, 0xd7343966, 0x4e9cde52, 0x39047c41, - 0xc40e05a6, 0xb396a7b5, 0x2a3e4081, 0x5da6e292, - 0x186f8fe8, 0x6ff72dfb, 0xf65fcacf, 0x81c768dc, - 0xffdf5f7b, 0x8847fd68, 0x11ef1a5c, 0x6677b84f, - 0x23bed535, 0x54267726, 0xcd8e9012, 0xba163201, - 0x471c4be6, 0x3084e9f5, 0xa92c0ec1, 0xdeb4acd2, - 0x9b7dc1a8, 0xece563bb, 0x754d848f, 0x02d5269c, - 0x7e2e9a44, 0x09b63857, 0x901edf63, 0xe7867d70, - 0xa24f100a, 0xd5d7b219, 0x4c7f552d, 0x3be7f73e, - 0xc6ed8ed9, 0xb1752cca, 0x28ddcbfe, 0x5f4569ed, - 0x1a8c0497, 0x6d14a684, 0xf4bc41b0, 0x8324e3a3 - },{ - 0x00000000, 0x7e9241a5, 0x0d526f4f, 0x73c02eea, - 0x1aa4de9e, 0x64369f3b, 0x17f6b1d1, 0x6964f074, - 0xc53e5138, 0xbbac109d, 0xc86c3e77, 0xb6fe7fd2, - 0xdf9a8fa6, 0xa108ce03, 0xd2c8e0e9, 0xac5aa14c, - 0x8a7da270, 0xf4efe3d5, 0x872fcd3f, 0xf9bd8c9a, - 0x90d97cee, 0xee4b3d4b, 0x9d8b13a1, 0xe3195204, - 0x4f43f348, 0x31d1b2ed, 0x42119c07, 0x3c83dda2, - 0x55e72dd6, 0x2b756c73, 0x58b54299, 0x2627033c, - 0x14fb44e1, 0x6a690544, 0x19a92bae, 0x673b6a0b, - 0x0e5f9a7f, 0x70cddbda, 0x030df530, 0x7d9fb495, - 0xd1c515d9, 0xaf57547c, 0xdc977a96, 0xa2053b33, - 0xcb61cb47, 0xb5f38ae2, 0xc633a408, 0xb8a1e5ad, - 0x9e86e691, 0xe014a734, 0x93d489de, 0xed46c87b, - 0x8422380f, 0xfab079aa, 0x89705740, 0xf7e216e5, - 0x5bb8b7a9, 0x252af60c, 0x56ead8e6, 0x28789943, - 0x411c6937, 0x3f8e2892, 0x4c4e0678, 0x32dc47dd, - 0xd98065c7, 0xa7122462, 0xd4d20a88, 0xaa404b2d, - 0xc324bb59, 0xbdb6fafc, 0xce76d416, 0xb0e495b3, - 0x1cbe34ff, 0x622c755a, 0x11ec5bb0, 0x6f7e1a15, - 0x061aea61, 0x7888abc4, 0x0b48852e, 0x75dac48b, - 0x53fdc7b7, 0x2d6f8612, 0x5eafa8f8, 0x203de95d, - 0x49591929, 0x37cb588c, 0x440b7666, 0x3a9937c3, - 0x96c3968f, 0xe851d72a, 0x9b91f9c0, 0xe503b865, - 0x8c674811, 0xf2f509b4, 0x8135275e, 0xffa766fb, - 0xcd7b2126, 0xb3e96083, 0xc0294e69, 0xbebb0fcc, - 0xd7dfffb8, 0xa94dbe1d, 0xda8d90f7, 0xa41fd152, - 0x0845701e, 0x76d731bb, 0x05171f51, 0x7b855ef4, - 0x12e1ae80, 0x6c73ef25, 0x1fb3c1cf, 0x6121806a, - 0x47068356, 0x3994c2f3, 0x4a54ec19, 0x34c6adbc, - 0x5da25dc8, 0x23301c6d, 0x50f03287, 0x2e627322, - 0x8238d26e, 0xfcaa93cb, 0x8f6abd21, 0xf1f8fc84, - 0x989c0cf0, 0xe60e4d55, 0x95ce63bf, 0xeb5c221a, - 0x4377278b, 0x3de5662e, 0x4e2548c4, 0x30b70961, - 0x59d3f915, 0x2741b8b0, 0x5481965a, 0x2a13d7ff, - 0x864976b3, 0xf8db3716, 0x8b1b19fc, 0xf5895859, - 0x9ceda82d, 0xe27fe988, 0x91bfc762, 0xef2d86c7, - 0xc90a85fb, 0xb798c45e, 0xc458eab4, 0xbacaab11, - 0xd3ae5b65, 0xad3c1ac0, 0xdefc342a, 0xa06e758f, - 0x0c34d4c3, 0x72a69566, 0x0166bb8c, 0x7ff4fa29, - 0x16900a5d, 0x68024bf8, 0x1bc26512, 0x655024b7, - 0x578c636a, 0x291e22cf, 0x5ade0c25, 0x244c4d80, - 0x4d28bdf4, 0x33bafc51, 0x407ad2bb, 0x3ee8931e, - 0x92b23252, 0xec2073f7, 0x9fe05d1d, 0xe1721cb8, - 0x8816eccc, 0xf684ad69, 0x85448383, 0xfbd6c226, - 0xddf1c11a, 0xa36380bf, 0xd0a3ae55, 0xae31eff0, - 0xc7551f84, 0xb9c75e21, 0xca0770cb, 0xb495316e, - 0x18cf9022, 0x665dd187, 0x159dff6d, 0x6b0fbec8, - 0x026b4ebc, 0x7cf90f19, 0x0f3921f3, 0x71ab6056, - 0x9af7424c, 0xe46503e9, 0x97a52d03, 0xe9376ca6, - 0x80539cd2, 0xfec1dd77, 0x8d01f39d, 0xf393b238, - 0x5fc91374, 0x215b52d1, 0x529b7c3b, 0x2c093d9e, - 0x456dcdea, 0x3bff8c4f, 0x483fa2a5, 0x36ade300, - 0x108ae03c, 0x6e18a199, 0x1dd88f73, 0x634aced6, - 0x0a2e3ea2, 0x74bc7f07, 0x077c51ed, 0x79ee1048, - 0xd5b4b104, 0xab26f0a1, 0xd8e6de4b, 0xa6749fee, - 0xcf106f9a, 0xb1822e3f, 0xc24200d5, 0xbcd04170, - 0x8e0c06ad, 0xf09e4708, 0x835e69e2, 0xfdcc2847, - 0x94a8d833, 0xea3a9996, 0x99fab77c, 0xe768f6d9, - 0x4b325795, 0x35a01630, 0x466038da, 0x38f2797f, - 0x5196890b, 0x2f04c8ae, 0x5cc4e644, 0x2256a7e1, - 0x0471a4dd, 0x7ae3e578, 0x0923cb92, 0x77b18a37, - 0x1ed57a43, 0x60473be6, 0x1387150c, 0x6d1554a9, - 0xc14ff5e5, 0xbfddb440, 0xcc1d9aaa, 0xb28fdb0f, - 0xdbeb2b7b, 0xa5796ade, 0xd6b94434, 0xa82b0591 - },{ - 0x00000000, 0xb8aa45dd, 0x812367bf, 0x39892262, - 0xf331227b, 0x4b9b67a6, 0x721245c4, 0xcab80019, - 0xe66344f6, 0x5ec9012b, 0x67402349, 0xdfea6694, - 0x1552668d, 0xadf82350, 0x94710132, 0x2cdb44ef, - 0x3db164e9, 0x851b2134, 0xbc920356, 0x0438468b, - 0xce804692, 0x762a034f, 0x4fa3212d, 0xf70964f0, - 0xdbd2201f, 0x637865c2, 0x5af147a0, 0xe25b027d, - 0x28e30264, 0x904947b9, 0xa9c065db, 0x116a2006, - 0x8b1425d7, 0x33be600a, 0x0a374268, 0xb29d07b5, - 0x782507ac, 0xc08f4271, 0xf9066013, 0x41ac25ce, - 0x6d776121, 0xd5dd24fc, 0xec54069e, 0x54fe4343, - 0x9e46435a, 0x26ec0687, 0x1f6524e5, 0xa7cf6138, - 0xb6a5413e, 0x0e0f04e3, 0x37862681, 0x8f2c635c, - 0x45946345, 0xfd3e2698, 0xc4b704fa, 0x7c1d4127, - 0x50c605c8, 0xe86c4015, 0xd1e56277, 0x694f27aa, - 0xa3f727b3, 0x1b5d626e, 0x22d4400c, 0x9a7e05d1, - 0xe75fa6ab, 0x5ff5e376, 0x667cc114, 0xded684c9, - 0x146e84d0, 0xacc4c10d, 0x954de36f, 0x2de7a6b2, - 0x013ce25d, 0xb996a780, 0x801f85e2, 0x38b5c03f, - 0xf20dc026, 0x4aa785fb, 0x732ea799, 0xcb84e244, - 0xdaeec242, 0x6244879f, 0x5bcda5fd, 0xe367e020, - 0x29dfe039, 0x9175a5e4, 0xa8fc8786, 0x1056c25b, - 0x3c8d86b4, 0x8427c369, 0xbdaee10b, 0x0504a4d6, - 0xcfbca4cf, 0x7716e112, 0x4e9fc370, 0xf63586ad, - 0x6c4b837c, 0xd4e1c6a1, 0xed68e4c3, 0x55c2a11e, - 0x9f7aa107, 0x27d0e4da, 0x1e59c6b8, 0xa6f38365, - 0x8a28c78a, 0x32828257, 0x0b0ba035, 0xb3a1e5e8, - 0x7919e5f1, 0xc1b3a02c, 0xf83a824e, 0x4090c793, - 0x51fae795, 0xe950a248, 0xd0d9802a, 0x6873c5f7, - 0xa2cbc5ee, 0x1a618033, 0x23e8a251, 0x9b42e78c, - 0xb799a363, 0x0f33e6be, 0x36bac4dc, 0x8e108101, - 0x44a88118, 0xfc02c4c5, 0xc58be6a7, 0x7d21a37a, - 0x3fc9a052, 0x8763e58f, 0xbeeac7ed, 0x06408230, - 0xccf88229, 0x7452c7f4, 0x4ddbe596, 0xf571a04b, - 0xd9aae4a4, 0x6100a179, 0x5889831b, 0xe023c6c6, - 0x2a9bc6df, 0x92318302, 0xabb8a160, 0x1312e4bd, - 0x0278c4bb, 0xbad28166, 0x835ba304, 0x3bf1e6d9, - 0xf149e6c0, 0x49e3a31d, 0x706a817f, 0xc8c0c4a2, - 0xe41b804d, 0x5cb1c590, 0x6538e7f2, 0xdd92a22f, - 0x172aa236, 0xaf80e7eb, 0x9609c589, 0x2ea38054, - 0xb4dd8585, 0x0c77c058, 0x35fee23a, 0x8d54a7e7, - 0x47eca7fe, 0xff46e223, 0xc6cfc041, 0x7e65859c, - 0x52bec173, 0xea1484ae, 0xd39da6cc, 0x6b37e311, - 0xa18fe308, 0x1925a6d5, 0x20ac84b7, 0x9806c16a, - 0x896ce16c, 0x31c6a4b1, 0x084f86d3, 0xb0e5c30e, - 0x7a5dc317, 0xc2f786ca, 0xfb7ea4a8, 0x43d4e175, - 0x6f0fa59a, 0xd7a5e047, 0xee2cc225, 0x568687f8, - 0x9c3e87e1, 0x2494c23c, 0x1d1de05e, 0xa5b7a583, - 0xd89606f9, 0x603c4324, 0x59b56146, 0xe11f249b, - 0x2ba72482, 0x930d615f, 0xaa84433d, 0x122e06e0, - 0x3ef5420f, 0x865f07d2, 0xbfd625b0, 0x077c606d, - 0xcdc46074, 0x756e25a9, 0x4ce707cb, 0xf44d4216, - 0xe5276210, 0x5d8d27cd, 0x640405af, 0xdcae4072, - 0x1616406b, 0xaebc05b6, 0x973527d4, 0x2f9f6209, - 0x034426e6, 0xbbee633b, 0x82674159, 0x3acd0484, - 0xf075049d, 0x48df4140, 0x71566322, 0xc9fc26ff, - 0x5382232e, 0xeb2866f3, 0xd2a14491, 0x6a0b014c, - 0xa0b30155, 0x18194488, 0x219066ea, 0x993a2337, - 0xb5e167d8, 0x0d4b2205, 0x34c20067, 0x8c6845ba, - 0x46d045a3, 0xfe7a007e, 0xc7f3221c, 0x7f5967c1, - 0x6e3347c7, 0xd699021a, 0xef102078, 0x57ba65a5, - 0x9d0265bc, 0x25a82061, 0x1c210203, 0xa48b47de, - 0x88500331, 0x30fa46ec, 0x0973648e, 0xb1d92153, - 0x7b61214a, 0xc3cb6497, 0xfa4246f5, 0x42e80328 - },{ - 0x00000000, 0xac6f1138, 0x58df2270, 0xf4b03348, - 0xb0be45e0, 0x1cd154d8, 0xe8616790, 0x440e76a8, - 0x910b67c5, 0x3d6476fd, 0xc9d445b5, 0x65bb548d, - 0x21b52225, 0x8dda331d, 0x796a0055, 0xd505116d, - 0xd361228f, 0x7f0e33b7, 0x8bbe00ff, 0x27d111c7, - 0x63df676f, 0xcfb07657, 0x3b00451f, 0x976f5427, - 0x426a454a, 0xee055472, 0x1ab5673a, 0xb6da7602, - 0xf2d400aa, 0x5ebb1192, 0xaa0b22da, 0x066433e2, - 0x57b5a81b, 0xfbdab923, 0x0f6a8a6b, 0xa3059b53, - 0xe70bedfb, 0x4b64fcc3, 0xbfd4cf8b, 0x13bbdeb3, - 0xc6becfde, 0x6ad1dee6, 0x9e61edae, 0x320efc96, - 0x76008a3e, 0xda6f9b06, 0x2edfa84e, 0x82b0b976, - 0x84d48a94, 0x28bb9bac, 0xdc0ba8e4, 0x7064b9dc, - 0x346acf74, 0x9805de4c, 0x6cb5ed04, 0xc0dafc3c, - 0x15dfed51, 0xb9b0fc69, 0x4d00cf21, 0xe16fde19, - 0xa561a8b1, 0x090eb989, 0xfdbe8ac1, 0x51d19bf9, - 0xae6a5137, 0x0205400f, 0xf6b57347, 0x5ada627f, - 0x1ed414d7, 0xb2bb05ef, 0x460b36a7, 0xea64279f, - 0x3f6136f2, 0x930e27ca, 0x67be1482, 0xcbd105ba, - 0x8fdf7312, 0x23b0622a, 0xd7005162, 0x7b6f405a, - 0x7d0b73b8, 0xd1646280, 0x25d451c8, 0x89bb40f0, - 0xcdb53658, 0x61da2760, 0x956a1428, 0x39050510, - 0xec00147d, 0x406f0545, 0xb4df360d, 0x18b02735, - 0x5cbe519d, 0xf0d140a5, 0x046173ed, 0xa80e62d5, - 0xf9dff92c, 0x55b0e814, 0xa100db5c, 0x0d6fca64, - 0x4961bccc, 0xe50eadf4, 0x11be9ebc, 0xbdd18f84, - 0x68d49ee9, 0xc4bb8fd1, 0x300bbc99, 0x9c64ada1, - 0xd86adb09, 0x7405ca31, 0x80b5f979, 0x2cdae841, - 0x2abedba3, 0x86d1ca9b, 0x7261f9d3, 0xde0ee8eb, - 0x9a009e43, 0x366f8f7b, 0xc2dfbc33, 0x6eb0ad0b, - 0xbbb5bc66, 0x17daad5e, 0xe36a9e16, 0x4f058f2e, - 0x0b0bf986, 0xa764e8be, 0x53d4dbf6, 0xffbbcace, - 0x5cd5a26e, 0xf0bab356, 0x040a801e, 0xa8659126, - 0xec6be78e, 0x4004f6b6, 0xb4b4c5fe, 0x18dbd4c6, - 0xcddec5ab, 0x61b1d493, 0x9501e7db, 0x396ef6e3, - 0x7d60804b, 0xd10f9173, 0x25bfa23b, 0x89d0b303, - 0x8fb480e1, 0x23db91d9, 0xd76ba291, 0x7b04b3a9, - 0x3f0ac501, 0x9365d439, 0x67d5e771, 0xcbbaf649, - 0x1ebfe724, 0xb2d0f61c, 0x4660c554, 0xea0fd46c, - 0xae01a2c4, 0x026eb3fc, 0xf6de80b4, 0x5ab1918c, - 0x0b600a75, 0xa70f1b4d, 0x53bf2805, 0xffd0393d, - 0xbbde4f95, 0x17b15ead, 0xe3016de5, 0x4f6e7cdd, - 0x9a6b6db0, 0x36047c88, 0xc2b44fc0, 0x6edb5ef8, - 0x2ad52850, 0x86ba3968, 0x720a0a20, 0xde651b18, - 0xd80128fa, 0x746e39c2, 0x80de0a8a, 0x2cb11bb2, - 0x68bf6d1a, 0xc4d07c22, 0x30604f6a, 0x9c0f5e52, - 0x490a4f3f, 0xe5655e07, 0x11d56d4f, 0xbdba7c77, - 0xf9b40adf, 0x55db1be7, 0xa16b28af, 0x0d043997, - 0xf2bff359, 0x5ed0e261, 0xaa60d129, 0x060fc011, - 0x4201b6b9, 0xee6ea781, 0x1ade94c9, 0xb6b185f1, - 0x63b4949c, 0xcfdb85a4, 0x3b6bb6ec, 0x9704a7d4, - 0xd30ad17c, 0x7f65c044, 0x8bd5f30c, 0x27bae234, - 0x21ded1d6, 0x8db1c0ee, 0x7901f3a6, 0xd56ee29e, - 0x91609436, 0x3d0f850e, 0xc9bfb646, 0x65d0a77e, - 0xb0d5b613, 0x1cbaa72b, 0xe80a9463, 0x4465855b, - 0x006bf3f3, 0xac04e2cb, 0x58b4d183, 0xf4dbc0bb, - 0xa50a5b42, 0x09654a7a, 0xfdd57932, 0x51ba680a, - 0x15b41ea2, 0xb9db0f9a, 0x4d6b3cd2, 0xe1042dea, - 0x34013c87, 0x986e2dbf, 0x6cde1ef7, 0xc0b10fcf, - 0x84bf7967, 0x28d0685f, 0xdc605b17, 0x700f4a2f, - 0x766b79cd, 0xda0468f5, 0x2eb45bbd, 0x82db4a85, - 0xc6d53c2d, 0x6aba2d15, 0x9e0a1e5d, 0x32650f65, - 0xe7601e08, 0x4b0f0f30, 0xbfbf3c78, 0x13d02d40, - 0x57de5be8, 0xfbb14ad0, 0x0f017998, 0xa36e68a0 - },{ - 0x00000000, 0x196b30ef, 0xc3a08cdb, 0xdacbbc34, - 0x7737f5b2, 0x6e5cc55d, 0xb4977969, 0xadfc4986, - 0x1f180660, 0x0673368f, 0xdcb88abb, 0xc5d3ba54, - 0x682ff3d2, 0x7144c33d, 0xab8f7f09, 0xb2e44fe6, - 0x3e300cc0, 0x275b3c2f, 0xfd90801b, 0xe4fbb0f4, - 0x4907f972, 0x506cc99d, 0x8aa775a9, 0x93cc4546, - 0x21280aa0, 0x38433a4f, 0xe288867b, 0xfbe3b694, - 0x561fff12, 0x4f74cffd, 0x95bf73c9, 0x8cd44326, - 0x8d16f485, 0x947dc46a, 0x4eb6785e, 0x57dd48b1, - 0xfa210137, 0xe34a31d8, 0x39818dec, 0x20eabd03, - 0x920ef2e5, 0x8b65c20a, 0x51ae7e3e, 0x48c54ed1, - 0xe5390757, 0xfc5237b8, 0x26998b8c, 0x3ff2bb63, - 0xb326f845, 0xaa4dc8aa, 0x7086749e, 0x69ed4471, - 0xc4110df7, 0xdd7a3d18, 0x07b1812c, 0x1edab1c3, - 0xac3efe25, 0xb555ceca, 0x6f9e72fe, 0x76f54211, - 0xdb090b97, 0xc2623b78, 0x18a9874c, 0x01c2b7a3, - 0xeb5b040e, 0xf23034e1, 0x28fb88d5, 0x3190b83a, - 0x9c6cf1bc, 0x8507c153, 0x5fcc7d67, 0x46a74d88, - 0xf443026e, 0xed283281, 0x37e38eb5, 0x2e88be5a, - 0x8374f7dc, 0x9a1fc733, 0x40d47b07, 0x59bf4be8, - 0xd56b08ce, 0xcc003821, 0x16cb8415, 0x0fa0b4fa, - 0xa25cfd7c, 0xbb37cd93, 0x61fc71a7, 0x78974148, - 0xca730eae, 0xd3183e41, 0x09d38275, 0x10b8b29a, - 0xbd44fb1c, 0xa42fcbf3, 0x7ee477c7, 0x678f4728, - 0x664df08b, 0x7f26c064, 0xa5ed7c50, 0xbc864cbf, - 0x117a0539, 0x081135d6, 0xd2da89e2, 0xcbb1b90d, - 0x7955f6eb, 0x603ec604, 0xbaf57a30, 0xa39e4adf, - 0x0e620359, 0x170933b6, 0xcdc28f82, 0xd4a9bf6d, - 0x587dfc4b, 0x4116cca4, 0x9bdd7090, 0x82b6407f, - 0x2f4a09f9, 0x36213916, 0xecea8522, 0xf581b5cd, - 0x4765fa2b, 0x5e0ecac4, 0x84c576f0, 0x9dae461f, - 0x30520f99, 0x29393f76, 0xf3f28342, 0xea99b3ad, - 0xd6b7081c, 0xcfdc38f3, 0x151784c7, 0x0c7cb428, - 0xa180fdae, 0xb8ebcd41, 0x62207175, 0x7b4b419a, - 0xc9af0e7c, 0xd0c43e93, 0x0a0f82a7, 0x1364b248, - 0xbe98fbce, 0xa7f3cb21, 0x7d387715, 0x645347fa, - 0xe88704dc, 0xf1ec3433, 0x2b278807, 0x324cb8e8, - 0x9fb0f16e, 0x86dbc181, 0x5c107db5, 0x457b4d5a, - 0xf79f02bc, 0xeef43253, 0x343f8e67, 0x2d54be88, - 0x80a8f70e, 0x99c3c7e1, 0x43087bd5, 0x5a634b3a, - 0x5ba1fc99, 0x42cacc76, 0x98017042, 0x816a40ad, - 0x2c96092b, 0x35fd39c4, 0xef3685f0, 0xf65db51f, - 0x44b9faf9, 0x5dd2ca16, 0x87197622, 0x9e7246cd, - 0x338e0f4b, 0x2ae53fa4, 0xf02e8390, 0xe945b37f, - 0x6591f059, 0x7cfac0b6, 0xa6317c82, 0xbf5a4c6d, - 0x12a605eb, 0x0bcd3504, 0xd1068930, 0xc86db9df, - 0x7a89f639, 0x63e2c6d6, 0xb9297ae2, 0xa0424a0d, - 0x0dbe038b, 0x14d53364, 0xce1e8f50, 0xd775bfbf, - 0x3dec0c12, 0x24873cfd, 0xfe4c80c9, 0xe727b026, - 0x4adbf9a0, 0x53b0c94f, 0x897b757b, 0x90104594, - 0x22f40a72, 0x3b9f3a9d, 0xe15486a9, 0xf83fb646, - 0x55c3ffc0, 0x4ca8cf2f, 0x9663731b, 0x8f0843f4, - 0x03dc00d2, 0x1ab7303d, 0xc07c8c09, 0xd917bce6, - 0x74ebf560, 0x6d80c58f, 0xb74b79bb, 0xae204954, - 0x1cc406b2, 0x05af365d, 0xdf648a69, 0xc60fba86, - 0x6bf3f300, 0x7298c3ef, 0xa8537fdb, 0xb1384f34, - 0xb0faf897, 0xa991c878, 0x735a744c, 0x6a3144a3, - 0xc7cd0d25, 0xdea63dca, 0x046d81fe, 0x1d06b111, - 0xafe2fef7, 0xb689ce18, 0x6c42722c, 0x752942c3, - 0xd8d50b45, 0xc1be3baa, 0x1b75879e, 0x021eb771, - 0x8ecaf457, 0x97a1c4b8, 0x4d6a788c, 0x54014863, - 0xf9fd01e5, 0xe096310a, 0x3a5d8d3e, 0x2336bdd1, - 0x91d2f237, 0x88b9c2d8, 0x52727eec, 0x4b194e03, - 0xe6e50785, 0xff8e376a, 0x25458b5e, 0x3c2ebbb1 - },{ - 0x00000000, 0xc82c0368, 0x905906d0, 0x587505b8, - 0xd1c5e0a5, 0x19e9e3cd, 0x419ce675, 0x89b0e51d, - 0x53fd2d4e, 0x9bd12e26, 0xc3a42b9e, 0x0b8828f6, - 0x8238cdeb, 0x4a14ce83, 0x1261cb3b, 0xda4dc853, - 0xa6fa5b9c, 0x6ed658f4, 0x36a35d4c, 0xfe8f5e24, - 0x773fbb39, 0xbf13b851, 0xe766bde9, 0x2f4abe81, - 0xf50776d2, 0x3d2b75ba, 0x655e7002, 0xad72736a, - 0x24c29677, 0xecee951f, 0xb49b90a7, 0x7cb793cf, - 0xbd835b3d, 0x75af5855, 0x2dda5ded, 0xe5f65e85, - 0x6c46bb98, 0xa46ab8f0, 0xfc1fbd48, 0x3433be20, - 0xee7e7673, 0x2652751b, 0x7e2770a3, 0xb60b73cb, - 0x3fbb96d6, 0xf79795be, 0xafe29006, 0x67ce936e, - 0x1b7900a1, 0xd35503c9, 0x8b200671, 0x430c0519, - 0xcabce004, 0x0290e36c, 0x5ae5e6d4, 0x92c9e5bc, - 0x48842def, 0x80a82e87, 0xd8dd2b3f, 0x10f12857, - 0x9941cd4a, 0x516dce22, 0x0918cb9a, 0xc134c8f2, - 0x7a07b77a, 0xb22bb412, 0xea5eb1aa, 0x2272b2c2, - 0xabc257df, 0x63ee54b7, 0x3b9b510f, 0xf3b75267, - 0x29fa9a34, 0xe1d6995c, 0xb9a39ce4, 0x718f9f8c, - 0xf83f7a91, 0x301379f9, 0x68667c41, 0xa04a7f29, - 0xdcfdece6, 0x14d1ef8e, 0x4ca4ea36, 0x8488e95e, - 0x0d380c43, 0xc5140f2b, 0x9d610a93, 0x554d09fb, - 0x8f00c1a8, 0x472cc2c0, 0x1f59c778, 0xd775c410, - 0x5ec5210d, 0x96e92265, 0xce9c27dd, 0x06b024b5, - 0xc784ec47, 0x0fa8ef2f, 0x57ddea97, 0x9ff1e9ff, - 0x16410ce2, 0xde6d0f8a, 0x86180a32, 0x4e34095a, - 0x9479c109, 0x5c55c261, 0x0420c7d9, 0xcc0cc4b1, - 0x45bc21ac, 0x8d9022c4, 0xd5e5277c, 0x1dc92414, - 0x617eb7db, 0xa952b4b3, 0xf127b10b, 0x390bb263, - 0xb0bb577e, 0x78975416, 0x20e251ae, 0xe8ce52c6, - 0x32839a95, 0xfaaf99fd, 0xa2da9c45, 0x6af69f2d, - 0xe3467a30, 0x2b6a7958, 0x731f7ce0, 0xbb337f88, - 0xf40e6ef5, 0x3c226d9d, 0x64576825, 0xac7b6b4d, - 0x25cb8e50, 0xede78d38, 0xb5928880, 0x7dbe8be8, - 0xa7f343bb, 0x6fdf40d3, 0x37aa456b, 0xff864603, - 0x7636a31e, 0xbe1aa076, 0xe66fa5ce, 0x2e43a6a6, - 0x52f43569, 0x9ad83601, 0xc2ad33b9, 0x0a8130d1, - 0x8331d5cc, 0x4b1dd6a4, 0x1368d31c, 0xdb44d074, - 0x01091827, 0xc9251b4f, 0x91501ef7, 0x597c1d9f, - 0xd0ccf882, 0x18e0fbea, 0x4095fe52, 0x88b9fd3a, - 0x498d35c8, 0x81a136a0, 0xd9d43318, 0x11f83070, - 0x9848d56d, 0x5064d605, 0x0811d3bd, 0xc03dd0d5, - 0x1a701886, 0xd25c1bee, 0x8a291e56, 0x42051d3e, - 0xcbb5f823, 0x0399fb4b, 0x5becfef3, 0x93c0fd9b, - 0xef776e54, 0x275b6d3c, 0x7f2e6884, 0xb7026bec, - 0x3eb28ef1, 0xf69e8d99, 0xaeeb8821, 0x66c78b49, - 0xbc8a431a, 0x74a64072, 0x2cd345ca, 0xe4ff46a2, - 0x6d4fa3bf, 0xa563a0d7, 0xfd16a56f, 0x353aa607, - 0x8e09d98f, 0x4625dae7, 0x1e50df5f, 0xd67cdc37, - 0x5fcc392a, 0x97e03a42, 0xcf953ffa, 0x07b93c92, - 0xddf4f4c1, 0x15d8f7a9, 0x4dadf211, 0x8581f179, - 0x0c311464, 0xc41d170c, 0x9c6812b4, 0x544411dc, - 0x28f38213, 0xe0df817b, 0xb8aa84c3, 0x708687ab, - 0xf93662b6, 0x311a61de, 0x696f6466, 0xa143670e, - 0x7b0eaf5d, 0xb322ac35, 0xeb57a98d, 0x237baae5, - 0xaacb4ff8, 0x62e74c90, 0x3a924928, 0xf2be4a40, - 0x338a82b2, 0xfba681da, 0xa3d38462, 0x6bff870a, - 0xe24f6217, 0x2a63617f, 0x721664c7, 0xba3a67af, - 0x6077affc, 0xa85bac94, 0xf02ea92c, 0x3802aa44, - 0xb1b24f59, 0x799e4c31, 0x21eb4989, 0xe9c74ae1, - 0x9570d92e, 0x5d5cda46, 0x0529dffe, 0xcd05dc96, - 0x44b5398b, 0x8c993ae3, 0xd4ec3f5b, 0x1cc03c33, - 0xc68df460, 0x0ea1f708, 0x56d4f2b0, 0x9ef8f1d8, - 0x174814c5, 0xdf6417ad, 0x87111215, 0x4f3d117d - },{ - 0x00000000, 0x277d3c49, 0x4efa7892, 0x698744db, - 0x6d821d21, 0x4aff2168, 0x237865b3, 0x040559fa, - 0xda043b42, 0xfd79070b, 0x94fe43d0, 0xb3837f99, - 0xb7862663, 0x90fb1a2a, 0xf97c5ef1, 0xde0162b8, - 0xb4097684, 0x93744acd, 0xfaf30e16, 0xdd8e325f, - 0xd98b6ba5, 0xfef657ec, 0x97711337, 0xb00c2f7e, - 0x6e0d4dc6, 0x4970718f, 0x20f73554, 0x078a091d, - 0x038f50e7, 0x24f26cae, 0x4d752875, 0x6a08143c, - 0x9965000d, 0xbe183c44, 0xd79f789f, 0xf0e244d6, - 0xf4e71d2c, 0xd39a2165, 0xba1d65be, 0x9d6059f7, - 0x43613b4f, 0x641c0706, 0x0d9b43dd, 0x2ae67f94, - 0x2ee3266e, 0x099e1a27, 0x60195efc, 0x476462b5, - 0x2d6c7689, 0x0a114ac0, 0x63960e1b, 0x44eb3252, - 0x40ee6ba8, 0x679357e1, 0x0e14133a, 0x29692f73, - 0xf7684dcb, 0xd0157182, 0xb9923559, 0x9eef0910, - 0x9aea50ea, 0xbd976ca3, 0xd4102878, 0xf36d1431, - 0x32cb001a, 0x15b63c53, 0x7c317888, 0x5b4c44c1, - 0x5f491d3b, 0x78342172, 0x11b365a9, 0x36ce59e0, - 0xe8cf3b58, 0xcfb20711, 0xa63543ca, 0x81487f83, - 0x854d2679, 0xa2301a30, 0xcbb75eeb, 0xecca62a2, - 0x86c2769e, 0xa1bf4ad7, 0xc8380e0c, 0xef453245, - 0xeb406bbf, 0xcc3d57f6, 0xa5ba132d, 0x82c72f64, - 0x5cc64ddc, 0x7bbb7195, 0x123c354e, 0x35410907, - 0x314450fd, 0x16396cb4, 0x7fbe286f, 0x58c31426, - 0xabae0017, 0x8cd33c5e, 0xe5547885, 0xc22944cc, - 0xc62c1d36, 0xe151217f, 0x88d665a4, 0xafab59ed, - 0x71aa3b55, 0x56d7071c, 0x3f5043c7, 0x182d7f8e, - 0x1c282674, 0x3b551a3d, 0x52d25ee6, 0x75af62af, - 0x1fa77693, 0x38da4ada, 0x515d0e01, 0x76203248, - 0x72256bb2, 0x555857fb, 0x3cdf1320, 0x1ba22f69, - 0xc5a34dd1, 0xe2de7198, 0x8b593543, 0xac24090a, - 0xa82150f0, 0x8f5c6cb9, 0xe6db2862, 0xc1a6142b, - 0x64960134, 0x43eb3d7d, 0x2a6c79a6, 0x0d1145ef, - 0x09141c15, 0x2e69205c, 0x47ee6487, 0x609358ce, - 0xbe923a76, 0x99ef063f, 0xf06842e4, 0xd7157ead, - 0xd3102757, 0xf46d1b1e, 0x9dea5fc5, 0xba97638c, - 0xd09f77b0, 0xf7e24bf9, 0x9e650f22, 0xb918336b, - 0xbd1d6a91, 0x9a6056d8, 0xf3e71203, 0xd49a2e4a, - 0x0a9b4cf2, 0x2de670bb, 0x44613460, 0x631c0829, - 0x671951d3, 0x40646d9a, 0x29e32941, 0x0e9e1508, - 0xfdf30139, 0xda8e3d70, 0xb30979ab, 0x947445e2, - 0x90711c18, 0xb70c2051, 0xde8b648a, 0xf9f658c3, - 0x27f73a7b, 0x008a0632, 0x690d42e9, 0x4e707ea0, - 0x4a75275a, 0x6d081b13, 0x048f5fc8, 0x23f26381, - 0x49fa77bd, 0x6e874bf4, 0x07000f2f, 0x207d3366, - 0x24786a9c, 0x030556d5, 0x6a82120e, 0x4dff2e47, - 0x93fe4cff, 0xb48370b6, 0xdd04346d, 0xfa790824, - 0xfe7c51de, 0xd9016d97, 0xb086294c, 0x97fb1505, - 0x565d012e, 0x71203d67, 0x18a779bc, 0x3fda45f5, - 0x3bdf1c0f, 0x1ca22046, 0x7525649d, 0x525858d4, - 0x8c593a6c, 0xab240625, 0xc2a342fe, 0xe5de7eb7, - 0xe1db274d, 0xc6a61b04, 0xaf215fdf, 0x885c6396, - 0xe25477aa, 0xc5294be3, 0xacae0f38, 0x8bd33371, - 0x8fd66a8b, 0xa8ab56c2, 0xc12c1219, 0xe6512e50, - 0x38504ce8, 0x1f2d70a1, 0x76aa347a, 0x51d70833, - 0x55d251c9, 0x72af6d80, 0x1b28295b, 0x3c551512, - 0xcf380123, 0xe8453d6a, 0x81c279b1, 0xa6bf45f8, - 0xa2ba1c02, 0x85c7204b, 0xec406490, 0xcb3d58d9, - 0x153c3a61, 0x32410628, 0x5bc642f3, 0x7cbb7eba, - 0x78be2740, 0x5fc31b09, 0x36445fd2, 0x1139639b, - 0x7b3177a7, 0x5c4c4bee, 0x35cb0f35, 0x12b6337c, - 0x16b36a86, 0x31ce56cf, 0x58491214, 0x7f342e5d, - 0xa1354ce5, 0x864870ac, 0xefcf3477, 0xc8b2083e, - 0xccb751c4, 0xebca6d8d, 0x824d2956, 0xa530151f - } -#else - { - 0x00000000, 0xf26b8303, 0xe13b70f7, 0x1350f3f4, - 0xc79a971f, 0x35f1141c, 0x26a1e7e8, 0xd4ca64eb, - 0x8ad958cf, 0x78b2dbcc, 0x6be22838, 0x9989ab3b, - 0x4d43cfd0, 0xbf284cd3, 0xac78bf27, 0x5e133c24, - 0x105ec76f, 0xe235446c, 0xf165b798, 0x030e349b, - 0xd7c45070, 0x25afd373, 0x36ff2087, 0xc494a384, - 0x9a879fa0, 0x68ec1ca3, 0x7bbcef57, 0x89d76c54, - 0x5d1d08bf, 0xaf768bbc, 0xbc267848, 0x4e4dfb4b, - 0x20bd8ede, 0xd2d60ddd, 0xc186fe29, 0x33ed7d2a, - 0xe72719c1, 0x154c9ac2, 0x061c6936, 0xf477ea35, - 0xaa64d611, 0x580f5512, 0x4b5fa6e6, 0xb93425e5, - 0x6dfe410e, 0x9f95c20d, 0x8cc531f9, 0x7eaeb2fa, - 0x30e349b1, 0xc288cab2, 0xd1d83946, 0x23b3ba45, - 0xf779deae, 0x05125dad, 0x1642ae59, 0xe4292d5a, - 0xba3a117e, 0x4851927d, 0x5b016189, 0xa96ae28a, - 0x7da08661, 0x8fcb0562, 0x9c9bf696, 0x6ef07595, - 0x417b1dbc, 0xb3109ebf, 0xa0406d4b, 0x522bee48, - 0x86e18aa3, 0x748a09a0, 0x67dafa54, 0x95b17957, - 0xcba24573, 0x39c9c670, 0x2a993584, 0xd8f2b687, - 0x0c38d26c, 0xfe53516f, 0xed03a29b, 0x1f682198, - 0x5125dad3, 0xa34e59d0, 0xb01eaa24, 0x42752927, - 0x96bf4dcc, 0x64d4cecf, 0x77843d3b, 0x85efbe38, - 0xdbfc821c, 0x2997011f, 0x3ac7f2eb, 0xc8ac71e8, - 0x1c661503, 0xee0d9600, 0xfd5d65f4, 0x0f36e6f7, - 0x61c69362, 0x93ad1061, 0x80fde395, 0x72966096, - 0xa65c047d, 0x5437877e, 0x4767748a, 0xb50cf789, - 0xeb1fcbad, 0x197448ae, 0x0a24bb5a, 0xf84f3859, - 0x2c855cb2, 0xdeeedfb1, 0xcdbe2c45, 0x3fd5af46, - 0x7198540d, 0x83f3d70e, 0x90a324fa, 0x62c8a7f9, - 0xb602c312, 0x44694011, 0x5739b3e5, 0xa55230e6, - 0xfb410cc2, 0x092a8fc1, 0x1a7a7c35, 0xe811ff36, - 0x3cdb9bdd, 0xceb018de, 0xdde0eb2a, 0x2f8b6829, - 0x82f63b78, 0x709db87b, 0x63cd4b8f, 0x91a6c88c, - 0x456cac67, 0xb7072f64, 0xa457dc90, 0x563c5f93, - 0x082f63b7, 0xfa44e0b4, 0xe9141340, 0x1b7f9043, - 0xcfb5f4a8, 0x3dde77ab, 0x2e8e845f, 0xdce5075c, - 0x92a8fc17, 0x60c37f14, 0x73938ce0, 0x81f80fe3, - 0x55326b08, 0xa759e80b, 0xb4091bff, 0x466298fc, - 0x1871a4d8, 0xea1a27db, 0xf94ad42f, 0x0b21572c, - 0xdfeb33c7, 0x2d80b0c4, 0x3ed04330, 0xccbbc033, - 0xa24bb5a6, 0x502036a5, 0x4370c551, 0xb11b4652, - 0x65d122b9, 0x97baa1ba, 0x84ea524e, 0x7681d14d, - 0x2892ed69, 0xdaf96e6a, 0xc9a99d9e, 0x3bc21e9d, - 0xef087a76, 0x1d63f975, 0x0e330a81, 0xfc588982, - 0xb21572c9, 0x407ef1ca, 0x532e023e, 0xa145813d, - 0x758fe5d6, 0x87e466d5, 0x94b49521, 0x66df1622, - 0x38cc2a06, 0xcaa7a905, 0xd9f75af1, 0x2b9cd9f2, - 0xff56bd19, 0x0d3d3e1a, 0x1e6dcdee, 0xec064eed, - 0xc38d26c4, 0x31e6a5c7, 0x22b65633, 0xd0ddd530, - 0x0417b1db, 0xf67c32d8, 0xe52cc12c, 0x1747422f, - 0x49547e0b, 0xbb3ffd08, 0xa86f0efc, 0x5a048dff, - 0x8ecee914, 0x7ca56a17, 0x6ff599e3, 0x9d9e1ae0, - 0xd3d3e1ab, 0x21b862a8, 0x32e8915c, 0xc083125f, - 0x144976b4, 0xe622f5b7, 0xf5720643, 0x07198540, - 0x590ab964, 0xab613a67, 0xb831c993, 0x4a5a4a90, - 0x9e902e7b, 0x6cfbad78, 0x7fab5e8c, 0x8dc0dd8f, - 0xe330a81a, 0x115b2b19, 0x020bd8ed, 0xf0605bee, - 0x24aa3f05, 0xd6c1bc06, 0xc5914ff2, 0x37faccf1, - 0x69e9f0d5, 0x9b8273d6, 0x88d28022, 0x7ab90321, - 0xae7367ca, 0x5c18e4c9, 0x4f48173d, 0xbd23943e, - 0xf36e6f75, 0x0105ec76, 0x12551f82, 0xe03e9c81, - 0x34f4f86a, 0xc69f7b69, 0xd5cf889d, 0x27a40b9e, - 0x79b737ba, 0x8bdcb4b9, 0x988c474d, 0x6ae7c44e, - 0xbe2da0a5, 0x4c4623a6, 0x5f16d052, 0xad7d5351 - },{ - 0x00000000, 0x13a29877, 0x274530ee, 0x34e7a899, - 0x4e8a61dc, 0x5d28f9ab, 0x69cf5132, 0x7a6dc945, - 0x9d14c3b8, 0x8eb65bcf, 0xba51f356, 0xa9f36b21, - 0xd39ea264, 0xc03c3a13, 0xf4db928a, 0xe7790afd, - 0x3fc5f181, 0x2c6769f6, 0x1880c16f, 0x0b225918, - 0x714f905d, 0x62ed082a, 0x560aa0b3, 0x45a838c4, - 0xa2d13239, 0xb173aa4e, 0x859402d7, 0x96369aa0, - 0xec5b53e5, 0xfff9cb92, 0xcb1e630b, 0xd8bcfb7c, - 0x7f8be302, 0x6c297b75, 0x58ced3ec, 0x4b6c4b9b, - 0x310182de, 0x22a31aa9, 0x1644b230, 0x05e62a47, - 0xe29f20ba, 0xf13db8cd, 0xc5da1054, 0xd6788823, - 0xac154166, 0xbfb7d911, 0x8b507188, 0x98f2e9ff, - 0x404e1283, 0x53ec8af4, 0x670b226d, 0x74a9ba1a, - 0x0ec4735f, 0x1d66eb28, 0x298143b1, 0x3a23dbc6, - 0xdd5ad13b, 0xcef8494c, 0xfa1fe1d5, 0xe9bd79a2, - 0x93d0b0e7, 0x80722890, 0xb4958009, 0xa737187e, - 0xff17c604, 0xecb55e73, 0xd852f6ea, 0xcbf06e9d, - 0xb19da7d8, 0xa23f3faf, 0x96d89736, 0x857a0f41, - 0x620305bc, 0x71a19dcb, 0x45463552, 0x56e4ad25, - 0x2c896460, 0x3f2bfc17, 0x0bcc548e, 0x186eccf9, - 0xc0d23785, 0xd370aff2, 0xe797076b, 0xf4359f1c, - 0x8e585659, 0x9dface2e, 0xa91d66b7, 0xbabffec0, - 0x5dc6f43d, 0x4e646c4a, 0x7a83c4d3, 0x69215ca4, - 0x134c95e1, 0x00ee0d96, 0x3409a50f, 0x27ab3d78, - 0x809c2506, 0x933ebd71, 0xa7d915e8, 0xb47b8d9f, - 0xce1644da, 0xddb4dcad, 0xe9537434, 0xfaf1ec43, - 0x1d88e6be, 0x0e2a7ec9, 0x3acdd650, 0x296f4e27, - 0x53028762, 0x40a01f15, 0x7447b78c, 0x67e52ffb, - 0xbf59d487, 0xacfb4cf0, 0x981ce469, 0x8bbe7c1e, - 0xf1d3b55b, 0xe2712d2c, 0xd69685b5, 0xc5341dc2, - 0x224d173f, 0x31ef8f48, 0x050827d1, 0x16aabfa6, - 0x6cc776e3, 0x7f65ee94, 0x4b82460d, 0x5820de7a, - 0xfbc3faf9, 0xe861628e, 0xdc86ca17, 0xcf245260, - 0xb5499b25, 0xa6eb0352, 0x920cabcb, 0x81ae33bc, - 0x66d73941, 0x7575a136, 0x419209af, 0x523091d8, - 0x285d589d, 0x3bffc0ea, 0x0f186873, 0x1cbaf004, - 0xc4060b78, 0xd7a4930f, 0xe3433b96, 0xf0e1a3e1, - 0x8a8c6aa4, 0x992ef2d3, 0xadc95a4a, 0xbe6bc23d, - 0x5912c8c0, 0x4ab050b7, 0x7e57f82e, 0x6df56059, - 0x1798a91c, 0x043a316b, 0x30dd99f2, 0x237f0185, - 0x844819fb, 0x97ea818c, 0xa30d2915, 0xb0afb162, - 0xcac27827, 0xd960e050, 0xed8748c9, 0xfe25d0be, - 0x195cda43, 0x0afe4234, 0x3e19eaad, 0x2dbb72da, - 0x57d6bb9f, 0x447423e8, 0x70938b71, 0x63311306, - 0xbb8de87a, 0xa82f700d, 0x9cc8d894, 0x8f6a40e3, - 0xf50789a6, 0xe6a511d1, 0xd242b948, 0xc1e0213f, - 0x26992bc2, 0x353bb3b5, 0x01dc1b2c, 0x127e835b, - 0x68134a1e, 0x7bb1d269, 0x4f567af0, 0x5cf4e287, - 0x04d43cfd, 0x1776a48a, 0x23910c13, 0x30339464, - 0x4a5e5d21, 0x59fcc556, 0x6d1b6dcf, 0x7eb9f5b8, - 0x99c0ff45, 0x8a626732, 0xbe85cfab, 0xad2757dc, - 0xd74a9e99, 0xc4e806ee, 0xf00fae77, 0xe3ad3600, - 0x3b11cd7c, 0x28b3550b, 0x1c54fd92, 0x0ff665e5, - 0x759baca0, 0x663934d7, 0x52de9c4e, 0x417c0439, - 0xa6050ec4, 0xb5a796b3, 0x81403e2a, 0x92e2a65d, - 0xe88f6f18, 0xfb2df76f, 0xcfca5ff6, 0xdc68c781, - 0x7b5fdfff, 0x68fd4788, 0x5c1aef11, 0x4fb87766, - 0x35d5be23, 0x26772654, 0x12908ecd, 0x013216ba, - 0xe64b1c47, 0xf5e98430, 0xc10e2ca9, 0xd2acb4de, - 0xa8c17d9b, 0xbb63e5ec, 0x8f844d75, 0x9c26d502, - 0x449a2e7e, 0x5738b609, 0x63df1e90, 0x707d86e7, - 0x0a104fa2, 0x19b2d7d5, 0x2d557f4c, 0x3ef7e73b, - 0xd98eedc6, 0xca2c75b1, 0xfecbdd28, 0xed69455f, - 0x97048c1a, 0x84a6146d, 0xb041bcf4, 0xa3e32483 - },{ - 0x00000000, 0xa541927e, 0x4f6f520d, 0xea2ec073, - 0x9edea41a, 0x3b9f3664, 0xd1b1f617, 0x74f06469, - 0x38513ec5, 0x9d10acbb, 0x773e6cc8, 0xd27ffeb6, - 0xa68f9adf, 0x03ce08a1, 0xe9e0c8d2, 0x4ca15aac, - 0x70a27d8a, 0xd5e3eff4, 0x3fcd2f87, 0x9a8cbdf9, - 0xee7cd990, 0x4b3d4bee, 0xa1138b9d, 0x045219e3, - 0x48f3434f, 0xedb2d131, 0x079c1142, 0xa2dd833c, - 0xd62de755, 0x736c752b, 0x9942b558, 0x3c032726, - 0xe144fb14, 0x4405696a, 0xae2ba919, 0x0b6a3b67, - 0x7f9a5f0e, 0xdadbcd70, 0x30f50d03, 0x95b49f7d, - 0xd915c5d1, 0x7c5457af, 0x967a97dc, 0x333b05a2, - 0x47cb61cb, 0xe28af3b5, 0x08a433c6, 0xade5a1b8, - 0x91e6869e, 0x34a714e0, 0xde89d493, 0x7bc846ed, - 0x0f382284, 0xaa79b0fa, 0x40577089, 0xe516e2f7, - 0xa9b7b85b, 0x0cf62a25, 0xe6d8ea56, 0x43997828, - 0x37691c41, 0x92288e3f, 0x78064e4c, 0xdd47dc32, - 0xc76580d9, 0x622412a7, 0x880ad2d4, 0x2d4b40aa, - 0x59bb24c3, 0xfcfab6bd, 0x16d476ce, 0xb395e4b0, - 0xff34be1c, 0x5a752c62, 0xb05bec11, 0x151a7e6f, - 0x61ea1a06, 0xc4ab8878, 0x2e85480b, 0x8bc4da75, - 0xb7c7fd53, 0x12866f2d, 0xf8a8af5e, 0x5de93d20, - 0x29195949, 0x8c58cb37, 0x66760b44, 0xc337993a, - 0x8f96c396, 0x2ad751e8, 0xc0f9919b, 0x65b803e5, - 0x1148678c, 0xb409f5f2, 0x5e273581, 0xfb66a7ff, - 0x26217bcd, 0x8360e9b3, 0x694e29c0, 0xcc0fbbbe, - 0xb8ffdfd7, 0x1dbe4da9, 0xf7908dda, 0x52d11fa4, - 0x1e704508, 0xbb31d776, 0x511f1705, 0xf45e857b, - 0x80aee112, 0x25ef736c, 0xcfc1b31f, 0x6a802161, - 0x56830647, 0xf3c29439, 0x19ec544a, 0xbcadc634, - 0xc85da25d, 0x6d1c3023, 0x8732f050, 0x2273622e, - 0x6ed23882, 0xcb93aafc, 0x21bd6a8f, 0x84fcf8f1, - 0xf00c9c98, 0x554d0ee6, 0xbf63ce95, 0x1a225ceb, - 0x8b277743, 0x2e66e53d, 0xc448254e, 0x6109b730, - 0x15f9d359, 0xb0b84127, 0x5a968154, 0xffd7132a, - 0xb3764986, 0x1637dbf8, 0xfc191b8b, 0x595889f5, - 0x2da8ed9c, 0x88e97fe2, 0x62c7bf91, 0xc7862def, - 0xfb850ac9, 0x5ec498b7, 0xb4ea58c4, 0x11abcaba, - 0x655baed3, 0xc01a3cad, 0x2a34fcde, 0x8f756ea0, - 0xc3d4340c, 0x6695a672, 0x8cbb6601, 0x29faf47f, - 0x5d0a9016, 0xf84b0268, 0x1265c21b, 0xb7245065, - 0x6a638c57, 0xcf221e29, 0x250cde5a, 0x804d4c24, - 0xf4bd284d, 0x51fcba33, 0xbbd27a40, 0x1e93e83e, - 0x5232b292, 0xf77320ec, 0x1d5de09f, 0xb81c72e1, - 0xccec1688, 0x69ad84f6, 0x83834485, 0x26c2d6fb, - 0x1ac1f1dd, 0xbf8063a3, 0x55aea3d0, 0xf0ef31ae, - 0x841f55c7, 0x215ec7b9, 0xcb7007ca, 0x6e3195b4, - 0x2290cf18, 0x87d15d66, 0x6dff9d15, 0xc8be0f6b, - 0xbc4e6b02, 0x190ff97c, 0xf321390f, 0x5660ab71, - 0x4c42f79a, 0xe90365e4, 0x032da597, 0xa66c37e9, - 0xd29c5380, 0x77ddc1fe, 0x9df3018d, 0x38b293f3, - 0x7413c95f, 0xd1525b21, 0x3b7c9b52, 0x9e3d092c, - 0xeacd6d45, 0x4f8cff3b, 0xa5a23f48, 0x00e3ad36, - 0x3ce08a10, 0x99a1186e, 0x738fd81d, 0xd6ce4a63, - 0xa23e2e0a, 0x077fbc74, 0xed517c07, 0x4810ee79, - 0x04b1b4d5, 0xa1f026ab, 0x4bdee6d8, 0xee9f74a6, - 0x9a6f10cf, 0x3f2e82b1, 0xd50042c2, 0x7041d0bc, - 0xad060c8e, 0x08479ef0, 0xe2695e83, 0x4728ccfd, - 0x33d8a894, 0x96993aea, 0x7cb7fa99, 0xd9f668e7, - 0x9557324b, 0x3016a035, 0xda386046, 0x7f79f238, - 0x0b899651, 0xaec8042f, 0x44e6c45c, 0xe1a75622, - 0xdda47104, 0x78e5e37a, 0x92cb2309, 0x378ab177, - 0x437ad51e, 0xe63b4760, 0x0c158713, 0xa954156d, - 0xe5f54fc1, 0x40b4ddbf, 0xaa9a1dcc, 0x0fdb8fb2, - 0x7b2bebdb, 0xde6a79a5, 0x3444b9d6, 0x91052ba8 - },{ - 0x00000000, 0xdd45aab8, 0xbf672381, 0x62228939, - 0x7b2231f3, 0xa6679b4b, 0xc4451272, 0x1900b8ca, - 0xf64463e6, 0x2b01c95e, 0x49234067, 0x9466eadf, - 0x8d665215, 0x5023f8ad, 0x32017194, 0xef44db2c, - 0xe964b13d, 0x34211b85, 0x560392bc, 0x8b463804, - 0x924680ce, 0x4f032a76, 0x2d21a34f, 0xf06409f7, - 0x1f20d2db, 0xc2657863, 0xa047f15a, 0x7d025be2, - 0x6402e328, 0xb9474990, 0xdb65c0a9, 0x06206a11, - 0xd725148b, 0x0a60be33, 0x6842370a, 0xb5079db2, - 0xac072578, 0x71428fc0, 0x136006f9, 0xce25ac41, - 0x2161776d, 0xfc24ddd5, 0x9e0654ec, 0x4343fe54, - 0x5a43469e, 0x8706ec26, 0xe524651f, 0x3861cfa7, - 0x3e41a5b6, 0xe3040f0e, 0x81268637, 0x5c632c8f, - 0x45639445, 0x98263efd, 0xfa04b7c4, 0x27411d7c, - 0xc805c650, 0x15406ce8, 0x7762e5d1, 0xaa274f69, - 0xb327f7a3, 0x6e625d1b, 0x0c40d422, 0xd1057e9a, - 0xaba65fe7, 0x76e3f55f, 0x14c17c66, 0xc984d6de, - 0xd0846e14, 0x0dc1c4ac, 0x6fe34d95, 0xb2a6e72d, - 0x5de23c01, 0x80a796b9, 0xe2851f80, 0x3fc0b538, - 0x26c00df2, 0xfb85a74a, 0x99a72e73, 0x44e284cb, - 0x42c2eeda, 0x9f874462, 0xfda5cd5b, 0x20e067e3, - 0x39e0df29, 0xe4a57591, 0x8687fca8, 0x5bc25610, - 0xb4868d3c, 0x69c32784, 0x0be1aebd, 0xd6a40405, - 0xcfa4bccf, 0x12e11677, 0x70c39f4e, 0xad8635f6, - 0x7c834b6c, 0xa1c6e1d4, 0xc3e468ed, 0x1ea1c255, - 0x07a17a9f, 0xdae4d027, 0xb8c6591e, 0x6583f3a6, - 0x8ac7288a, 0x57828232, 0x35a00b0b, 0xe8e5a1b3, - 0xf1e51979, 0x2ca0b3c1, 0x4e823af8, 0x93c79040, - 0x95e7fa51, 0x48a250e9, 0x2a80d9d0, 0xf7c57368, - 0xeec5cba2, 0x3380611a, 0x51a2e823, 0x8ce7429b, - 0x63a399b7, 0xbee6330f, 0xdcc4ba36, 0x0181108e, - 0x1881a844, 0xc5c402fc, 0xa7e68bc5, 0x7aa3217d, - 0x52a0c93f, 0x8fe56387, 0xedc7eabe, 0x30824006, - 0x2982f8cc, 0xf4c75274, 0x96e5db4d, 0x4ba071f5, - 0xa4e4aad9, 0x79a10061, 0x1b838958, 0xc6c623e0, - 0xdfc69b2a, 0x02833192, 0x60a1b8ab, 0xbde41213, - 0xbbc47802, 0x6681d2ba, 0x04a35b83, 0xd9e6f13b, - 0xc0e649f1, 0x1da3e349, 0x7f816a70, 0xa2c4c0c8, - 0x4d801be4, 0x90c5b15c, 0xf2e73865, 0x2fa292dd, - 0x36a22a17, 0xebe780af, 0x89c50996, 0x5480a32e, - 0x8585ddb4, 0x58c0770c, 0x3ae2fe35, 0xe7a7548d, - 0xfea7ec47, 0x23e246ff, 0x41c0cfc6, 0x9c85657e, - 0x73c1be52, 0xae8414ea, 0xcca69dd3, 0x11e3376b, - 0x08e38fa1, 0xd5a62519, 0xb784ac20, 0x6ac10698, - 0x6ce16c89, 0xb1a4c631, 0xd3864f08, 0x0ec3e5b0, - 0x17c35d7a, 0xca86f7c2, 0xa8a47efb, 0x75e1d443, - 0x9aa50f6f, 0x47e0a5d7, 0x25c22cee, 0xf8878656, - 0xe1873e9c, 0x3cc29424, 0x5ee01d1d, 0x83a5b7a5, - 0xf90696d8, 0x24433c60, 0x4661b559, 0x9b241fe1, - 0x8224a72b, 0x5f610d93, 0x3d4384aa, 0xe0062e12, - 0x0f42f53e, 0xd2075f86, 0xb025d6bf, 0x6d607c07, - 0x7460c4cd, 0xa9256e75, 0xcb07e74c, 0x16424df4, - 0x106227e5, 0xcd278d5d, 0xaf050464, 0x7240aedc, - 0x6b401616, 0xb605bcae, 0xd4273597, 0x09629f2f, - 0xe6264403, 0x3b63eebb, 0x59416782, 0x8404cd3a, - 0x9d0475f0, 0x4041df48, 0x22635671, 0xff26fcc9, - 0x2e238253, 0xf36628eb, 0x9144a1d2, 0x4c010b6a, - 0x5501b3a0, 0x88441918, 0xea669021, 0x37233a99, - 0xd867e1b5, 0x05224b0d, 0x6700c234, 0xba45688c, - 0xa345d046, 0x7e007afe, 0x1c22f3c7, 0xc167597f, - 0xc747336e, 0x1a0299d6, 0x782010ef, 0xa565ba57, - 0xbc65029d, 0x6120a825, 0x0302211c, 0xde478ba4, - 0x31035088, 0xec46fa30, 0x8e647309, 0x5321d9b1, - 0x4a21617b, 0x9764cbc3, 0xf54642fa, 0x2803e842 - },{ - 0x00000000, 0x38116fac, 0x7022df58, 0x4833b0f4, - 0xe045beb0, 0xd854d11c, 0x906761e8, 0xa8760e44, - 0xc5670b91, 0xfd76643d, 0xb545d4c9, 0x8d54bb65, - 0x2522b521, 0x1d33da8d, 0x55006a79, 0x6d1105d5, - 0x8f2261d3, 0xb7330e7f, 0xff00be8b, 0xc711d127, - 0x6f67df63, 0x5776b0cf, 0x1f45003b, 0x27546f97, - 0x4a456a42, 0x725405ee, 0x3a67b51a, 0x0276dab6, - 0xaa00d4f2, 0x9211bb5e, 0xda220baa, 0xe2336406, - 0x1ba8b557, 0x23b9dafb, 0x6b8a6a0f, 0x539b05a3, - 0xfbed0be7, 0xc3fc644b, 0x8bcfd4bf, 0xb3debb13, - 0xdecfbec6, 0xe6ded16a, 0xaeed619e, 0x96fc0e32, - 0x3e8a0076, 0x069b6fda, 0x4ea8df2e, 0x76b9b082, - 0x948ad484, 0xac9bbb28, 0xe4a80bdc, 0xdcb96470, - 0x74cf6a34, 0x4cde0598, 0x04edb56c, 0x3cfcdac0, - 0x51eddf15, 0x69fcb0b9, 0x21cf004d, 0x19de6fe1, - 0xb1a861a5, 0x89b90e09, 0xc18abefd, 0xf99bd151, - 0x37516aae, 0x0f400502, 0x4773b5f6, 0x7f62da5a, - 0xd714d41e, 0xef05bbb2, 0xa7360b46, 0x9f2764ea, - 0xf236613f, 0xca270e93, 0x8214be67, 0xba05d1cb, - 0x1273df8f, 0x2a62b023, 0x625100d7, 0x5a406f7b, - 0xb8730b7d, 0x806264d1, 0xc851d425, 0xf040bb89, - 0x5836b5cd, 0x6027da61, 0x28146a95, 0x10050539, - 0x7d1400ec, 0x45056f40, 0x0d36dfb4, 0x3527b018, - 0x9d51be5c, 0xa540d1f0, 0xed736104, 0xd5620ea8, - 0x2cf9dff9, 0x14e8b055, 0x5cdb00a1, 0x64ca6f0d, - 0xccbc6149, 0xf4ad0ee5, 0xbc9ebe11, 0x848fd1bd, - 0xe99ed468, 0xd18fbbc4, 0x99bc0b30, 0xa1ad649c, - 0x09db6ad8, 0x31ca0574, 0x79f9b580, 0x41e8da2c, - 0xa3dbbe2a, 0x9bcad186, 0xd3f96172, 0xebe80ede, - 0x439e009a, 0x7b8f6f36, 0x33bcdfc2, 0x0badb06e, - 0x66bcb5bb, 0x5eadda17, 0x169e6ae3, 0x2e8f054f, - 0x86f90b0b, 0xbee864a7, 0xf6dbd453, 0xcecabbff, - 0x6ea2d55c, 0x56b3baf0, 0x1e800a04, 0x269165a8, - 0x8ee76bec, 0xb6f60440, 0xfec5b4b4, 0xc6d4db18, - 0xabc5decd, 0x93d4b161, 0xdbe70195, 0xe3f66e39, - 0x4b80607d, 0x73910fd1, 0x3ba2bf25, 0x03b3d089, - 0xe180b48f, 0xd991db23, 0x91a26bd7, 0xa9b3047b, - 0x01c50a3f, 0x39d46593, 0x71e7d567, 0x49f6bacb, - 0x24e7bf1e, 0x1cf6d0b2, 0x54c56046, 0x6cd40fea, - 0xc4a201ae, 0xfcb36e02, 0xb480def6, 0x8c91b15a, - 0x750a600b, 0x4d1b0fa7, 0x0528bf53, 0x3d39d0ff, - 0x954fdebb, 0xad5eb117, 0xe56d01e3, 0xdd7c6e4f, - 0xb06d6b9a, 0x887c0436, 0xc04fb4c2, 0xf85edb6e, - 0x5028d52a, 0x6839ba86, 0x200a0a72, 0x181b65de, - 0xfa2801d8, 0xc2396e74, 0x8a0ade80, 0xb21bb12c, - 0x1a6dbf68, 0x227cd0c4, 0x6a4f6030, 0x525e0f9c, - 0x3f4f0a49, 0x075e65e5, 0x4f6dd511, 0x777cbabd, - 0xdf0ab4f9, 0xe71bdb55, 0xaf286ba1, 0x9739040d, - 0x59f3bff2, 0x61e2d05e, 0x29d160aa, 0x11c00f06, - 0xb9b60142, 0x81a76eee, 0xc994de1a, 0xf185b1b6, - 0x9c94b463, 0xa485dbcf, 0xecb66b3b, 0xd4a70497, - 0x7cd10ad3, 0x44c0657f, 0x0cf3d58b, 0x34e2ba27, - 0xd6d1de21, 0xeec0b18d, 0xa6f30179, 0x9ee26ed5, - 0x36946091, 0x0e850f3d, 0x46b6bfc9, 0x7ea7d065, - 0x13b6d5b0, 0x2ba7ba1c, 0x63940ae8, 0x5b856544, - 0xf3f36b00, 0xcbe204ac, 0x83d1b458, 0xbbc0dbf4, - 0x425b0aa5, 0x7a4a6509, 0x3279d5fd, 0x0a68ba51, - 0xa21eb415, 0x9a0fdbb9, 0xd23c6b4d, 0xea2d04e1, - 0x873c0134, 0xbf2d6e98, 0xf71ede6c, 0xcf0fb1c0, - 0x6779bf84, 0x5f68d028, 0x175b60dc, 0x2f4a0f70, - 0xcd796b76, 0xf56804da, 0xbd5bb42e, 0x854adb82, - 0x2d3cd5c6, 0x152dba6a, 0x5d1e0a9e, 0x650f6532, - 0x081e60e7, 0x300f0f4b, 0x783cbfbf, 0x402dd013, - 0xe85bde57, 0xd04ab1fb, 0x9879010f, 0xa0686ea3 - },{ - 0x00000000, 0xef306b19, 0xdb8ca0c3, 0x34bccbda, - 0xb2f53777, 0x5dc55c6e, 0x697997b4, 0x8649fcad, - 0x6006181f, 0x8f367306, 0xbb8ab8dc, 0x54bad3c5, - 0xd2f32f68, 0x3dc34471, 0x097f8fab, 0xe64fe4b2, - 0xc00c303e, 0x2f3c5b27, 0x1b8090fd, 0xf4b0fbe4, - 0x72f90749, 0x9dc96c50, 0xa975a78a, 0x4645cc93, - 0xa00a2821, 0x4f3a4338, 0x7b8688e2, 0x94b6e3fb, - 0x12ff1f56, 0xfdcf744f, 0xc973bf95, 0x2643d48c, - 0x85f4168d, 0x6ac47d94, 0x5e78b64e, 0xb148dd57, - 0x370121fa, 0xd8314ae3, 0xec8d8139, 0x03bdea20, - 0xe5f20e92, 0x0ac2658b, 0x3e7eae51, 0xd14ec548, - 0x570739e5, 0xb83752fc, 0x8c8b9926, 0x63bbf23f, - 0x45f826b3, 0xaac84daa, 0x9e748670, 0x7144ed69, - 0xf70d11c4, 0x183d7add, 0x2c81b107, 0xc3b1da1e, - 0x25fe3eac, 0xcace55b5, 0xfe729e6f, 0x1142f576, - 0x970b09db, 0x783b62c2, 0x4c87a918, 0xa3b7c201, - 0x0e045beb, 0xe13430f2, 0xd588fb28, 0x3ab89031, - 0xbcf16c9c, 0x53c10785, 0x677dcc5f, 0x884da746, - 0x6e0243f4, 0x813228ed, 0xb58ee337, 0x5abe882e, - 0xdcf77483, 0x33c71f9a, 0x077bd440, 0xe84bbf59, - 0xce086bd5, 0x213800cc, 0x1584cb16, 0xfab4a00f, - 0x7cfd5ca2, 0x93cd37bb, 0xa771fc61, 0x48419778, - 0xae0e73ca, 0x413e18d3, 0x7582d309, 0x9ab2b810, - 0x1cfb44bd, 0xf3cb2fa4, 0xc777e47e, 0x28478f67, - 0x8bf04d66, 0x64c0267f, 0x507ceda5, 0xbf4c86bc, - 0x39057a11, 0xd6351108, 0xe289dad2, 0x0db9b1cb, - 0xebf65579, 0x04c63e60, 0x307af5ba, 0xdf4a9ea3, - 0x5903620e, 0xb6330917, 0x828fc2cd, 0x6dbfa9d4, - 0x4bfc7d58, 0xa4cc1641, 0x9070dd9b, 0x7f40b682, - 0xf9094a2f, 0x16392136, 0x2285eaec, 0xcdb581f5, - 0x2bfa6547, 0xc4ca0e5e, 0xf076c584, 0x1f46ae9d, - 0x990f5230, 0x763f3929, 0x4283f2f3, 0xadb399ea, - 0x1c08b7d6, 0xf338dccf, 0xc7841715, 0x28b47c0c, - 0xaefd80a1, 0x41cdebb8, 0x75712062, 0x9a414b7b, - 0x7c0eafc9, 0x933ec4d0, 0xa7820f0a, 0x48b26413, - 0xcefb98be, 0x21cbf3a7, 0x1577387d, 0xfa475364, - 0xdc0487e8, 0x3334ecf1, 0x0788272b, 0xe8b84c32, - 0x6ef1b09f, 0x81c1db86, 0xb57d105c, 0x5a4d7b45, - 0xbc029ff7, 0x5332f4ee, 0x678e3f34, 0x88be542d, - 0x0ef7a880, 0xe1c7c399, 0xd57b0843, 0x3a4b635a, - 0x99fca15b, 0x76ccca42, 0x42700198, 0xad406a81, - 0x2b09962c, 0xc439fd35, 0xf08536ef, 0x1fb55df6, - 0xf9fab944, 0x16cad25d, 0x22761987, 0xcd46729e, - 0x4b0f8e33, 0xa43fe52a, 0x90832ef0, 0x7fb345e9, - 0x59f09165, 0xb6c0fa7c, 0x827c31a6, 0x6d4c5abf, - 0xeb05a612, 0x0435cd0b, 0x308906d1, 0xdfb96dc8, - 0x39f6897a, 0xd6c6e263, 0xe27a29b9, 0x0d4a42a0, - 0x8b03be0d, 0x6433d514, 0x508f1ece, 0xbfbf75d7, - 0x120cec3d, 0xfd3c8724, 0xc9804cfe, 0x26b027e7, - 0xa0f9db4a, 0x4fc9b053, 0x7b757b89, 0x94451090, - 0x720af422, 0x9d3a9f3b, 0xa98654e1, 0x46b63ff8, - 0xc0ffc355, 0x2fcfa84c, 0x1b736396, 0xf443088f, - 0xd200dc03, 0x3d30b71a, 0x098c7cc0, 0xe6bc17d9, - 0x60f5eb74, 0x8fc5806d, 0xbb794bb7, 0x544920ae, - 0xb206c41c, 0x5d36af05, 0x698a64df, 0x86ba0fc6, - 0x00f3f36b, 0xefc39872, 0xdb7f53a8, 0x344f38b1, - 0x97f8fab0, 0x78c891a9, 0x4c745a73, 0xa344316a, - 0x250dcdc7, 0xca3da6de, 0xfe816d04, 0x11b1061d, - 0xf7fee2af, 0x18ce89b6, 0x2c72426c, 0xc3422975, - 0x450bd5d8, 0xaa3bbec1, 0x9e87751b, 0x71b71e02, - 0x57f4ca8e, 0xb8c4a197, 0x8c786a4d, 0x63480154, - 0xe501fdf9, 0x0a3196e0, 0x3e8d5d3a, 0xd1bd3623, - 0x37f2d291, 0xd8c2b988, 0xec7e7252, 0x034e194b, - 0x8507e5e6, 0x6a378eff, 0x5e8b4525, 0xb1bb2e3c - },{ - 0x00000000, 0x68032cc8, 0xd0065990, 0xb8057558, - 0xa5e0c5d1, 0xcde3e919, 0x75e69c41, 0x1de5b089, - 0x4e2dfd53, 0x262ed19b, 0x9e2ba4c3, 0xf628880b, - 0xebcd3882, 0x83ce144a, 0x3bcb6112, 0x53c84dda, - 0x9c5bfaa6, 0xf458d66e, 0x4c5da336, 0x245e8ffe, - 0x39bb3f77, 0x51b813bf, 0xe9bd66e7, 0x81be4a2f, - 0xd27607f5, 0xba752b3d, 0x02705e65, 0x6a7372ad, - 0x7796c224, 0x1f95eeec, 0xa7909bb4, 0xcf93b77c, - 0x3d5b83bd, 0x5558af75, 0xed5dda2d, 0x855ef6e5, - 0x98bb466c, 0xf0b86aa4, 0x48bd1ffc, 0x20be3334, - 0x73767eee, 0x1b755226, 0xa370277e, 0xcb730bb6, - 0xd696bb3f, 0xbe9597f7, 0x0690e2af, 0x6e93ce67, - 0xa100791b, 0xc90355d3, 0x7106208b, 0x19050c43, - 0x04e0bcca, 0x6ce39002, 0xd4e6e55a, 0xbce5c992, - 0xef2d8448, 0x872ea880, 0x3f2bddd8, 0x5728f110, - 0x4acd4199, 0x22ce6d51, 0x9acb1809, 0xf2c834c1, - 0x7ab7077a, 0x12b42bb2, 0xaab15eea, 0xc2b27222, - 0xdf57c2ab, 0xb754ee63, 0x0f519b3b, 0x6752b7f3, - 0x349afa29, 0x5c99d6e1, 0xe49ca3b9, 0x8c9f8f71, - 0x917a3ff8, 0xf9791330, 0x417c6668, 0x297f4aa0, - 0xe6ecfddc, 0x8eefd114, 0x36eaa44c, 0x5ee98884, - 0x430c380d, 0x2b0f14c5, 0x930a619d, 0xfb094d55, - 0xa8c1008f, 0xc0c22c47, 0x78c7591f, 0x10c475d7, - 0x0d21c55e, 0x6522e996, 0xdd279cce, 0xb524b006, - 0x47ec84c7, 0x2fefa80f, 0x97eadd57, 0xffe9f19f, - 0xe20c4116, 0x8a0f6dde, 0x320a1886, 0x5a09344e, - 0x09c17994, 0x61c2555c, 0xd9c72004, 0xb1c40ccc, - 0xac21bc45, 0xc422908d, 0x7c27e5d5, 0x1424c91d, - 0xdbb77e61, 0xb3b452a9, 0x0bb127f1, 0x63b20b39, - 0x7e57bbb0, 0x16549778, 0xae51e220, 0xc652cee8, - 0x959a8332, 0xfd99affa, 0x459cdaa2, 0x2d9ff66a, - 0x307a46e3, 0x58796a2b, 0xe07c1f73, 0x887f33bb, - 0xf56e0ef4, 0x9d6d223c, 0x25685764, 0x4d6b7bac, - 0x508ecb25, 0x388de7ed, 0x808892b5, 0xe88bbe7d, - 0xbb43f3a7, 0xd340df6f, 0x6b45aa37, 0x034686ff, - 0x1ea33676, 0x76a01abe, 0xcea56fe6, 0xa6a6432e, - 0x6935f452, 0x0136d89a, 0xb933adc2, 0xd130810a, - 0xccd53183, 0xa4d61d4b, 0x1cd36813, 0x74d044db, - 0x27180901, 0x4f1b25c9, 0xf71e5091, 0x9f1d7c59, - 0x82f8ccd0, 0xeafbe018, 0x52fe9540, 0x3afdb988, - 0xc8358d49, 0xa036a181, 0x1833d4d9, 0x7030f811, - 0x6dd54898, 0x05d66450, 0xbdd31108, 0xd5d03dc0, - 0x8618701a, 0xee1b5cd2, 0x561e298a, 0x3e1d0542, - 0x23f8b5cb, 0x4bfb9903, 0xf3feec5b, 0x9bfdc093, - 0x546e77ef, 0x3c6d5b27, 0x84682e7f, 0xec6b02b7, - 0xf18eb23e, 0x998d9ef6, 0x2188ebae, 0x498bc766, - 0x1a438abc, 0x7240a674, 0xca45d32c, 0xa246ffe4, - 0xbfa34f6d, 0xd7a063a5, 0x6fa516fd, 0x07a63a35, - 0x8fd9098e, 0xe7da2546, 0x5fdf501e, 0x37dc7cd6, - 0x2a39cc5f, 0x423ae097, 0xfa3f95cf, 0x923cb907, - 0xc1f4f4dd, 0xa9f7d815, 0x11f2ad4d, 0x79f18185, - 0x6414310c, 0x0c171dc4, 0xb412689c, 0xdc114454, - 0x1382f328, 0x7b81dfe0, 0xc384aab8, 0xab878670, - 0xb66236f9, 0xde611a31, 0x66646f69, 0x0e6743a1, - 0x5daf0e7b, 0x35ac22b3, 0x8da957eb, 0xe5aa7b23, - 0xf84fcbaa, 0x904ce762, 0x2849923a, 0x404abef2, - 0xb2828a33, 0xda81a6fb, 0x6284d3a3, 0x0a87ff6b, - 0x17624fe2, 0x7f61632a, 0xc7641672, 0xaf673aba, - 0xfcaf7760, 0x94ac5ba8, 0x2ca92ef0, 0x44aa0238, - 0x594fb2b1, 0x314c9e79, 0x8949eb21, 0xe14ac7e9, - 0x2ed97095, 0x46da5c5d, 0xfedf2905, 0x96dc05cd, - 0x8b39b544, 0xe33a998c, 0x5b3fecd4, 0x333cc01c, - 0x60f48dc6, 0x08f7a10e, 0xb0f2d456, 0xd8f1f89e, - 0xc5144817, 0xad1764df, 0x15121187, 0x7d113d4f - },{ - 0x00000000, 0x493c7d27, 0x9278fa4e, 0xdb448769, - 0x211d826d, 0x6821ff4a, 0xb3657823, 0xfa590504, - 0x423b04da, 0x0b0779fd, 0xd043fe94, 0x997f83b3, - 0x632686b7, 0x2a1afb90, 0xf15e7cf9, 0xb86201de, - 0x847609b4, 0xcd4a7493, 0x160ef3fa, 0x5f328edd, - 0xa56b8bd9, 0xec57f6fe, 0x37137197, 0x7e2f0cb0, - 0xc64d0d6e, 0x8f717049, 0x5435f720, 0x1d098a07, - 0xe7508f03, 0xae6cf224, 0x7528754d, 0x3c14086a, - 0x0d006599, 0x443c18be, 0x9f789fd7, 0xd644e2f0, - 0x2c1de7f4, 0x65219ad3, 0xbe651dba, 0xf759609d, - 0x4f3b6143, 0x06071c64, 0xdd439b0d, 0x947fe62a, - 0x6e26e32e, 0x271a9e09, 0xfc5e1960, 0xb5626447, - 0x89766c2d, 0xc04a110a, 0x1b0e9663, 0x5232eb44, - 0xa86bee40, 0xe1579367, 0x3a13140e, 0x732f6929, - 0xcb4d68f7, 0x827115d0, 0x593592b9, 0x1009ef9e, - 0xea50ea9a, 0xa36c97bd, 0x782810d4, 0x31146df3, - 0x1a00cb32, 0x533cb615, 0x8878317c, 0xc1444c5b, - 0x3b1d495f, 0x72213478, 0xa965b311, 0xe059ce36, - 0x583bcfe8, 0x1107b2cf, 0xca4335a6, 0x837f4881, - 0x79264d85, 0x301a30a2, 0xeb5eb7cb, 0xa262caec, - 0x9e76c286, 0xd74abfa1, 0x0c0e38c8, 0x453245ef, - 0xbf6b40eb, 0xf6573dcc, 0x2d13baa5, 0x642fc782, - 0xdc4dc65c, 0x9571bb7b, 0x4e353c12, 0x07094135, - 0xfd504431, 0xb46c3916, 0x6f28be7f, 0x2614c358, - 0x1700aeab, 0x5e3cd38c, 0x857854e5, 0xcc4429c2, - 0x361d2cc6, 0x7f2151e1, 0xa465d688, 0xed59abaf, - 0x553baa71, 0x1c07d756, 0xc743503f, 0x8e7f2d18, - 0x7426281c, 0x3d1a553b, 0xe65ed252, 0xaf62af75, - 0x9376a71f, 0xda4ada38, 0x010e5d51, 0x48322076, - 0xb26b2572, 0xfb575855, 0x2013df3c, 0x692fa21b, - 0xd14da3c5, 0x9871dee2, 0x4335598b, 0x0a0924ac, - 0xf05021a8, 0xb96c5c8f, 0x6228dbe6, 0x2b14a6c1, - 0x34019664, 0x7d3deb43, 0xa6796c2a, 0xef45110d, - 0x151c1409, 0x5c20692e, 0x8764ee47, 0xce589360, - 0x763a92be, 0x3f06ef99, 0xe44268f0, 0xad7e15d7, - 0x572710d3, 0x1e1b6df4, 0xc55fea9d, 0x8c6397ba, - 0xb0779fd0, 0xf94be2f7, 0x220f659e, 0x6b3318b9, - 0x916a1dbd, 0xd856609a, 0x0312e7f3, 0x4a2e9ad4, - 0xf24c9b0a, 0xbb70e62d, 0x60346144, 0x29081c63, - 0xd3511967, 0x9a6d6440, 0x4129e329, 0x08159e0e, - 0x3901f3fd, 0x703d8eda, 0xab7909b3, 0xe2457494, - 0x181c7190, 0x51200cb7, 0x8a648bde, 0xc358f6f9, - 0x7b3af727, 0x32068a00, 0xe9420d69, 0xa07e704e, - 0x5a27754a, 0x131b086d, 0xc85f8f04, 0x8163f223, - 0xbd77fa49, 0xf44b876e, 0x2f0f0007, 0x66337d20, - 0x9c6a7824, 0xd5560503, 0x0e12826a, 0x472eff4d, - 0xff4cfe93, 0xb67083b4, 0x6d3404dd, 0x240879fa, - 0xde517cfe, 0x976d01d9, 0x4c2986b0, 0x0515fb97, - 0x2e015d56, 0x673d2071, 0xbc79a718, 0xf545da3f, - 0x0f1cdf3b, 0x4620a21c, 0x9d642575, 0xd4585852, - 0x6c3a598c, 0x250624ab, 0xfe42a3c2, 0xb77edee5, - 0x4d27dbe1, 0x041ba6c6, 0xdf5f21af, 0x96635c88, - 0xaa7754e2, 0xe34b29c5, 0x380faeac, 0x7133d38b, - 0x8b6ad68f, 0xc256aba8, 0x19122cc1, 0x502e51e6, - 0xe84c5038, 0xa1702d1f, 0x7a34aa76, 0x3308d751, - 0xc951d255, 0x806daf72, 0x5b29281b, 0x1215553c, - 0x230138cf, 0x6a3d45e8, 0xb179c281, 0xf845bfa6, - 0x021cbaa2, 0x4b20c785, 0x906440ec, 0xd9583dcb, - 0x613a3c15, 0x28064132, 0xf342c65b, 0xba7ebb7c, - 0x4027be78, 0x091bc35f, 0xd25f4436, 0x9b633911, - 0xa777317b, 0xee4b4c5c, 0x350fcb35, 0x7c33b612, - 0x866ab316, 0xcf56ce31, 0x14124958, 0x5d2e347f, - 0xe54c35a1, 0xac704886, 0x7734cfef, 0x3e08b2c8, - 0xc451b7cc, 0x8d6dcaeb, 0x56294d82, 0x1f1530a5 - } -#endif -}; - -/* - * __wt_cksum_sw -- - * Return a checksum for a chunk of memory, computed in software. - */ -static uint32_t -__wt_cksum_sw(const void *chunk, size_t len) -{ - uint32_t crc, next; - size_t nqwords; - const uint8_t *p; - - crc = 0xffffffff; - - /* Checksum one byte at a time to the first 4B boundary. */ - for (p = chunk; - ((uintptr_t)p & (sizeof(uint32_t) - 1)) != 0 && - len > 0; ++p, --len) -#ifdef WORDS_BIGENDIAN - crc = g_crc_slicing[0][((crc >> 24) ^ *p) & 0xFF] ^ (crc << 8); -#else - crc = g_crc_slicing[0][(crc ^ *p) & 0xFF] ^ (crc >> 8); -#endif - - /* Checksum in 8B chunks. */ - for (nqwords = len / sizeof(uint64_t); nqwords; nqwords--) { - crc ^= *(uint32_t *)p; - p += sizeof(uint32_t); - next = *(uint32_t *)p; - p += sizeof(uint32_t); - crc = -#ifdef WORDS_BIGENDIAN - g_crc_slicing[4][(crc ) & 0xFF] ^ - g_crc_slicing[5][(crc >> 8) & 0xFF] ^ - g_crc_slicing[6][(crc >> 16) & 0xFF] ^ - g_crc_slicing[7][(crc >> 24)] ^ - g_crc_slicing[0][(next ) & 0xFF] ^ - g_crc_slicing[1][(next >> 8) & 0xFF] ^ - g_crc_slicing[2][(next >> 16) & 0xFF] ^ - g_crc_slicing[3][(next >> 24)]; -#else - g_crc_slicing[7][(crc ) & 0xFF] ^ - g_crc_slicing[6][(crc >> 8) & 0xFF] ^ - g_crc_slicing[5][(crc >> 16) & 0xFF] ^ - g_crc_slicing[4][(crc >> 24)] ^ - g_crc_slicing[3][(next ) & 0xFF] ^ - g_crc_slicing[2][(next >> 8) & 0xFF] ^ - g_crc_slicing[1][(next >> 16) & 0xFF] ^ - g_crc_slicing[0][(next >> 24)]; -#endif - } - - /* Checksum trailing bytes one byte at a time. */ -#ifdef WORDS_BIGENDIAN - for (len &= 0x7; len > 0; ++p, len--) - crc = g_crc_slicing[0][((crc >> 24) ^ *p) & 0xFF] ^ (crc << 8); - - /* Do final byte swap to produce a result identical to little endian */ - crc = - ((crc << 24) & 0xFF000000) | - ((crc << 8) & 0x00FF0000) | - ((crc >> 8) & 0x0000FF00) | - ((crc >> 24) & 0x000000FF); -#else - for (len &= 0x7; len > 0; ++p, len--) - crc = g_crc_slicing[0][(crc ^ *p) & 0xFF] ^ (crc >> 8); -#endif - return (~crc); -} - -#if (defined(__amd64) || defined(__x86_64)) -/* - * __wt_cksum_hw -- - * Return a checksum for a chunk of memory, computed in hardware - * using 8 byte steps. - */ -static uint32_t -__wt_cksum_hw(const void *chunk, size_t len) -{ - uint32_t crc; - size_t nqwords; - const uint8_t *p; - const uint64_t *p64; - - crc = 0xffffffff; - - /* Checksum one byte at a time to the first 4B boundary. */ - for (p = chunk; - ((uintptr_t)p & (sizeof(uint32_t) - 1)) != 0 && - len > 0; ++p, --len) { - __asm__ __volatile__( - ".byte 0xF2, 0x0F, 0x38, 0xF0, 0xF1" - : "=S" (crc) - : "0" (crc), "c" (*p)); - } - - p64 = (const uint64_t *)p; - /* Checksum in 8B chunks. */ - for (nqwords = len / sizeof(uint64_t); nqwords; nqwords--) { - __asm__ __volatile__ ( - ".byte 0xF2, 0x48, 0x0F, 0x38, 0xF1, 0xF1" - : "=S"(crc) - : "0"(crc), "c" (*p64)); - p64++; - } - - /* Checksum trailing bytes one byte at a time. */ - p = (const uint8_t *)p64; - for (len &= 0x7; len > 0; ++p, len--) { - __asm__ __volatile__( - ".byte 0xF2, 0x0F, 0x38, 0xF0, 0xF1" - : "=S" (crc) - : "0" (crc), "c" (*p)); - } - return (~crc); -} -#endif - -#if defined(_M_AMD64) -/* - * __wt_cksum_hw -- - * Return a checksum for a chunk of memory, computed in hardware - * using 8 byte steps. - */ -static uint32_t -__wt_cksum_hw(const void *chunk, size_t len) -{ - uint32_t crc; - size_t nqwords; - const uint8_t *p; - const uint64_t *p64; - - crc = 0xffffffff; - - /* Checksum one byte at a time to the first 4B boundary. */ - for (p = chunk; - ((uintptr_t)p & (sizeof(uint32_t) - 1)) != 0 && - len > 0; ++p, --len) { - crc = _mm_crc32_u8(crc, *p); - } - - p64 = (const uint64_t *)p; - /* Checksum in 8B chunks. */ - for (nqwords = len / sizeof(uint64_t); nqwords; nqwords--) { - crc = (uint32_t)_mm_crc32_u64(crc, *p64); - p64++; - } - - /* Checksum trailing bytes one byte at a time. */ - p = (const uint8_t *)p64; - for (len &= 0x7; len > 0; ++p, len--) { - crc = _mm_crc32_u8(crc, *p); - } - - return (~crc); -} -#endif - -#if defined(__powerpc64__) - -unsigned int crc32_vpmsum(unsigned int crc, const unsigned char *p, - unsigned long len); - -/* - * __wt_cksum_hw -- - * Return a checksum for a chunk of memory, computed in hardware - * using 8 byte steps. - */ -static uint32_t -__wt_cksum_hw(const void *chunk, size_t len) -{ - return crc32_vpmsum(0, chunk, len); -} -#endif - -/* - * __wt_cksum -- - * Return a checksum for a chunk of memory using the fastest method - * available. - */ -uint32_t -__wt_cksum(const void *chunk, size_t len) -{ - return (*__wt_cksum_func)(chunk, len); -} - -/* - * __wt_cksum_init -- - * Detect CRC hardware and set the checksum function. - */ -void -__wt_cksum_init(void) -{ -#define CPUID_ECX_HAS_SSE42 (1 << 20) - -#if (defined(__amd64) || defined(__x86_64)) - unsigned int eax, ebx, ecx, edx; - - __asm__ __volatile__ ( - "cpuid" - : "=a" (eax), "=b" (ebx), "=c" (ecx), "=d" (edx) - : "a" (1)); - - if (ecx & CPUID_ECX_HAS_SSE42) - __wt_cksum_func = __wt_cksum_hw; - else - __wt_cksum_func = __wt_cksum_sw; - -#elif defined(_M_AMD64) - int cpuInfo[4]; - - __cpuid(cpuInfo, 1); - - if (cpuInfo[2] & CPUID_ECX_HAS_SSE42) - __wt_cksum_func = __wt_cksum_hw; - else - __wt_cksum_func = __wt_cksum_sw; -#elif defined(__powerpc64__) - __wt_cksum_func = __wt_cksum_hw; -#else - __wt_cksum_func = __wt_cksum_sw; -#endif -} diff --git a/src/third_party/wiredtiger/src/support/power8/LICENSE.TXT b/src/third_party/wiredtiger/src/support/power8/LICENSE.TXT deleted file mode 100644 index 2f4bb91f574..00000000000 --- a/src/third_party/wiredtiger/src/support/power8/LICENSE.TXT +++ /dev/null @@ -1,476 +0,0 @@ -Copyright (C) 2015 Anton Blanchard <anton@au.ibm.com>, IBM - -crc32-vpmsum is free software; you can redistribute it and/or -modify it under the terms of either: - - a) the GNU General Public License as published by the Free Software Foundation; either - version 2 of the License, or (at your option) any later version., or - b) the Apache License, Version 2.0 - - - - - - - - GNU GENERAL PUBLIC LICENSE - Version 2, June 1991 - - Copyright (C) 1989, 1991 Free Software Foundation, Inc., - 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA - Everyone is permitted to copy and distribute verbatim copies - of this license document, but changing it is not allowed. - - Preamble - - The licenses for most software are designed to take away your -freedom to share and change it. By contrast, the GNU General Public -License is intended to guarantee your freedom to share and change free -software--to make sure the software is free for all its users. This -General Public License applies to most of the Free Software -Foundation's software and to any other program whose authors commit to -using it. (Some other Free Software Foundation software is covered by -the GNU Lesser General Public License instead.) You can apply it to -your programs, too. - - When we speak of free software, we are referring to freedom, not -price. Our General Public Licenses are designed to make sure that you -have the freedom to distribute copies of free software (and charge for -this service if you wish), that you receive source code or can get it -if you want it, that you can change the software or use pieces of it -in new free programs; and that you know you can do these things. - - To protect your rights, we need to make restrictions that forbid -anyone to deny you these rights or to ask you to surrender the rights. -These restrictions translate to certain responsibilities for you if you -distribute copies of the software, or if you modify it. - - For example, if you distribute copies of such a program, whether -gratis or for a fee, you must give the recipients all the rights that -you have. You must make sure that they, too, receive or can get the -source code. And you must show them these terms so they know their -rights. - - We protect your rights with two steps: (1) copyright the software, and -(2) offer you this license which gives you legal permission to copy, -distribute and/or modify the software. - - Also, for each author's protection and ours, we want to make certain -that everyone understands that there is no warranty for this free -software. If the software is modified by someone else and passed on, we -want its recipients to know that what they have is not the original, so -that any problems introduced by others will not reflect on the original -authors' reputations. - - Finally, any free program is threatened constantly by software -patents. We wish to avoid the danger that redistributors of a free -program will individually obtain patent licenses, in effect making the -program proprietary. To prevent this, we have made it clear that any -patent must be licensed for everyone's free use or not licensed at all. - - The precise terms and conditions for copying, distribution and -modification follow. - - GNU GENERAL PUBLIC LICENSE - TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION - - 0. This License applies to any program or other work which contains -a notice placed by the copyright holder saying it may be distributed -under the terms of this General Public License. The "Program", below, -refers to any such program or work, and a "work based on the Program" -means either the Program or any derivative work under copyright law: -that is to say, a work containing the Program or a portion of it, -either verbatim or with modifications and/or translated into another -language. (Hereinafter, translation is included without limitation in -the term "modification".) Each licensee is addressed as "you". - -Activities other than copying, distribution and modification are not -covered by this License; they are outside its scope. The act of -running the Program is not restricted, and the output from the Program -is covered only if its contents constitute a work based on the -Program (independent of having been made by running the Program). -Whether that is true depends on what the Program does. - - 1. You may copy and distribute verbatim copies of the Program's -source code as you receive it, in any medium, provided that you -conspicuously and appropriately publish on each copy an appropriate -copyright notice and disclaimer of warranty; keep intact all the -notices that refer to this License and to the absence of any warranty; -and give any other recipients of the Program a copy of this License -along with the Program. - -You may charge a fee for the physical act of transferring a copy, and -you may at your option offer warranty protection in exchange for a fee. - - 2. You may modify your copy or copies of the Program or any portion -of it, thus forming a work based on the Program, and copy and -distribute such modifications or work under the terms of Section 1 -above, provided that you also meet all of these conditions: - - a) You must cause the modified files to carry prominent notices - stating that you changed the files and the date of any change. - - b) You must cause any work that you distribute or publish, that in - whole or in part contains or is derived from the Program or any - part thereof, to be licensed as a whole at no charge to all third - parties under the terms of this License. - - c) If the modified program normally reads commands interactively - when run, you must cause it, when started running for such - interactive use in the most ordinary way, to print or display an - announcement including an appropriate copyright notice and a - notice that there is no warranty (or else, saying that you provide - a warranty) and that users may redistribute the program under - these conditions, and telling the user how to view a copy of this - License. (Exception: if the Program itself is interactive but - does not normally print such an announcement, your work based on - the Program is not required to print an announcement.) - -These requirements apply to the modified work as a whole. If -identifiable sections of that work are not derived from the Program, -and can be reasonably considered independent and separate works in -themselves, then this License, and its terms, do not apply to those -sections when you distribute them as separate works. But when you -distribute the same sections as part of a whole which is a work based -on the Program, the distribution of the whole must be on the terms of -this License, whose permissions for other licensees extend to the -entire whole, and thus to each and every part regardless of who wrote it. - -Thus, it is not the intent of this section to claim rights or contest -your rights to work written entirely by you; rather, the intent is to -exercise the right to control the distribution of derivative or -collective works based on the Program. - -In addition, mere aggregation of another work not based on the Program -with the Program (or with a work based on the Program) on a volume of -a storage or distribution medium does not bring the other work under -the scope of this License. - - 3. You may copy and distribute the Program (or a work based on it, -under Section 2) in object code or executable form under the terms of -Sections 1 and 2 above provided that you also do one of the following: - - a) Accompany it with the complete corresponding machine-readable - source code, which must be distributed under the terms of Sections - 1 and 2 above on a medium customarily used for software interchange; or, - - b) Accompany it with a written offer, valid for at least three - years, to give any third party, for a charge no more than your - cost of physically performing source distribution, a complete - machine-readable copy of the corresponding source code, to be - distributed under the terms of Sections 1 and 2 above on a medium - customarily used for software interchange; or, - - c) Accompany it with the information you received as to the offer - to distribute corresponding source code. (This alternative is - allowed only for noncommercial distribution and only if you - received the program in object code or executable form with such - an offer, in accord with Subsection b above.) - -The source code for a work means the preferred form of the work for -making modifications to it. For an executable work, complete source -code means all the source code for all modules it contains, plus any -associated interface definition files, plus the scripts used to -control compilation and installation of the executable. However, as a -special exception, the source code distributed need not include -anything that is normally distributed (in either source or binary -form) with the major components (compiler, kernel, and so on) of the -operating system on which the executable runs, unless that component -itself accompanies the executable. - -If distribution of executable or object code is made by offering -access to copy from a designated place, then offering equivalent -access to copy the source code from the same place counts as -distribution of the source code, even though third parties are not -compelled to copy the source along with the object code. - - 4. You may not copy, modify, sublicense, or distribute the Program -except as expressly provided under this License. Any attempt -otherwise to copy, modify, sublicense or distribute the Program is -void, and will automatically terminate your rights under this License. -However, parties who have received copies, or rights, from you under -this License will not have their licenses terminated so long as such -parties remain in full compliance. - - 5. You are not required to accept this License, since you have not -signed it. However, nothing else grants you permission to modify or -distribute the Program or its derivative works. These actions are -prohibited by law if you do not accept this License. Therefore, by -modifying or distributing the Program (or any work based on the -Program), you indicate your acceptance of this License to do so, and -all its terms and conditions for copying, distributing or modifying -the Program or works based on it. - - 6. Each time you redistribute the Program (or any work based on the -Program), the recipient automatically receives a license from the -original licensor to copy, distribute or modify the Program subject to -these terms and conditions. You may not impose any further -restrictions on the recipients' exercise of the rights granted herein. -You are not responsible for enforcing compliance by third parties to -this License. - - 7. If, as a consequence of a court judgment or allegation of patent -infringement or for any other reason (not limited to patent issues), -conditions are imposed on you (whether by court order, agreement or -otherwise) that contradict the conditions of this License, they do not -excuse you from the conditions of this License. If you cannot -distribute so as to satisfy simultaneously your obligations under this -License and any other pertinent obligations, then as a consequence you -may not distribute the Program at all. For example, if a patent -license would not permit royalty-free redistribution of the Program by -all those who receive copies directly or indirectly through you, then -the only way you could satisfy both it and this License would be to -refrain entirely from distribution of the Program. - -If any portion of this section is held invalid or unenforceable under -any particular circumstance, the balance of the section is intended to -apply and the section as a whole is intended to apply in other -circumstances. - -It is not the purpose of this section to induce you to infringe any -patents or other property right claims or to contest validity of any -such claims; this section has the sole purpose of protecting the -integrity of the free software distribution system, which is -implemented by public license practices. Many people have made -generous contributions to the wide range of software distributed -through that system in reliance on consistent application of that -system; it is up to the author/donor to decide if he or she is willing -to distribute software through any other system and a licensee cannot -impose that choice. - -This section is intended to make thoroughly clear what is believed to -be a consequence of the rest of this License. - - 8. If the distribution and/or use of the Program is restricted in -certain countries either by patents or by copyrighted interfaces, the -original copyright holder who places the Program under this License -may add an explicit geographical distribution limitation excluding -those countries, so that distribution is permitted only in or among -countries not thus excluded. In such case, this License incorporates -the limitation as if written in the body of this License. - - 9. The Free Software Foundation may publish revised and/or new versions -of the General Public License from time to time. Such new versions will -be similar in spirit to the present version, but may differ in detail to -address new problems or concerns. - -Each version is given a distinguishing version number. If the Program -specifies a version number of this License which applies to it and "any -later version", you have the option of following the terms and conditions -either of that version or of any later version published by the Free -Software Foundation. If the Program does not specify a version number of -this License, you may choose any version ever published by the Free Software -Foundation. - - 10. If you wish to incorporate parts of the Program into other free -programs whose distribution conditions are different, write to the author -to ask for permission. For software which is copyrighted by the Free -Software Foundation, write to the Free Software Foundation; we sometimes -make exceptions for this. Our decision will be guided by the two goals -of preserving the free status of all derivatives of our free software and -of promoting the sharing and reuse of software generally. - - NO WARRANTY - - 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY -FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN -OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES -PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED -OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF -MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS -TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE -PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, -REPAIR OR CORRECTION. - - 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING -WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR -REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, -INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING -OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED -TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY -YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER -PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE -POSSIBILITY OF SUCH DAMAGES. - - END OF TERMS AND CONDITIONS - - - - - - Apache License - Version 2.0, January 2004 - http://www.apache.org/licenses/ - - TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION - - 1. Definitions. - - "License" shall mean the terms and conditions for use, reproduction, - and distribution as defined by Sections 1 through 9 of this document. - - "Licensor" shall mean the copyright owner or entity authorized by - the copyright owner that is granting the License. - - "Legal Entity" shall mean the union of the acting entity and all - other entities that control, are controlled by, or are under common - control with that entity. For the purposes of this definition, - "control" means (i) the power, direct or indirect, to cause the - direction or management of such entity, whether by contract or - otherwise, or (ii) ownership of fifty percent (50%) or more of the - outstanding shares, or (iii) beneficial ownership of such entity. - - "You" (or "Your") shall mean an individual or Legal Entity - exercising permissions granted by this License. - - "Source" form shall mean the preferred form for making modifications, - including but not limited to software source code, documentation - source, and configuration files. - - "Object" form shall mean any form resulting from mechanical - transformation or translation of a Source form, including but - not limited to compiled object code, generated documentation, - and conversions to other media types. - - "Work" shall mean the work of authorship, whether in Source or - Object form, made available under the License, as indicated by a - copyright notice that is included in or attached to the work - (an example is provided in the Appendix below). - - "Derivative Works" shall mean any work, whether in Source or Object - form, that is based on (or derived from) the Work and for which the - editorial revisions, annotations, elaborations, or other modifications - represent, as a whole, an original work of authorship. For the purposes - of this License, Derivative Works shall not include works that remain - separable from, or merely link (or bind by name) to the interfaces of, - the Work and Derivative Works thereof. - - "Contribution" shall mean any work of authorship, including - the original version of the Work and any modifications or additions - to that Work or Derivative Works thereof, that is intentionally - submitted to Licensor for inclusion in the Work by the copyright owner - or by an individual or Legal Entity authorized to submit on behalf of - the copyright owner. For the purposes of this definition, "submitted" - means any form of electronic, verbal, or written communication sent - to the Licensor or its representatives, including but not limited to - communication on electronic mailing lists, source code control systems, - and issue tracking systems that are managed by, or on behalf of, the - Licensor for the purpose of discussing and improving the Work, but - excluding communication that is conspicuously marked or otherwise - designated in writing by the copyright owner as "Not a Contribution." - - "Contributor" shall mean Licensor and any individual or Legal Entity - on behalf of whom a Contribution has been received by Licensor and - subsequently incorporated within the Work. - - 2. Grant of Copyright License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - copyright license to reproduce, prepare Derivative Works of, - publicly display, publicly perform, sublicense, and distribute the - Work and such Derivative Works in Source or Object form. - - 3. Grant of Patent License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - (except as stated in this section) patent license to make, have made, - use, offer to sell, sell, import, and otherwise transfer the Work, - where such license applies only to those patent claims licensable - by such Contributor that are necessarily infringed by their - Contribution(s) alone or by combination of their Contribution(s) - with the Work to which such Contribution(s) was submitted. If You - institute patent litigation against any entity (including a - cross-claim or counterclaim in a lawsuit) alleging that the Work - or a Contribution incorporated within the Work constitutes direct - or contributory patent infringement, then any patent licenses - granted to You under this License for that Work shall terminate - as of the date such litigation is filed. - - 4. Redistribution. You may reproduce and distribute copies of the - Work or Derivative Works thereof in any medium, with or without - modifications, and in Source or Object form, provided that You - meet the following conditions: - - (a) You must give any other recipients of the Work or - Derivative Works a copy of this License; and - - (b) You must cause any modified files to carry prominent notices - stating that You changed the files; and - - (c) You must retain, in the Source form of any Derivative Works - that You distribute, all copyright, patent, trademark, and - attribution notices from the Source form of the Work, - excluding those notices that do not pertain to any part of - the Derivative Works; and - - (d) If the Work includes a "NOTICE" text file as part of its - distribution, then any Derivative Works that You distribute must - include a readable copy of the attribution notices contained - within such NOTICE file, excluding those notices that do not - pertain to any part of the Derivative Works, in at least one - of the following places: within a NOTICE text file distributed - as part of the Derivative Works; within the Source form or - documentation, if provided along with the Derivative Works; or, - within a display generated by the Derivative Works, if and - wherever such third-party notices normally appear. The contents - of the NOTICE file are for informational purposes only and - do not modify the License. You may add Your own attribution - notices within Derivative Works that You distribute, alongside - or as an addendum to the NOTICE text from the Work, provided - that such additional attribution notices cannot be construed - as modifying the License. - - You may add Your own copyright statement to Your modifications and - may provide additional or different license terms and conditions - for use, reproduction, or distribution of Your modifications, or - for any such Derivative Works as a whole, provided Your use, - reproduction, and distribution of the Work otherwise complies with - the conditions stated in this License. - - 5. Submission of Contributions. Unless You explicitly state otherwise, - any Contribution intentionally submitted for inclusion in the Work - by You to the Licensor shall be under the terms and conditions of - this License, without any additional terms or conditions. - Notwithstanding the above, nothing herein shall supersede or modify - the terms of any separate license agreement you may have executed - with Licensor regarding such Contributions. - - 6. Trademarks. This License does not grant permission to use the trade - names, trademarks, service marks, or product names of the Licensor, - except as required for reasonable and customary use in describing the - origin of the Work and reproducing the content of the NOTICE file. - - 7. Disclaimer of Warranty. Unless required by applicable law or - agreed to in writing, Licensor provides the Work (and each - Contributor provides its Contributions) on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied, including, without limitation, any warranties or conditions - of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A - PARTICULAR PURPOSE. You are solely responsible for determining the - appropriateness of using or redistributing the Work and assume any - risks associated with Your exercise of permissions under this License. - - 8. Limitation of Liability. In no event and under no legal theory, - whether in tort (including negligence), contract, or otherwise, - unless required by applicable law (such as deliberate and grossly - negligent acts) or agreed to in writing, shall any Contributor be - liable to You for damages, including any direct, indirect, special, - incidental, or consequential damages of any character arising as a - result of this License or out of the use or inability to use the - Work (including but not limited to damages for loss of goodwill, - work stoppage, computer failure or malfunction, or any and all - other commercial damages or losses), even if such Contributor - has been advised of the possibility of such damages. - - 9. Accepting Warranty or Additional Liability. While redistributing - the Work or Derivative Works thereof, You may choose to offer, - and charge a fee for, acceptance of support, warranty, indemnity, - or other liability obligations and/or rights consistent with this - License. However, in accepting such obligations, You may act only - on Your own behalf and on Your sole responsibility, not on behalf - of any other Contributor, and only if You agree to indemnify, - defend, and hold each Contributor harmless for any liability - incurred by, or claims asserted against, such Contributor by reason - of your accepting any such warranty or additional liability. - - END OF TERMS AND CONDITIONS diff --git a/src/third_party/wiredtiger/src/support/power8/README.md b/src/third_party/wiredtiger/src/support/power8/README.md deleted file mode 100644 index 3e2976650cd..00000000000 --- a/src/third_party/wiredtiger/src/support/power8/README.md +++ /dev/null @@ -1,208 +0,0 @@ -crc32-vpmsum -============ - -A set of examples for accelerating CRC32 calculations using the vector -polynomial multiply sum (vpmsum) instructions introduced in POWER8. These -instructions implement byte, halfword, word and doubleword carryless -multiply/add. - -Performance ------------ - -An implementation of slice-by-8, one of the fastest lookup table methods -is included so we can compare performance against it. Testing 5000000 -iterations of a CRC of 32 kB of data (to keep it L1 cache contained): - -``` -# time slice_by_8_bench 32768 5000000 -122.220 seconds - -# time crc32_bench 32768 5000000 -2.937 seconds -``` - -The vpmsum accelerated CRC is just over 41x faster. - -This test was run on a 4.1 GHz POWER8, so the algorithm sustains about -52 GiB/sec or 13.6 bytes/cycle. The theoretical limit is 16 bytes/cycle -since we can execute a maximum of one vpmsum instruction per cycle. - -In another test, a version was added to the kernel and btrfs write -performance was shown to be 3.8x faster. The test was done to a ramdisk -to mitigate any I/O induced variability. - -Quick start ------------ - -- Modify CRC and OPTIONS in the Makefile. There are examples for the two most - common crc32s. - -- Type make to create the constants (crc32_constants.h) - -- Import the code into your application (crc32.S crc32_wrapper.c - crc32_constants.h ppc-opcode.h) and call the CRC: - -``` -unsigned int crc32_vpmsum(unsigned int crc, unsigned char *p, unsigned long len); -``` - -CRC background --------------- - -For a good background on CRCs, check out: - -http://www.ross.net/crc/download/crc_v3.txt - -A few key points: - -- A CRC is the remainder after dividing a message by the CRC polynomial, - ie M mod CRC_POLY -- multiply/divide is carryless -- add/subtract is an xor -- n (where n is the order of the CRC) bits of zeroes are appended to the - end of the message. - -One more important piece of information - a CRC is a linear function, so: - -``` - CRC(A xor B) = CRC(A) xor CRC(B) - - CRC(A . B) = CRC(A) . CRC(B) (remember this is carryless multiply) -``` - -If we take 64bits of data, represented by two 32 bit chunks (AAAAAAAA -and BBBBBBBB): - -``` -CRC(AAAAAAAABBBBBBBB) - = CRC(AAAAAAAA00000000 xor BBBBBBBB) - = CRC(AAAAAAAA00000000) xor CRC(BBBBBBBB) -``` - -If we operate on AAAAAAAA: - -``` -CRC(AAAAAAAA00000000) - = CRC(AAAAAAAA . 100000000) - = CRC(AAAAAAAA) . CRC(100000000) -``` - -And CRC(100000000) is a constant which we can pre-calculate: - -``` -CRC(100000000) - = 100000000 mod CRC_POLY - = 2^32 mod CRC_POLY -``` - -Finally we can add our modified AAAAAAAA to BBBBBBBB: - -``` -CRC(AAAAAAAABBBBBBBB) - = ((2^32 mod CRC_POLY) . CRC(AAAAAAAA)) xor CRC(BBBBBBBB) -``` - -In other words, with the right constants pre-calculated we can shift the -input data around and we can also calculate the CRC in as many parallel -chunks as we want. - -No matter how much shifting we do, the final result will be be 64 bits of -data (63 actually, because there is no carry into the top bit). To reduce -it further we need a another trick, and that is Barrett reduction: - -http://en.wikipedia.org/wiki/Barrett_reduction - -Barrett reduction is a method of calculating a mod n. The idea is to -calculate q, the multiple of our polynomial that we need to subtract. By -doing the computation 2x bits higher (ie 64 bits) and shifting the -result back down 2x bits, we round down to the nearest multiple. - -``` - k = 32 - m = floor((4^k)/n) = floor((4^32))/n) - n = 64 bits of data - a = 32 bit CRC - - q = floor(ma/(2^64)) - result = a - qn -``` - -An example in the floating point domain makes it clearer how this works: - -``` -a mod n = a - floor(am) * n -``` - -Let's use it to calculate 22 mod 10: - -``` - a = 22 - n = 10 - m = 1/n = 1/10 = 0.1 - -22 mod 10 - = 22 - floor(22*0.1) * 10 - = 22 - 2 * 10 - = 22 - 20 - = 2 -``` - -There is one more issue left - bit reflection. Some CRCs are defined to -operate on the least significant bit first (eg CRC32c). Lets look at -how this would get laid out in a register, and lets simplify it to just -two bytes (vs a 16 byte VMX register): - - [ 8..15 ] [ 0..7 ] - -Notice how the bits and bytes are out of order. Since we are doing -multi word multiplication on these values we need them to both be -in order. - -The simplest way to fix this is to reflect the bits in each byte: - - [ 15..8 ] [ 7..0 ] - -However shuffling bits in a byte is expensive on most CPUs. It is -however relatively cheap to shuffle bytes around. What if we load -the bytes in reversed: - - [ 0..7 ] [ 8..15 ] - -Now the bits and bytes are in order, except the least significant bit -of the register is now on the left and the most significant bit is on the -right. We operate as if the register is reflected, which normally we -cannot do. The reason we get away with this is our multiplies are carryless -and our addition and subtraction is xor, so our operations never create -carries. - -The only trick is we have to shift the result of multiplies left one -because the high bit of the multiply is always 0, and we want that high bit -on the right not the left. - -Implementation --------------- - -The vpmsum instructions on POWER8 have a 6 cycle latency and we can -execute one every cycle. In light of this the main loop has 8 parallel -streams which consume 8 x 16 B each iteration. At the completion of this -loop we have taken 32 kB of data and reduced it to 8 x 16 B (128 B). - -The next step is to take this 128 B and reduce it to 8 B. At this stage -we also add 32 bits of 0 to the end. - -We then apply Barrett reduction to get our CRC. - -Examples --------- -- barrett_reduction: An example of Barrett reduction - -- final_fold: Starting with 128 bits, add 32 bits of zeros and reduce it to - 64 bits, then apply Barrett reduction - -- final_fold2: A second method of reduction - -Acknowledgements ----------------- - -Thanks to Michael Gschwind, Jeff Derby, Lorena Pesantez and Stewart Smith -for their ideas and assistance. diff --git a/src/third_party/wiredtiger/src/support/power8/crc32.S b/src/third_party/wiredtiger/src/support/power8/crc32.S deleted file mode 100644 index f990acb7b12..00000000000 --- a/src/third_party/wiredtiger/src/support/power8/crc32.S +++ /dev/null @@ -1,778 +0,0 @@ -#if defined(__powerpc64__) -/* - * Calculate the checksum of data that is 16 byte aligned and a multiple of - * 16 bytes. - * - * The first step is to reduce it to 1024 bits. We do this in 8 parallel - * chunks in order to mask the latency of the vpmsum instructions. If we - * have more than 32 kB of data to checksum we repeat this step multiple - * times, passing in the previous 1024 bits. - * - * The next step is to reduce the 1024 bits to 64 bits. This step adds - * 32 bits of 0s to the end - this matches what a CRC does. We just - * calculate constants that land the data in this 32 bits. - * - * We then use fixed point Barrett reduction to compute a mod n over GF(2) - * for n = CRC using POWER8 instructions. We use x = 32. - * - * http://en.wikipedia.org/wiki/Barrett_reduction - * - * Copyright (C) 2015 Anton Blanchard <anton@au.ibm.com>, IBM - * - * This program is free software; you can redistribute it and/or - * modify it under the terms of the GNU General Public License - * as published by the Free Software Foundation; either version - * 2 of the License, or (at your option) any later version. - */ -#include <ppc-asm.h> -#include "ppc-opcode.h" - -#undef toc - -#ifndef r1 -#define r1 1 -#endif - -#ifndef r2 -#define r2 2 -#endif - - .section .rodata -.balign 16 - -.byteswap_constant: - /* byte reverse permute constant */ - .octa 0x0F0E0D0C0B0A09080706050403020100 - -#define __ASSEMBLY__ -#include "crc32_constants.h" - - .text - -#if defined(__BIG_ENDIAN__) && defined(REFLECT) -#define BYTESWAP_DATA -#elif defined(__LITTLE_ENDIAN__) && !defined(REFLECT) -#define BYTESWAP_DATA -#else -#undef BYTESWAP_DATA -#endif - -#define off16 r25 -#define off32 r26 -#define off48 r27 -#define off64 r28 -#define off80 r29 -#define off96 r30 -#define off112 r31 - -#define const1 v24 -#define const2 v25 - -#define byteswap v26 -#define mask_32bit v27 -#define mask_64bit v28 -#define zeroes v29 - -#ifdef BYTESWAP_DATA -#define VPERM(A, B, C, D) vperm A, B, C, D -#else -#define VPERM(A, B, C, D) -#endif - -/* unsigned int __crc32_vpmsum(unsigned int crc, void *p, unsigned long len) */ -FUNC_START(__crc32_vpmsum) - std r31,-8(r1) - std r30,-16(r1) - std r29,-24(r1) - std r28,-32(r1) - std r27,-40(r1) - std r26,-48(r1) - std r25,-56(r1) - - li off16,16 - li off32,32 - li off48,48 - li off64,64 - li off80,80 - li off96,96 - li off112,112 - li r0,0 - - /* Enough room for saving 10 non volatile VMX registers */ - subi r6,r1,56+10*16 - subi r7,r1,56+2*16 - - stvx v20,0,r6 - stvx v21,off16,r6 - stvx v22,off32,r6 - stvx v23,off48,r6 - stvx v24,off64,r6 - stvx v25,off80,r6 - stvx v26,off96,r6 - stvx v27,off112,r6 - stvx v28,0,r7 - stvx v29,off16,r7 - - mr r10,r3 - - vxor zeroes,zeroes,zeroes - vspltisw v0,-1 - - vsldoi mask_32bit,zeroes,v0,4 - vsldoi mask_64bit,zeroes,v0,8 - - /* Get the initial value into v8 */ - vxor v8,v8,v8 - MTVRD(v8, r3) -#ifdef REFLECT - vsldoi v8,zeroes,v8,8 /* shift into bottom 32 bits */ -#else - vsldoi v8,v8,zeroes,4 /* shift into top 32 bits */ -#endif - -#ifdef BYTESWAP_DATA - addis r3,r2,.byteswap_constant@toc@ha - addi r3,r3,.byteswap_constant@toc@l - - lvx byteswap,0,r3 - addi r3,r3,16 -#endif - - cmpdi r5,256 - blt .Lshort - - rldicr r6,r5,0,56 - - /* Checksum in blocks of MAX_SIZE */ -1: lis r7,MAX_SIZE@h - ori r7,r7,MAX_SIZE@l - mr r9,r7 - cmpd r6,r7 - bgt 2f - mr r7,r6 -2: subf r6,r7,r6 - - /* our main loop does 128 bytes at a time */ - srdi r7,r7,7 - - /* - * Work out the offset into the constants table to start at. Each - * constant is 16 bytes, and it is used against 128 bytes of input - * data - 128 / 16 = 8 - */ - sldi r8,r7,4 - srdi r9,r9,3 - subf r8,r8,r9 - - /* We reduce our final 128 bytes in a separate step */ - addi r7,r7,-1 - mtctr r7 - - addis r3,r2,.constants@toc@ha - addi r3,r3,.constants@toc@l - - /* Find the start of our constants */ - add r3,r3,r8 - - /* zero v0-v7 which will contain our checksums */ - vxor v0,v0,v0 - vxor v1,v1,v1 - vxor v2,v2,v2 - vxor v3,v3,v3 - vxor v4,v4,v4 - vxor v5,v5,v5 - vxor v6,v6,v6 - vxor v7,v7,v7 - - lvx const1,0,r3 - - /* - * If we are looping back to consume more data we use the values - * already in v16-v23. - */ - cmpdi r0,1 - beq 2f - - /* First warm up pass */ - lvx v16,0,r4 - lvx v17,off16,r4 - VPERM(v16,v16,v16,byteswap) - VPERM(v17,v17,v17,byteswap) - lvx v18,off32,r4 - lvx v19,off48,r4 - VPERM(v18,v18,v18,byteswap) - VPERM(v19,v19,v19,byteswap) - lvx v20,off64,r4 - lvx v21,off80,r4 - VPERM(v20,v20,v20,byteswap) - VPERM(v21,v21,v21,byteswap) - lvx v22,off96,r4 - lvx v23,off112,r4 - VPERM(v22,v22,v22,byteswap) - VPERM(v23,v23,v23,byteswap) - addi r4,r4,8*16 - - /* xor in initial value */ - vxor v16,v16,v8 - -2: bdz .Lfirst_warm_up_done - - addi r3,r3,16 - lvx const2,0,r3 - - /* Second warm up pass */ - VPMSUMD(v8,v16,const1) - lvx v16,0,r4 - VPERM(v16,v16,v16,byteswap) - ori r2,r2,0 - - VPMSUMD(v9,v17,const1) - lvx v17,off16,r4 - VPERM(v17,v17,v17,byteswap) - ori r2,r2,0 - - VPMSUMD(v10,v18,const1) - lvx v18,off32,r4 - VPERM(v18,v18,v18,byteswap) - ori r2,r2,0 - - VPMSUMD(v11,v19,const1) - lvx v19,off48,r4 - VPERM(v19,v19,v19,byteswap) - ori r2,r2,0 - - VPMSUMD(v12,v20,const1) - lvx v20,off64,r4 - VPERM(v20,v20,v20,byteswap) - ori r2,r2,0 - - VPMSUMD(v13,v21,const1) - lvx v21,off80,r4 - VPERM(v21,v21,v21,byteswap) - ori r2,r2,0 - - VPMSUMD(v14,v22,const1) - lvx v22,off96,r4 - VPERM(v22,v22,v22,byteswap) - ori r2,r2,0 - - VPMSUMD(v15,v23,const1) - lvx v23,off112,r4 - VPERM(v23,v23,v23,byteswap) - - addi r4,r4,8*16 - - bdz .Lfirst_cool_down - - /* - * main loop. We modulo schedule it such that it takes three iterations - * to complete - first iteration load, second iteration vpmsum, third - * iteration xor. - */ - .balign 16 -4: lvx const1,0,r3 - addi r3,r3,16 - ori r2,r2,0 - - vxor v0,v0,v8 - VPMSUMD(v8,v16,const2) - lvx v16,0,r4 - VPERM(v16,v16,v16,byteswap) - ori r2,r2,0 - - vxor v1,v1,v9 - VPMSUMD(v9,v17,const2) - lvx v17,off16,r4 - VPERM(v17,v17,v17,byteswap) - ori r2,r2,0 - - vxor v2,v2,v10 - VPMSUMD(v10,v18,const2) - lvx v18,off32,r4 - VPERM(v18,v18,v18,byteswap) - ori r2,r2,0 - - vxor v3,v3,v11 - VPMSUMD(v11,v19,const2) - lvx v19,off48,r4 - VPERM(v19,v19,v19,byteswap) - lvx const2,0,r3 - ori r2,r2,0 - - vxor v4,v4,v12 - VPMSUMD(v12,v20,const1) - lvx v20,off64,r4 - VPERM(v20,v20,v20,byteswap) - ori r2,r2,0 - - vxor v5,v5,v13 - VPMSUMD(v13,v21,const1) - lvx v21,off80,r4 - VPERM(v21,v21,v21,byteswap) - ori r2,r2,0 - - vxor v6,v6,v14 - VPMSUMD(v14,v22,const1) - lvx v22,off96,r4 - VPERM(v22,v22,v22,byteswap) - ori r2,r2,0 - - vxor v7,v7,v15 - VPMSUMD(v15,v23,const1) - lvx v23,off112,r4 - VPERM(v23,v23,v23,byteswap) - - addi r4,r4,8*16 - - bdnz 4b - -.Lfirst_cool_down: - /* First cool down pass */ - lvx const1,0,r3 - addi r3,r3,16 - - vxor v0,v0,v8 - VPMSUMD(v8,v16,const1) - ori r2,r2,0 - - vxor v1,v1,v9 - VPMSUMD(v9,v17,const1) - ori r2,r2,0 - - vxor v2,v2,v10 - VPMSUMD(v10,v18,const1) - ori r2,r2,0 - - vxor v3,v3,v11 - VPMSUMD(v11,v19,const1) - ori r2,r2,0 - - vxor v4,v4,v12 - VPMSUMD(v12,v20,const1) - ori r2,r2,0 - - vxor v5,v5,v13 - VPMSUMD(v13,v21,const1) - ori r2,r2,0 - - vxor v6,v6,v14 - VPMSUMD(v14,v22,const1) - ori r2,r2,0 - - vxor v7,v7,v15 - VPMSUMD(v15,v23,const1) - ori r2,r2,0 - -.Lsecond_cool_down: - /* Second cool down pass */ - vxor v0,v0,v8 - vxor v1,v1,v9 - vxor v2,v2,v10 - vxor v3,v3,v11 - vxor v4,v4,v12 - vxor v5,v5,v13 - vxor v6,v6,v14 - vxor v7,v7,v15 - -#ifdef REFLECT - /* - * vpmsumd produces a 96 bit result in the least significant bits - * of the register. Since we are bit reflected we have to shift it - * left 32 bits so it occupies the least significant bits in the - * bit reflected domain. - */ - vsldoi v0,v0,zeroes,4 - vsldoi v1,v1,zeroes,4 - vsldoi v2,v2,zeroes,4 - vsldoi v3,v3,zeroes,4 - vsldoi v4,v4,zeroes,4 - vsldoi v5,v5,zeroes,4 - vsldoi v6,v6,zeroes,4 - vsldoi v7,v7,zeroes,4 -#endif - - /* xor with last 1024 bits */ - lvx v8,0,r4 - lvx v9,off16,r4 - VPERM(v8,v8,v8,byteswap) - VPERM(v9,v9,v9,byteswap) - lvx v10,off32,r4 - lvx v11,off48,r4 - VPERM(v10,v10,v10,byteswap) - VPERM(v11,v11,v11,byteswap) - lvx v12,off64,r4 - lvx v13,off80,r4 - VPERM(v12,v12,v12,byteswap) - VPERM(v13,v13,v13,byteswap) - lvx v14,off96,r4 - lvx v15,off112,r4 - VPERM(v14,v14,v14,byteswap) - VPERM(v15,v15,v15,byteswap) - - addi r4,r4,8*16 - - vxor v16,v0,v8 - vxor v17,v1,v9 - vxor v18,v2,v10 - vxor v19,v3,v11 - vxor v20,v4,v12 - vxor v21,v5,v13 - vxor v22,v6,v14 - vxor v23,v7,v15 - - li r0,1 - cmpdi r6,0 - addi r6,r6,128 - bne 1b - - /* Work out how many bytes we have left */ - andi. r5,r5,127 - - /* Calculate where in the constant table we need to start */ - subfic r6,r5,128 - add r3,r3,r6 - - /* How many 16 byte chunks are in the tail */ - srdi r7,r5,4 - mtctr r7 - - /* - * Reduce the previously calculated 1024 bits to 64 bits, shifting - * 32 bits to include the trailing 32 bits of zeros - */ - lvx v0,0,r3 - lvx v1,off16,r3 - lvx v2,off32,r3 - lvx v3,off48,r3 - lvx v4,off64,r3 - lvx v5,off80,r3 - lvx v6,off96,r3 - lvx v7,off112,r3 - addi r3,r3,8*16 - - VPMSUMW(v0,v16,v0) - VPMSUMW(v1,v17,v1) - VPMSUMW(v2,v18,v2) - VPMSUMW(v3,v19,v3) - VPMSUMW(v4,v20,v4) - VPMSUMW(v5,v21,v5) - VPMSUMW(v6,v22,v6) - VPMSUMW(v7,v23,v7) - - /* Now reduce the tail (0 - 112 bytes) */ - cmpdi r7,0 - beq 1f - - lvx v16,0,r4 - lvx v17,0,r3 - VPERM(v16,v16,v16,byteswap) - VPMSUMW(v16,v16,v17) - vxor v0,v0,v16 - bdz 1f - - lvx v16,off16,r4 - lvx v17,off16,r3 - VPERM(v16,v16,v16,byteswap) - VPMSUMW(v16,v16,v17) - vxor v0,v0,v16 - bdz 1f - - lvx v16,off32,r4 - lvx v17,off32,r3 - VPERM(v16,v16,v16,byteswap) - VPMSUMW(v16,v16,v17) - vxor v0,v0,v16 - bdz 1f - - lvx v16,off48,r4 - lvx v17,off48,r3 - VPERM(v16,v16,v16,byteswap) - VPMSUMW(v16,v16,v17) - vxor v0,v0,v16 - bdz 1f - - lvx v16,off64,r4 - lvx v17,off64,r3 - VPERM(v16,v16,v16,byteswap) - VPMSUMW(v16,v16,v17) - vxor v0,v0,v16 - bdz 1f - - lvx v16,off80,r4 - lvx v17,off80,r3 - VPERM(v16,v16,v16,byteswap) - VPMSUMW(v16,v16,v17) - vxor v0,v0,v16 - bdz 1f - - lvx v16,off96,r4 - lvx v17,off96,r3 - VPERM(v16,v16,v16,byteswap) - VPMSUMW(v16,v16,v17) - vxor v0,v0,v16 - - /* Now xor all the parallel chunks together */ -1: vxor v0,v0,v1 - vxor v2,v2,v3 - vxor v4,v4,v5 - vxor v6,v6,v7 - - vxor v0,v0,v2 - vxor v4,v4,v6 - - vxor v0,v0,v4 - -.Lbarrett_reduction: - /* Barrett constants */ - addis r3,r2,.barrett_constants@toc@ha - addi r3,r3,.barrett_constants@toc@l - - lvx const1,0,r3 - lvx const2,off16,r3 - - vsldoi v1,v0,v0,8 - vxor v0,v0,v1 /* xor two 64 bit results together */ - -#ifdef REFLECT - /* shift left one bit */ - vspltisb v1,1 - vsl v0,v0,v1 -#endif - - vand v0,v0,mask_64bit - -#ifndef REFLECT - /* - * Now for the Barrett reduction algorithm. The idea is to calculate q, - * the multiple of our polynomial that we need to subtract. By - * doing the computation 2x bits higher (ie 64 bits) and shifting the - * result back down 2x bits, we round down to the nearest multiple. - */ - VPMSUMD(v1,v0,const1) /* ma */ - vsldoi v1,zeroes,v1,8 /* q = floor(ma/(2^64)) */ - VPMSUMD(v1,v1,const2) /* qn */ - vxor v0,v0,v1 /* a - qn, subtraction is xor in GF(2) */ - - /* - * Get the result into r3. We need to shift it left 8 bytes: - * V0 [ 0 1 2 X ] - * V0 [ 0 X 2 3 ] - */ - vsldoi v0,v0,zeroes,8 /* shift result into top 64 bits */ -#else - /* - * The reflected version of Barrett reduction. Instead of bit - * reflecting our data (which is expensive to do), we bit reflect our - * constants and our algorithm, which means the intermediate data in - * our vector registers goes from 0-63 instead of 63-0. We can reflect - * the algorithm because we don't carry in mod 2 arithmetic. - */ - vand v1,v0,mask_32bit /* bottom 32 bits of a */ - VPMSUMD(v1,v1,const1) /* ma */ - vand v1,v1,mask_32bit /* bottom 32bits of ma */ - VPMSUMD(v1,v1,const2) /* qn */ - vxor v0,v0,v1 /* a - qn, subtraction is xor in GF(2) */ - - /* - * Since we are bit reflected, the result (ie the low 32 bits) is in - * the high 32 bits. We just need to shift it left 4 bytes - * V0 [ 0 1 X 3 ] - * V0 [ 0 X 2 3 ] - */ - vsldoi v0,v0,zeroes,4 /* shift result into top 64 bits of */ -#endif - - /* Get it into r3 */ - MFVRD(r3, v0) - -.Lout: - subi r6,r1,56+10*16 - subi r7,r1,56+2*16 - - lvx v20,0,r6 - lvx v21,off16,r6 - lvx v22,off32,r6 - lvx v23,off48,r6 - lvx v24,off64,r6 - lvx v25,off80,r6 - lvx v26,off96,r6 - lvx v27,off112,r6 - lvx v28,0,r7 - lvx v29,off16,r7 - - ld r31,-8(r1) - ld r30,-16(r1) - ld r29,-24(r1) - ld r28,-32(r1) - ld r27,-40(r1) - ld r26,-48(r1) - ld r25,-56(r1) - - blr - -.Lfirst_warm_up_done: - lvx const1,0,r3 - addi r3,r3,16 - - VPMSUMD(v8,v16,const1) - VPMSUMD(v9,v17,const1) - VPMSUMD(v10,v18,const1) - VPMSUMD(v11,v19,const1) - VPMSUMD(v12,v20,const1) - VPMSUMD(v13,v21,const1) - VPMSUMD(v14,v22,const1) - VPMSUMD(v15,v23,const1) - - b .Lsecond_cool_down - -.Lshort: - cmpdi r5,0 - beq .Lzero - - addis r3,r2,.short_constants@toc@ha - addi r3,r3,.short_constants@toc@l - - /* Calculate where in the constant table we need to start */ - subfic r6,r5,256 - add r3,r3,r6 - - /* How many 16 byte chunks? */ - srdi r7,r5,4 - mtctr r7 - - vxor v19,v19,v19 - vxor v20,v20,v20 - - lvx v0,0,r4 - lvx v16,0,r3 - VPERM(v0,v0,v16,byteswap) - vxor v0,v0,v8 /* xor in initial value */ - VPMSUMW(v0,v0,v16) - bdz .Lv0 - - lvx v1,off16,r4 - lvx v17,off16,r3 - VPERM(v1,v1,v17,byteswap) - VPMSUMW(v1,v1,v17) - bdz .Lv1 - - lvx v2,off32,r4 - lvx v16,off32,r3 - VPERM(v2,v2,v16,byteswap) - VPMSUMW(v2,v2,v16) - bdz .Lv2 - - lvx v3,off48,r4 - lvx v17,off48,r3 - VPERM(v3,v3,v17,byteswap) - VPMSUMW(v3,v3,v17) - bdz .Lv3 - - lvx v4,off64,r4 - lvx v16,off64,r3 - VPERM(v4,v4,v16,byteswap) - VPMSUMW(v4,v4,v16) - bdz .Lv4 - - lvx v5,off80,r4 - lvx v17,off80,r3 - VPERM(v5,v5,v17,byteswap) - VPMSUMW(v5,v5,v17) - bdz .Lv5 - - lvx v6,off96,r4 - lvx v16,off96,r3 - VPERM(v6,v6,v16,byteswap) - VPMSUMW(v6,v6,v16) - bdz .Lv6 - - lvx v7,off112,r4 - lvx v17,off112,r3 - VPERM(v7,v7,v17,byteswap) - VPMSUMW(v7,v7,v17) - bdz .Lv7 - - addi r3,r3,128 - addi r4,r4,128 - - lvx v8,0,r4 - lvx v16,0,r3 - VPERM(v8,v8,v16,byteswap) - VPMSUMW(v8,v8,v16) - bdz .Lv8 - - lvx v9,off16,r4 - lvx v17,off16,r3 - VPERM(v9,v9,v17,byteswap) - VPMSUMW(v9,v9,v17) - bdz .Lv9 - - lvx v10,off32,r4 - lvx v16,off32,r3 - VPERM(v10,v10,v16,byteswap) - VPMSUMW(v10,v10,v16) - bdz .Lv10 - - lvx v11,off48,r4 - lvx v17,off48,r3 - VPERM(v11,v11,v17,byteswap) - VPMSUMW(v11,v11,v17) - bdz .Lv11 - - lvx v12,off64,r4 - lvx v16,off64,r3 - VPERM(v12,v12,v16,byteswap) - VPMSUMW(v12,v12,v16) - bdz .Lv12 - - lvx v13,off80,r4 - lvx v17,off80,r3 - VPERM(v13,v13,v17,byteswap) - VPMSUMW(v13,v13,v17) - bdz .Lv13 - - lvx v14,off96,r4 - lvx v16,off96,r3 - VPERM(v14,v14,v16,byteswap) - VPMSUMW(v14,v14,v16) - bdz .Lv14 - - lvx v15,off112,r4 - lvx v17,off112,r3 - VPERM(v15,v15,v17,byteswap) - VPMSUMW(v15,v15,v17) - -.Lv15: vxor v19,v19,v15 -.Lv14: vxor v20,v20,v14 -.Lv13: vxor v19,v19,v13 -.Lv12: vxor v20,v20,v12 -.Lv11: vxor v19,v19,v11 -.Lv10: vxor v20,v20,v10 -.Lv9: vxor v19,v19,v9 -.Lv8: vxor v20,v20,v8 -.Lv7: vxor v19,v19,v7 -.Lv6: vxor v20,v20,v6 -.Lv5: vxor v19,v19,v5 -.Lv4: vxor v20,v20,v4 -.Lv3: vxor v19,v19,v3 -.Lv2: vxor v20,v20,v2 -.Lv1: vxor v19,v19,v1 -.Lv0: vxor v20,v20,v0 - - vxor v0,v19,v20 - - b .Lbarrett_reduction - -.Lzero: - mr r3,r10 - b .Lout - -FUNC_END(__crc32_vpmsum) -#endif - -/* - * Make sure the stack isn't executable with GCC (regardless of platform). - */ -#ifndef __clang__ -.section .note.GNU-stack,"",@progbits -#endif diff --git a/src/third_party/wiredtiger/src/support/power8/crc32_constants.h b/src/third_party/wiredtiger/src/support/power8/crc32_constants.h deleted file mode 100644 index 02c471d1c56..00000000000 --- a/src/third_party/wiredtiger/src/support/power8/crc32_constants.h +++ /dev/null @@ -1,901 +0,0 @@ -#define CRC 0x1edc6f41 -#define CRC_XOR -#define REFLECT - -#ifndef __ASSEMBLY__ -#ifdef CRC_TABLE -static const unsigned int crc_table[] = { - 0x00000000, 0xf26b8303, 0xe13b70f7, 0x1350f3f4, - 0xc79a971f, 0x35f1141c, 0x26a1e7e8, 0xd4ca64eb, - 0x8ad958cf, 0x78b2dbcc, 0x6be22838, 0x9989ab3b, - 0x4d43cfd0, 0xbf284cd3, 0xac78bf27, 0x5e133c24, - 0x105ec76f, 0xe235446c, 0xf165b798, 0x030e349b, - 0xd7c45070, 0x25afd373, 0x36ff2087, 0xc494a384, - 0x9a879fa0, 0x68ec1ca3, 0x7bbcef57, 0x89d76c54, - 0x5d1d08bf, 0xaf768bbc, 0xbc267848, 0x4e4dfb4b, - 0x20bd8ede, 0xd2d60ddd, 0xc186fe29, 0x33ed7d2a, - 0xe72719c1, 0x154c9ac2, 0x061c6936, 0xf477ea35, - 0xaa64d611, 0x580f5512, 0x4b5fa6e6, 0xb93425e5, - 0x6dfe410e, 0x9f95c20d, 0x8cc531f9, 0x7eaeb2fa, - 0x30e349b1, 0xc288cab2, 0xd1d83946, 0x23b3ba45, - 0xf779deae, 0x05125dad, 0x1642ae59, 0xe4292d5a, - 0xba3a117e, 0x4851927d, 0x5b016189, 0xa96ae28a, - 0x7da08661, 0x8fcb0562, 0x9c9bf696, 0x6ef07595, - 0x417b1dbc, 0xb3109ebf, 0xa0406d4b, 0x522bee48, - 0x86e18aa3, 0x748a09a0, 0x67dafa54, 0x95b17957, - 0xcba24573, 0x39c9c670, 0x2a993584, 0xd8f2b687, - 0x0c38d26c, 0xfe53516f, 0xed03a29b, 0x1f682198, - 0x5125dad3, 0xa34e59d0, 0xb01eaa24, 0x42752927, - 0x96bf4dcc, 0x64d4cecf, 0x77843d3b, 0x85efbe38, - 0xdbfc821c, 0x2997011f, 0x3ac7f2eb, 0xc8ac71e8, - 0x1c661503, 0xee0d9600, 0xfd5d65f4, 0x0f36e6f7, - 0x61c69362, 0x93ad1061, 0x80fde395, 0x72966096, - 0xa65c047d, 0x5437877e, 0x4767748a, 0xb50cf789, - 0xeb1fcbad, 0x197448ae, 0x0a24bb5a, 0xf84f3859, - 0x2c855cb2, 0xdeeedfb1, 0xcdbe2c45, 0x3fd5af46, - 0x7198540d, 0x83f3d70e, 0x90a324fa, 0x62c8a7f9, - 0xb602c312, 0x44694011, 0x5739b3e5, 0xa55230e6, - 0xfb410cc2, 0x092a8fc1, 0x1a7a7c35, 0xe811ff36, - 0x3cdb9bdd, 0xceb018de, 0xdde0eb2a, 0x2f8b6829, - 0x82f63b78, 0x709db87b, 0x63cd4b8f, 0x91a6c88c, - 0x456cac67, 0xb7072f64, 0xa457dc90, 0x563c5f93, - 0x082f63b7, 0xfa44e0b4, 0xe9141340, 0x1b7f9043, - 0xcfb5f4a8, 0x3dde77ab, 0x2e8e845f, 0xdce5075c, - 0x92a8fc17, 0x60c37f14, 0x73938ce0, 0x81f80fe3, - 0x55326b08, 0xa759e80b, 0xb4091bff, 0x466298fc, - 0x1871a4d8, 0xea1a27db, 0xf94ad42f, 0x0b21572c, - 0xdfeb33c7, 0x2d80b0c4, 0x3ed04330, 0xccbbc033, - 0xa24bb5a6, 0x502036a5, 0x4370c551, 0xb11b4652, - 0x65d122b9, 0x97baa1ba, 0x84ea524e, 0x7681d14d, - 0x2892ed69, 0xdaf96e6a, 0xc9a99d9e, 0x3bc21e9d, - 0xef087a76, 0x1d63f975, 0x0e330a81, 0xfc588982, - 0xb21572c9, 0x407ef1ca, 0x532e023e, 0xa145813d, - 0x758fe5d6, 0x87e466d5, 0x94b49521, 0x66df1622, - 0x38cc2a06, 0xcaa7a905, 0xd9f75af1, 0x2b9cd9f2, - 0xff56bd19, 0x0d3d3e1a, 0x1e6dcdee, 0xec064eed, - 0xc38d26c4, 0x31e6a5c7, 0x22b65633, 0xd0ddd530, - 0x0417b1db, 0xf67c32d8, 0xe52cc12c, 0x1747422f, - 0x49547e0b, 0xbb3ffd08, 0xa86f0efc, 0x5a048dff, - 0x8ecee914, 0x7ca56a17, 0x6ff599e3, 0x9d9e1ae0, - 0xd3d3e1ab, 0x21b862a8, 0x32e8915c, 0xc083125f, - 0x144976b4, 0xe622f5b7, 0xf5720643, 0x07198540, - 0x590ab964, 0xab613a67, 0xb831c993, 0x4a5a4a90, - 0x9e902e7b, 0x6cfbad78, 0x7fab5e8c, 0x8dc0dd8f, - 0xe330a81a, 0x115b2b19, 0x020bd8ed, 0xf0605bee, - 0x24aa3f05, 0xd6c1bc06, 0xc5914ff2, 0x37faccf1, - 0x69e9f0d5, 0x9b8273d6, 0x88d28022, 0x7ab90321, - 0xae7367ca, 0x5c18e4c9, 0x4f48173d, 0xbd23943e, - 0xf36e6f75, 0x0105ec76, 0x12551f82, 0xe03e9c81, - 0x34f4f86a, 0xc69f7b69, 0xd5cf889d, 0x27a40b9e, - 0x79b737ba, 0x8bdcb4b9, 0x988c474d, 0x6ae7c44e, - 0xbe2da0a5, 0x4c4623a6, 0x5f16d052, 0xad7d5351,}; - -#endif -#else -#define MAX_SIZE 32768 -.constants: - - /* Reduce 262144 kbits to 1024 bits */ - /* x^261120 mod p(x)` << 1, x^261184 mod p(x)` << 1 */ - .octa 0x00000000b6ca9e20000000009c37c408 - - /* x^260096 mod p(x)` << 1, x^260160 mod p(x)` << 1 */ - .octa 0x00000000350249a800000001b51df26c - - /* x^259072 mod p(x)` << 1, x^259136 mod p(x)` << 1 */ - .octa 0x00000001862dac54000000000724b9d0 - - /* x^258048 mod p(x)` << 1, x^258112 mod p(x)` << 1 */ - .octa 0x00000001d87fb48c00000001c00532fe - - /* x^257024 mod p(x)` << 1, x^257088 mod p(x)` << 1 */ - .octa 0x00000001f39b699e00000000f05a9362 - - /* x^256000 mod p(x)` << 1, x^256064 mod p(x)` << 1 */ - .octa 0x0000000101da11b400000001e1007970 - - /* x^254976 mod p(x)` << 1, x^255040 mod p(x)` << 1 */ - .octa 0x00000001cab571e000000000a57366ee - - /* x^253952 mod p(x)` << 1, x^254016 mod p(x)` << 1 */ - .octa 0x00000000c7020cfe0000000192011284 - - /* x^252928 mod p(x)` << 1, x^252992 mod p(x)` << 1 */ - .octa 0x00000000cdaed1ae0000000162716d9a - - /* x^251904 mod p(x)` << 1, x^251968 mod p(x)` << 1 */ - .octa 0x00000001e804effc00000000cd97ecde - - /* x^250880 mod p(x)` << 1, x^250944 mod p(x)` << 1 */ - .octa 0x0000000077c3ea3a0000000058812bc0 - - /* x^249856 mod p(x)` << 1, x^249920 mod p(x)` << 1 */ - .octa 0x0000000068df31b40000000088b8c12e - - /* x^248832 mod p(x)` << 1, x^248896 mod p(x)` << 1 */ - .octa 0x00000000b059b6c200000001230b234c - - /* x^247808 mod p(x)` << 1, x^247872 mod p(x)` << 1 */ - .octa 0x0000000145fb8ed800000001120b416e - - /* x^246784 mod p(x)` << 1, x^246848 mod p(x)` << 1 */ - .octa 0x00000000cbc0916800000001974aecb0 - - /* x^245760 mod p(x)` << 1, x^245824 mod p(x)` << 1 */ - .octa 0x000000005ceeedc2000000008ee3f226 - - /* x^244736 mod p(x)` << 1, x^244800 mod p(x)` << 1 */ - .octa 0x0000000047d74e8600000001089aba9a - - /* x^243712 mod p(x)` << 1, x^243776 mod p(x)` << 1 */ - .octa 0x00000001407e9e220000000065113872 - - /* x^242688 mod p(x)` << 1, x^242752 mod p(x)` << 1 */ - .octa 0x00000001da967bda000000005c07ec10 - - /* x^241664 mod p(x)` << 1, x^241728 mod p(x)` << 1 */ - .octa 0x000000006c8983680000000187590924 - - /* x^240640 mod p(x)` << 1, x^240704 mod p(x)` << 1 */ - .octa 0x00000000f2d14c9800000000e35da7c6 - - /* x^239616 mod p(x)` << 1, x^239680 mod p(x)` << 1 */ - .octa 0x00000001993c6ad4000000000415855a - - /* x^238592 mod p(x)` << 1, x^238656 mod p(x)` << 1 */ - .octa 0x000000014683d1ac0000000073617758 - - /* x^237568 mod p(x)` << 1, x^237632 mod p(x)` << 1 */ - .octa 0x00000001a7c93e6c0000000176021d28 - - /* x^236544 mod p(x)` << 1, x^236608 mod p(x)` << 1 */ - .octa 0x000000010211e90a00000001c358fd0a - - /* x^235520 mod p(x)` << 1, x^235584 mod p(x)` << 1 */ - .octa 0x000000001119403e00000001ff7a2c18 - - /* x^234496 mod p(x)` << 1, x^234560 mod p(x)` << 1 */ - .octa 0x000000001c3261aa00000000f2d9f7e4 - - /* x^233472 mod p(x)` << 1, x^233536 mod p(x)` << 1 */ - .octa 0x000000014e37a634000000016cf1f9c8 - - /* x^232448 mod p(x)` << 1, x^232512 mod p(x)` << 1 */ - .octa 0x0000000073786c0c000000010af9279a - - /* x^231424 mod p(x)` << 1, x^231488 mod p(x)` << 1 */ - .octa 0x000000011dc037f80000000004f101e8 - - /* x^230400 mod p(x)` << 1, x^230464 mod p(x)` << 1 */ - .octa 0x0000000031433dfc0000000070bcf184 - - /* x^229376 mod p(x)` << 1, x^229440 mod p(x)` << 1 */ - .octa 0x000000009cde8348000000000a8de642 - - /* x^228352 mod p(x)` << 1, x^228416 mod p(x)` << 1 */ - .octa 0x0000000038d3c2a60000000062ea130c - - /* x^227328 mod p(x)` << 1, x^227392 mod p(x)` << 1 */ - .octa 0x000000011b25f26000000001eb31cbb2 - - /* x^226304 mod p(x)` << 1, x^226368 mod p(x)` << 1 */ - .octa 0x000000001629e6f00000000170783448 - - /* x^225280 mod p(x)` << 1, x^225344 mod p(x)` << 1 */ - .octa 0x0000000160838b4c00000001a684b4c6 - - /* x^224256 mod p(x)` << 1, x^224320 mod p(x)` << 1 */ - .octa 0x000000007a44011c00000000253ca5b4 - - /* x^223232 mod p(x)` << 1, x^223296 mod p(x)` << 1 */ - .octa 0x00000000226f417a0000000057b4b1e2 - - /* x^222208 mod p(x)` << 1, x^222272 mod p(x)` << 1 */ - .octa 0x0000000045eb2eb400000000b6bd084c - - /* x^221184 mod p(x)` << 1, x^221248 mod p(x)` << 1 */ - .octa 0x000000014459d70c0000000123c2d592 - - /* x^220160 mod p(x)` << 1, x^220224 mod p(x)` << 1 */ - .octa 0x00000001d406ed8200000000159dafce - - /* x^219136 mod p(x)` << 1, x^219200 mod p(x)` << 1 */ - .octa 0x0000000160c8e1a80000000127e1a64e - - /* x^218112 mod p(x)` << 1, x^218176 mod p(x)` << 1 */ - .octa 0x0000000027ba80980000000056860754 - - /* x^217088 mod p(x)` << 1, x^217152 mod p(x)` << 1 */ - .octa 0x000000006d92d01800000001e661aae8 - - /* x^216064 mod p(x)` << 1, x^216128 mod p(x)` << 1 */ - .octa 0x000000012ed7e3f200000000f82c6166 - - /* x^215040 mod p(x)` << 1, x^215104 mod p(x)` << 1 */ - .octa 0x000000002dc8778800000000c4f9c7ae - - /* x^214016 mod p(x)` << 1, x^214080 mod p(x)` << 1 */ - .octa 0x0000000018240bb80000000074203d20 - - /* x^212992 mod p(x)` << 1, x^213056 mod p(x)` << 1 */ - .octa 0x000000001ad381580000000198173052 - - /* x^211968 mod p(x)` << 1, x^212032 mod p(x)` << 1 */ - .octa 0x00000001396b78f200000001ce8aba54 - - /* x^210944 mod p(x)` << 1, x^211008 mod p(x)` << 1 */ - .octa 0x000000011a68133400000001850d5d94 - - /* x^209920 mod p(x)` << 1, x^209984 mod p(x)` << 1 */ - .octa 0x000000012104732e00000001d609239c - - /* x^208896 mod p(x)` << 1, x^208960 mod p(x)` << 1 */ - .octa 0x00000000a140d90c000000001595f048 - - /* x^207872 mod p(x)` << 1, x^207936 mod p(x)` << 1 */ - .octa 0x00000001b7215eda0000000042ccee08 - - /* x^206848 mod p(x)` << 1, x^206912 mod p(x)` << 1 */ - .octa 0x00000001aaf1df3c000000010a389d74 - - /* x^205824 mod p(x)` << 1, x^205888 mod p(x)` << 1 */ - .octa 0x0000000029d15b8a000000012a840da6 - - /* x^204800 mod p(x)` << 1, x^204864 mod p(x)` << 1 */ - .octa 0x00000000f1a96922000000001d181c0c - - /* x^203776 mod p(x)` << 1, x^203840 mod p(x)` << 1 */ - .octa 0x00000001ac80d03c0000000068b7d1f6 - - /* x^202752 mod p(x)` << 1, x^202816 mod p(x)` << 1 */ - .octa 0x000000000f11d56a000000005b0f14fc - - /* x^201728 mod p(x)` << 1, x^201792 mod p(x)` << 1 */ - .octa 0x00000001f1c022a20000000179e9e730 - - /* x^200704 mod p(x)` << 1, x^200768 mod p(x)` << 1 */ - .octa 0x0000000173d00ae200000001ce1368d6 - - /* x^199680 mod p(x)` << 1, x^199744 mod p(x)` << 1 */ - .octa 0x00000001d4ffe4ac0000000112c3a84c - - /* x^198656 mod p(x)` << 1, x^198720 mod p(x)` << 1 */ - .octa 0x000000016edc5ae400000000de940fee - - /* x^197632 mod p(x)` << 1, x^197696 mod p(x)` << 1 */ - .octa 0x00000001f1a0214000000000fe896b7e - - /* x^196608 mod p(x)` << 1, x^196672 mod p(x)` << 1 */ - .octa 0x00000000ca0b28a000000001f797431c - - /* x^195584 mod p(x)` << 1, x^195648 mod p(x)` << 1 */ - .octa 0x00000001928e30a20000000053e989ba - - /* x^194560 mod p(x)` << 1, x^194624 mod p(x)` << 1 */ - .octa 0x0000000097b1b002000000003920cd16 - - /* x^193536 mod p(x)` << 1, x^193600 mod p(x)` << 1 */ - .octa 0x00000000b15bf90600000001e6f579b8 - - /* x^192512 mod p(x)` << 1, x^192576 mod p(x)` << 1 */ - .octa 0x00000000411c5d52000000007493cb0a - - /* x^191488 mod p(x)` << 1, x^191552 mod p(x)` << 1 */ - .octa 0x00000001c36f330000000001bdd376d8 - - /* x^190464 mod p(x)` << 1, x^190528 mod p(x)` << 1 */ - .octa 0x00000001119227e0000000016badfee6 - - /* x^189440 mod p(x)` << 1, x^189504 mod p(x)` << 1 */ - .octa 0x00000000114d47020000000071de5c58 - - /* x^188416 mod p(x)` << 1, x^188480 mod p(x)` << 1 */ - .octa 0x00000000458b5b9800000000453f317c - - /* x^187392 mod p(x)` << 1, x^187456 mod p(x)` << 1 */ - .octa 0x000000012e31fb8e0000000121675cce - - /* x^186368 mod p(x)` << 1, x^186432 mod p(x)` << 1 */ - .octa 0x000000005cf619d800000001f409ee92 - - /* x^185344 mod p(x)` << 1, x^185408 mod p(x)` << 1 */ - .octa 0x0000000063f4d8b200000000f36b9c88 - - /* x^184320 mod p(x)` << 1, x^184384 mod p(x)` << 1 */ - .octa 0x000000004138dc8a0000000036b398f4 - - /* x^183296 mod p(x)` << 1, x^183360 mod p(x)` << 1 */ - .octa 0x00000001d29ee8e000000001748f9adc - - /* x^182272 mod p(x)` << 1, x^182336 mod p(x)` << 1 */ - .octa 0x000000006a08ace800000001be94ec00 - - /* x^181248 mod p(x)` << 1, x^181312 mod p(x)` << 1 */ - .octa 0x0000000127d4201000000000b74370d6 - - /* x^180224 mod p(x)` << 1, x^180288 mod p(x)` << 1 */ - .octa 0x0000000019d76b6200000001174d0b98 - - /* x^179200 mod p(x)` << 1, x^179264 mod p(x)` << 1 */ - .octa 0x00000001b1471f6e00000000befc06a4 - - /* x^178176 mod p(x)` << 1, x^178240 mod p(x)` << 1 */ - .octa 0x00000001f64c19cc00000001ae125288 - - /* x^177152 mod p(x)` << 1, x^177216 mod p(x)` << 1 */ - .octa 0x00000000003c0ea00000000095c19b34 - - /* x^176128 mod p(x)` << 1, x^176192 mod p(x)` << 1 */ - .octa 0x000000014d73abf600000001a78496f2 - - /* x^175104 mod p(x)` << 1, x^175168 mod p(x)` << 1 */ - .octa 0x00000001620eb84400000001ac5390a0 - - /* x^174080 mod p(x)` << 1, x^174144 mod p(x)` << 1 */ - .octa 0x0000000147655048000000002a80ed6e - - /* x^173056 mod p(x)` << 1, x^173120 mod p(x)` << 1 */ - .octa 0x0000000067b5077e00000001fa9b0128 - - /* x^172032 mod p(x)` << 1, x^172096 mod p(x)` << 1 */ - .octa 0x0000000010ffe20600000001ea94929e - - /* x^171008 mod p(x)` << 1, x^171072 mod p(x)` << 1 */ - .octa 0x000000000fee8f1e0000000125f4305c - - /* x^169984 mod p(x)` << 1, x^170048 mod p(x)` << 1 */ - .octa 0x00000001da26fbae00000001471e2002 - - /* x^168960 mod p(x)` << 1, x^169024 mod p(x)` << 1 */ - .octa 0x00000001b3a8bd880000000132d2253a - - /* x^167936 mod p(x)` << 1, x^168000 mod p(x)` << 1 */ - .octa 0x00000000e8f3898e00000000f26b3592 - - /* x^166912 mod p(x)` << 1, x^166976 mod p(x)` << 1 */ - .octa 0x00000000b0d0d28c00000000bc8b67b0 - - /* x^165888 mod p(x)` << 1, x^165952 mod p(x)` << 1 */ - .octa 0x0000000030f2a798000000013a826ef2 - - /* x^164864 mod p(x)` << 1, x^164928 mod p(x)` << 1 */ - .octa 0x000000000fba10020000000081482c84 - - /* x^163840 mod p(x)` << 1, x^163904 mod p(x)` << 1 */ - .octa 0x00000000bdb9bd7200000000e77307c2 - - /* x^162816 mod p(x)` << 1, x^162880 mod p(x)` << 1 */ - .octa 0x0000000075d3bf5a00000000d4a07ec8 - - /* x^161792 mod p(x)` << 1, x^161856 mod p(x)` << 1 */ - .octa 0x00000000ef1f98a00000000017102100 - - /* x^160768 mod p(x)` << 1, x^160832 mod p(x)` << 1 */ - .octa 0x00000000689c760200000000db406486 - - /* x^159744 mod p(x)` << 1, x^159808 mod p(x)` << 1 */ - .octa 0x000000016d5fa5fe0000000192db7f88 - - /* x^158720 mod p(x)` << 1, x^158784 mod p(x)` << 1 */ - .octa 0x00000001d0d2b9ca000000018bf67b1e - - /* x^157696 mod p(x)` << 1, x^157760 mod p(x)` << 1 */ - .octa 0x0000000041e7b470000000007c09163e - - /* x^156672 mod p(x)` << 1, x^156736 mod p(x)` << 1 */ - .octa 0x00000001cbb6495e000000000adac060 - - /* x^155648 mod p(x)` << 1, x^155712 mod p(x)` << 1 */ - .octa 0x000000010052a0b000000000bd8316ae - - /* x^154624 mod p(x)` << 1, x^154688 mod p(x)` << 1 */ - .octa 0x00000001d8effb5c000000019f09ab54 - - /* x^153600 mod p(x)` << 1, x^153664 mod p(x)` << 1 */ - .octa 0x00000001d969853c0000000125155542 - - /* x^152576 mod p(x)` << 1, x^152640 mod p(x)` << 1 */ - .octa 0x00000000523ccce2000000018fdb5882 - - /* x^151552 mod p(x)` << 1, x^151616 mod p(x)` << 1 */ - .octa 0x000000001e2436bc00000000e794b3f4 - - /* x^150528 mod p(x)` << 1, x^150592 mod p(x)` << 1 */ - .octa 0x00000000ddd1c3a2000000016f9bb022 - - /* x^149504 mod p(x)` << 1, x^149568 mod p(x)` << 1 */ - .octa 0x0000000019fcfe3800000000290c9978 - - /* x^148480 mod p(x)` << 1, x^148544 mod p(x)` << 1 */ - .octa 0x00000001ce95db640000000083c0f350 - - /* x^147456 mod p(x)` << 1, x^147520 mod p(x)` << 1 */ - .octa 0x00000000af5828060000000173ea6628 - - /* x^146432 mod p(x)` << 1, x^146496 mod p(x)` << 1 */ - .octa 0x00000001006388f600000001c8b4e00a - - /* x^145408 mod p(x)` << 1, x^145472 mod p(x)` << 1 */ - .octa 0x0000000179eca00a00000000de95d6aa - - /* x^144384 mod p(x)` << 1, x^144448 mod p(x)` << 1 */ - .octa 0x0000000122410a6a000000010b7f7248 - - /* x^143360 mod p(x)` << 1, x^143424 mod p(x)` << 1 */ - .octa 0x000000004288e87c00000001326e3a06 - - /* x^142336 mod p(x)` << 1, x^142400 mod p(x)` << 1 */ - .octa 0x000000016c5490da00000000bb62c2e6 - - /* x^141312 mod p(x)` << 1, x^141376 mod p(x)` << 1 */ - .octa 0x00000000d1c71f6e0000000156a4b2c2 - - /* x^140288 mod p(x)` << 1, x^140352 mod p(x)` << 1 */ - .octa 0x00000001b4ce08a6000000011dfe763a - - /* x^139264 mod p(x)` << 1, x^139328 mod p(x)` << 1 */ - .octa 0x00000001466ba60c000000007bcca8e2 - - /* x^138240 mod p(x)` << 1, x^138304 mod p(x)` << 1 */ - .octa 0x00000001f6c488a40000000186118faa - - /* x^137216 mod p(x)` << 1, x^137280 mod p(x)` << 1 */ - .octa 0x000000013bfb06820000000111a65a88 - - /* x^136192 mod p(x)` << 1, x^136256 mod p(x)` << 1 */ - .octa 0x00000000690e9e54000000003565e1c4 - - /* x^135168 mod p(x)` << 1, x^135232 mod p(x)` << 1 */ - .octa 0x00000000281346b6000000012ed02a82 - - /* x^134144 mod p(x)` << 1, x^134208 mod p(x)` << 1 */ - .octa 0x000000015646402400000000c486ecfc - - /* x^133120 mod p(x)` << 1, x^133184 mod p(x)` << 1 */ - .octa 0x000000016063a8dc0000000001b951b2 - - /* x^132096 mod p(x)` << 1, x^132160 mod p(x)` << 1 */ - .octa 0x0000000116a663620000000048143916 - - /* x^131072 mod p(x)` << 1, x^131136 mod p(x)` << 1 */ - .octa 0x000000017e8aa4d200000001dc2ae124 - - /* x^130048 mod p(x)` << 1, x^130112 mod p(x)` << 1 */ - .octa 0x00000001728eb10c00000001416c58d6 - - /* x^129024 mod p(x)` << 1, x^129088 mod p(x)` << 1 */ - .octa 0x00000001b08fd7fa00000000a479744a - - /* x^128000 mod p(x)` << 1, x^128064 mod p(x)` << 1 */ - .octa 0x00000001092a16e80000000096ca3a26 - - /* x^126976 mod p(x)` << 1, x^127040 mod p(x)` << 1 */ - .octa 0x00000000a505637c00000000ff223d4e - - /* x^125952 mod p(x)` << 1, x^126016 mod p(x)` << 1 */ - .octa 0x00000000d94869b2000000010e84da42 - - /* x^124928 mod p(x)` << 1, x^124992 mod p(x)` << 1 */ - .octa 0x00000001c8b203ae00000001b61ba3d0 - - /* x^123904 mod p(x)` << 1, x^123968 mod p(x)` << 1 */ - .octa 0x000000005704aea000000000680f2de8 - - /* x^122880 mod p(x)` << 1, x^122944 mod p(x)` << 1 */ - .octa 0x000000012e295fa2000000008772a9a8 - - /* x^121856 mod p(x)` << 1, x^121920 mod p(x)` << 1 */ - .octa 0x000000011d0908bc0000000155f295bc - - /* x^120832 mod p(x)` << 1, x^120896 mod p(x)` << 1 */ - .octa 0x0000000193ed97ea00000000595f9282 - - /* x^119808 mod p(x)` << 1, x^119872 mod p(x)` << 1 */ - .octa 0x000000013a0f1c520000000164b1c25a - - /* x^118784 mod p(x)` << 1, x^118848 mod p(x)` << 1 */ - .octa 0x000000010c2c40c000000000fbd67c50 - - /* x^117760 mod p(x)` << 1, x^117824 mod p(x)` << 1 */ - .octa 0x00000000ff6fac3e0000000096076268 - - /* x^116736 mod p(x)` << 1, x^116800 mod p(x)` << 1 */ - .octa 0x000000017b3609c000000001d288e4cc - - /* x^115712 mod p(x)` << 1, x^115776 mod p(x)` << 1 */ - .octa 0x0000000088c8c92200000001eaac1bdc - - /* x^114688 mod p(x)` << 1, x^114752 mod p(x)` << 1 */ - .octa 0x00000001751baae600000001f1ea39e2 - - /* x^113664 mod p(x)` << 1, x^113728 mod p(x)` << 1 */ - .octa 0x000000010795297200000001eb6506fc - - /* x^112640 mod p(x)` << 1, x^112704 mod p(x)` << 1 */ - .octa 0x0000000162b00abe000000010f806ffe - - /* x^111616 mod p(x)` << 1, x^111680 mod p(x)` << 1 */ - .octa 0x000000000d7b404c000000010408481e - - /* x^110592 mod p(x)` << 1, x^110656 mod p(x)` << 1 */ - .octa 0x00000000763b13d40000000188260534 - - /* x^109568 mod p(x)` << 1, x^109632 mod p(x)` << 1 */ - .octa 0x00000000f6dc22d80000000058fc73e0 - - /* x^108544 mod p(x)` << 1, x^108608 mod p(x)` << 1 */ - .octa 0x000000007daae06000000000391c59b8 - - /* x^107520 mod p(x)` << 1, x^107584 mod p(x)` << 1 */ - .octa 0x000000013359ab7c000000018b638400 - - /* x^106496 mod p(x)` << 1, x^106560 mod p(x)` << 1 */ - .octa 0x000000008add438a000000011738f5c4 - - /* x^105472 mod p(x)` << 1, x^105536 mod p(x)` << 1 */ - .octa 0x00000001edbefdea000000008cf7c6da - - /* x^104448 mod p(x)` << 1, x^104512 mod p(x)` << 1 */ - .octa 0x000000004104e0f800000001ef97fb16 - - /* x^103424 mod p(x)` << 1, x^103488 mod p(x)` << 1 */ - .octa 0x00000000b48a82220000000102130e20 - - /* x^102400 mod p(x)` << 1, x^102464 mod p(x)` << 1 */ - .octa 0x00000001bcb4684400000000db968898 - - /* x^101376 mod p(x)` << 1, x^101440 mod p(x)` << 1 */ - .octa 0x000000013293ce0a00000000b5047b5e - - /* x^100352 mod p(x)` << 1, x^100416 mod p(x)` << 1 */ - .octa 0x00000001710d0844000000010b90fdb2 - - /* x^99328 mod p(x)` << 1, x^99392 mod p(x)` << 1 */ - .octa 0x0000000117907f6e000000004834a32e - - /* x^98304 mod p(x)` << 1, x^98368 mod p(x)` << 1 */ - .octa 0x0000000087ddf93e0000000059c8f2b0 - - /* x^97280 mod p(x)` << 1, x^97344 mod p(x)` << 1 */ - .octa 0x000000005970e9b00000000122cec508 - - /* x^96256 mod p(x)` << 1, x^96320 mod p(x)` << 1 */ - .octa 0x0000000185b2b7d0000000000a330cda - - /* x^95232 mod p(x)` << 1, x^95296 mod p(x)` << 1 */ - .octa 0x00000001dcee0efc000000014a47148c - - /* x^94208 mod p(x)` << 1, x^94272 mod p(x)` << 1 */ - .octa 0x0000000030da27220000000042c61cb8 - - /* x^93184 mod p(x)` << 1, x^93248 mod p(x)` << 1 */ - .octa 0x000000012f925a180000000012fe6960 - - /* x^92160 mod p(x)` << 1, x^92224 mod p(x)` << 1 */ - .octa 0x00000000dd2e357c00000000dbda2c20 - - /* x^91136 mod p(x)` << 1, x^91200 mod p(x)` << 1 */ - .octa 0x00000000071c80de000000011122410c - - /* x^90112 mod p(x)` << 1, x^90176 mod p(x)` << 1 */ - .octa 0x000000011513140a00000000977b2070 - - /* x^89088 mod p(x)` << 1, x^89152 mod p(x)` << 1 */ - .octa 0x00000001df876e8e000000014050438e - - /* x^88064 mod p(x)` << 1, x^88128 mod p(x)` << 1 */ - .octa 0x000000015f81d6ce0000000147c840e8 - - /* x^87040 mod p(x)` << 1, x^87104 mod p(x)` << 1 */ - .octa 0x000000019dd94dbe00000001cc7c88ce - - /* x^86016 mod p(x)` << 1, x^86080 mod p(x)` << 1 */ - .octa 0x00000001373d206e00000001476b35a4 - - /* x^84992 mod p(x)` << 1, x^85056 mod p(x)` << 1 */ - .octa 0x00000000668ccade000000013d52d508 - - /* x^83968 mod p(x)` << 1, x^84032 mod p(x)` << 1 */ - .octa 0x00000001b192d268000000008e4be32e - - /* x^82944 mod p(x)` << 1, x^83008 mod p(x)` << 1 */ - .octa 0x00000000e30f3a7800000000024120fe - - /* x^81920 mod p(x)` << 1, x^81984 mod p(x)` << 1 */ - .octa 0x000000010ef1f7bc00000000ddecddb4 - - /* x^80896 mod p(x)` << 1, x^80960 mod p(x)` << 1 */ - .octa 0x00000001f5ac738000000000d4d403bc - - /* x^79872 mod p(x)` << 1, x^79936 mod p(x)` << 1 */ - .octa 0x000000011822ea7000000001734b89aa - - /* x^78848 mod p(x)` << 1, x^78912 mod p(x)` << 1 */ - .octa 0x00000000c3a33848000000010e7a58d6 - - /* x^77824 mod p(x)` << 1, x^77888 mod p(x)` << 1 */ - .octa 0x00000001bd151c2400000001f9f04e9c - - /* x^76800 mod p(x)` << 1, x^76864 mod p(x)` << 1 */ - .octa 0x0000000056002d7600000000b692225e - - /* x^75776 mod p(x)` << 1, x^75840 mod p(x)` << 1 */ - .octa 0x000000014657c4f4000000019b8d3f3e - - /* x^74752 mod p(x)` << 1, x^74816 mod p(x)` << 1 */ - .octa 0x0000000113742d7c00000001a874f11e - - /* x^73728 mod p(x)` << 1, x^73792 mod p(x)` << 1 */ - .octa 0x000000019c5920ba000000010d5a4254 - - /* x^72704 mod p(x)` << 1, x^72768 mod p(x)` << 1 */ - .octa 0x000000005216d2d600000000bbb2f5d6 - - /* x^71680 mod p(x)` << 1, x^71744 mod p(x)` << 1 */ - .octa 0x0000000136f5ad8a0000000179cc0e36 - - /* x^70656 mod p(x)` << 1, x^70720 mod p(x)` << 1 */ - .octa 0x000000018b07beb600000001dca1da4a - - /* x^69632 mod p(x)` << 1, x^69696 mod p(x)` << 1 */ - .octa 0x00000000db1e93b000000000feb1a192 - - /* x^68608 mod p(x)` << 1, x^68672 mod p(x)` << 1 */ - .octa 0x000000000b96fa3a00000000d1eeedd6 - - /* x^67584 mod p(x)` << 1, x^67648 mod p(x)` << 1 */ - .octa 0x00000001d9968af0000000008fad9bb4 - - /* x^66560 mod p(x)` << 1, x^66624 mod p(x)` << 1 */ - .octa 0x000000000e4a77a200000001884938e4 - - /* x^65536 mod p(x)` << 1, x^65600 mod p(x)` << 1 */ - .octa 0x00000000508c2ac800000001bc2e9bc0 - - /* x^64512 mod p(x)` << 1, x^64576 mod p(x)` << 1 */ - .octa 0x0000000021572a8000000001f9658a68 - - /* x^63488 mod p(x)` << 1, x^63552 mod p(x)` << 1 */ - .octa 0x00000001b859daf2000000001b9224fc - - /* x^62464 mod p(x)` << 1, x^62528 mod p(x)` << 1 */ - .octa 0x000000016f7884740000000055b2fb84 - - /* x^61440 mod p(x)` << 1, x^61504 mod p(x)` << 1 */ - .octa 0x00000001b438810e000000018b090348 - - /* x^60416 mod p(x)` << 1, x^60480 mod p(x)` << 1 */ - .octa 0x0000000095ddc6f2000000011ccbd5ea - - /* x^59392 mod p(x)` << 1, x^59456 mod p(x)` << 1 */ - .octa 0x00000001d977c20c0000000007ae47f8 - - /* x^58368 mod p(x)` << 1, x^58432 mod p(x)` << 1 */ - .octa 0x00000000ebedb99a0000000172acbec0 - - /* x^57344 mod p(x)` << 1, x^57408 mod p(x)` << 1 */ - .octa 0x00000001df9e9e9200000001c6e3ff20 - - /* x^56320 mod p(x)` << 1, x^56384 mod p(x)` << 1 */ - .octa 0x00000001a4a3f95200000000e1b38744 - - /* x^55296 mod p(x)` << 1, x^55360 mod p(x)` << 1 */ - .octa 0x00000000e2f5122000000000791585b2 - - /* x^54272 mod p(x)` << 1, x^54336 mod p(x)` << 1 */ - .octa 0x000000004aa01f3e00000000ac53b894 - - /* x^53248 mod p(x)` << 1, x^53312 mod p(x)` << 1 */ - .octa 0x00000000b3e90a5800000001ed5f2cf4 - - /* x^52224 mod p(x)` << 1, x^52288 mod p(x)` << 1 */ - .octa 0x000000000c9ca2aa00000001df48b2e0 - - /* x^51200 mod p(x)` << 1, x^51264 mod p(x)` << 1 */ - .octa 0x000000015168231600000000049c1c62 - - /* x^50176 mod p(x)` << 1, x^50240 mod p(x)` << 1 */ - .octa 0x0000000036fce78c000000017c460c12 - - /* x^49152 mod p(x)` << 1, x^49216 mod p(x)` << 1 */ - .octa 0x000000009037dc10000000015be4da7e - - /* x^48128 mod p(x)` << 1, x^48192 mod p(x)` << 1 */ - .octa 0x00000000d3298582000000010f38f668 - - /* x^47104 mod p(x)` << 1, x^47168 mod p(x)` << 1 */ - .octa 0x00000001b42e8ad60000000039f40a00 - - /* x^46080 mod p(x)` << 1, x^46144 mod p(x)` << 1 */ - .octa 0x00000000142a983800000000bd4c10c4 - - /* x^45056 mod p(x)` << 1, x^45120 mod p(x)` << 1 */ - .octa 0x0000000109c7f1900000000042db1d98 - - /* x^44032 mod p(x)` << 1, x^44096 mod p(x)` << 1 */ - .octa 0x0000000056ff931000000001c905bae6 - - /* x^43008 mod p(x)` << 1, x^43072 mod p(x)` << 1 */ - .octa 0x00000001594513aa00000000069d40ea - - /* x^41984 mod p(x)` << 1, x^42048 mod p(x)` << 1 */ - .octa 0x00000001e3b5b1e8000000008e4fbad0 - - /* x^40960 mod p(x)` << 1, x^41024 mod p(x)` << 1 */ - .octa 0x000000011dd5fc080000000047bedd46 - - /* x^39936 mod p(x)` << 1, x^40000 mod p(x)` << 1 */ - .octa 0x00000001675f0cc20000000026396bf8 - - /* x^38912 mod p(x)` << 1, x^38976 mod p(x)` << 1 */ - .octa 0x00000000d1c8dd4400000000379beb92 - - /* x^37888 mod p(x)` << 1, x^37952 mod p(x)` << 1 */ - .octa 0x0000000115ebd3d8000000000abae54a - - /* x^36864 mod p(x)` << 1, x^36928 mod p(x)` << 1 */ - .octa 0x00000001ecbd0dac0000000007e6a128 - - /* x^35840 mod p(x)` << 1, x^35904 mod p(x)` << 1 */ - .octa 0x00000000cdf67af2000000000ade29d2 - - /* x^34816 mod p(x)` << 1, x^34880 mod p(x)` << 1 */ - .octa 0x000000004c01ff4c00000000f974c45c - - /* x^33792 mod p(x)` << 1, x^33856 mod p(x)` << 1 */ - .octa 0x00000000f2d8657e00000000e77ac60a - - /* x^32768 mod p(x)` << 1, x^32832 mod p(x)` << 1 */ - .octa 0x000000006bae74c40000000145895816 - - /* x^31744 mod p(x)` << 1, x^31808 mod p(x)` << 1 */ - .octa 0x0000000152af8aa00000000038e362be - - /* x^30720 mod p(x)` << 1, x^30784 mod p(x)` << 1 */ - .octa 0x0000000004663802000000007f991a64 - - /* x^29696 mod p(x)` << 1, x^29760 mod p(x)` << 1 */ - .octa 0x00000001ab2f5afc00000000fa366d3a - - /* x^28672 mod p(x)` << 1, x^28736 mod p(x)` << 1 */ - .octa 0x0000000074a4ebd400000001a2bb34f0 - - /* x^27648 mod p(x)` << 1, x^27712 mod p(x)` << 1 */ - .octa 0x00000001d7ab3a4c0000000028a9981e - - /* x^26624 mod p(x)` << 1, x^26688 mod p(x)` << 1 */ - .octa 0x00000001a8da60c600000001dbc672be - - /* x^25600 mod p(x)` << 1, x^25664 mod p(x)` << 1 */ - .octa 0x000000013cf6382000000000b04d77f6 - - /* x^24576 mod p(x)` << 1, x^24640 mod p(x)` << 1 */ - .octa 0x00000000bec12e1e0000000124400d96 - - /* x^23552 mod p(x)` << 1, x^23616 mod p(x)` << 1 */ - .octa 0x00000001c6368010000000014ca4b414 - - /* x^22528 mod p(x)` << 1, x^22592 mod p(x)` << 1 */ - .octa 0x00000001e6e78758000000012fe2c938 - - /* x^21504 mod p(x)` << 1, x^21568 mod p(x)` << 1 */ - .octa 0x000000008d7f2b3c00000001faed01e6 - - /* x^20480 mod p(x)` << 1, x^20544 mod p(x)` << 1 */ - .octa 0x000000016b4a156e000000007e80ecfe - - /* x^19456 mod p(x)` << 1, x^19520 mod p(x)` << 1 */ - .octa 0x00000001c63cfeb60000000098daee94 - - /* x^18432 mod p(x)` << 1, x^18496 mod p(x)` << 1 */ - .octa 0x000000015f902670000000010a04edea - - /* x^17408 mod p(x)` << 1, x^17472 mod p(x)` << 1 */ - .octa 0x00000001cd5de11e00000001c00b4524 - - /* x^16384 mod p(x)` << 1, x^16448 mod p(x)` << 1 */ - .octa 0x000000001acaec540000000170296550 - - /* x^15360 mod p(x)` << 1, x^15424 mod p(x)` << 1 */ - .octa 0x000000002bd0ca780000000181afaa48 - - /* x^14336 mod p(x)` << 1, x^14400 mod p(x)` << 1 */ - .octa 0x0000000032d63d5c0000000185a31ffa - - /* x^13312 mod p(x)` << 1, x^13376 mod p(x)` << 1 */ - .octa 0x000000001c6d4e4c000000002469f608 - - /* x^12288 mod p(x)` << 1, x^12352 mod p(x)` << 1 */ - .octa 0x0000000106a60b92000000006980102a - - /* x^11264 mod p(x)` << 1, x^11328 mod p(x)` << 1 */ - .octa 0x00000000d3855e120000000111ea9ca8 - - /* x^10240 mod p(x)` << 1, x^10304 mod p(x)` << 1 */ - .octa 0x00000000e312563600000001bd1d29ce - - /* x^9216 mod p(x)` << 1, x^9280 mod p(x)` << 1 */ - .octa 0x000000009e8f7ea400000001b34b9580 - - /* x^8192 mod p(x)` << 1, x^8256 mod p(x)` << 1 */ - .octa 0x00000001c82e562c000000003076054e - - /* x^7168 mod p(x)` << 1, x^7232 mod p(x)` << 1 */ - .octa 0x00000000ca9f09ce000000012a608ea4 - - /* x^6144 mod p(x)` << 1, x^6208 mod p(x)` << 1 */ - .octa 0x00000000c63764e600000000784d05fe - - /* x^5120 mod p(x)` << 1, x^5184 mod p(x)` << 1 */ - .octa 0x0000000168d2e49e000000016ef0d82a - - /* x^4096 mod p(x)` << 1, x^4160 mod p(x)` << 1 */ - .octa 0x00000000e986c1480000000075bda454 - - /* x^3072 mod p(x)` << 1, x^3136 mod p(x)` << 1 */ - .octa 0x00000000cfb65894000000003dc0a1c4 - - /* x^2048 mod p(x)` << 1, x^2112 mod p(x)` << 1 */ - .octa 0x0000000111cadee400000000e9a5d8be - - /* x^1024 mod p(x)` << 1, x^1088 mod p(x)` << 1 */ - .octa 0x0000000171fb63ce00000001609bc4b4 - -.short_constants: - - /* Reduce final 1024-2048 bits to 64 bits, shifting 32 bits to include the trailing 32 bits of zeros */ - /* x^1952 mod p(x)`, x^1984 mod p(x)`, x^2016 mod p(x)`, x^2048 mod p(x)` */ - .octa 0x7fec2963e5bf80485cf015c388e56f72 - - /* x^1824 mod p(x)`, x^1856 mod p(x)`, x^1888 mod p(x)`, x^1920 mod p(x)` */ - .octa 0x38e888d4844752a9963a18920246e2e6 - - /* x^1696 mod p(x)`, x^1728 mod p(x)`, x^1760 mod p(x)`, x^1792 mod p(x)` */ - .octa 0x42316c00730206ad419a441956993a31 - - /* x^1568 mod p(x)`, x^1600 mod p(x)`, x^1632 mod p(x)`, x^1664 mod p(x)` */ - .octa 0x543d5c543e65ddf9924752ba2b830011 - - /* x^1440 mod p(x)`, x^1472 mod p(x)`, x^1504 mod p(x)`, x^1536 mod p(x)` */ - .octa 0x78e87aaf56767c9255bd7f9518e4a304 - - /* x^1312 mod p(x)`, x^1344 mod p(x)`, x^1376 mod p(x)`, x^1408 mod p(x)` */ - .octa 0x8f68fcec1903da7f6d76739fe0553f1e - - /* x^1184 mod p(x)`, x^1216 mod p(x)`, x^1248 mod p(x)`, x^1280 mod p(x)` */ - .octa 0x3f4840246791d588c133722b1fe0b5c3 - - /* x^1056 mod p(x)`, x^1088 mod p(x)`, x^1120 mod p(x)`, x^1152 mod p(x)` */ - .octa 0x34c96751b04de25a64b67ee0e55ef1f3 - - /* x^928 mod p(x)`, x^960 mod p(x)`, x^992 mod p(x)`, x^1024 mod p(x)` */ - .octa 0x156c8e180b4a395b069db049b8fdb1e7 - - /* x^800 mod p(x)`, x^832 mod p(x)`, x^864 mod p(x)`, x^896 mod p(x)` */ - .octa 0xe0b99ccbe661f7bea11bfaf3c9e90b9e - - /* x^672 mod p(x)`, x^704 mod p(x)`, x^736 mod p(x)`, x^768 mod p(x)` */ - .octa 0x041d37768cd75659817cdc5119b29a35 - - /* x^544 mod p(x)`, x^576 mod p(x)`, x^608 mod p(x)`, x^640 mod p(x)` */ - .octa 0x3a0777818cfaa9651ce9d94b36c41f1c - - /* x^416 mod p(x)`, x^448 mod p(x)`, x^480 mod p(x)`, x^512 mod p(x)` */ - .octa 0x0e148e8252377a554f256efcb82be955 - - /* x^288 mod p(x)`, x^320 mod p(x)`, x^352 mod p(x)`, x^384 mod p(x)` */ - .octa 0x9c25531d19e65ddeec1631edb2dea967 - - /* x^160 mod p(x)`, x^192 mod p(x)`, x^224 mod p(x)`, x^256 mod p(x)` */ - .octa 0x790606ff9957c0a65d27e147510ac59a - - /* x^32 mod p(x)`, x^64 mod p(x)`, x^96 mod p(x)`, x^128 mod p(x)` */ - .octa 0x82f63b786ea2d55ca66805eb18b8ea18 - - -.barrett_constants: - /* 33 bit reflected Barrett constant m - (4^32)/n */ - .octa 0x000000000000000000000000dea713f1 /* x^64 div p(x)` */ - /* 33 bit reflected Barrett constant n */ - .octa 0x00000000000000000000000105ec76f1 -#endif diff --git a/src/third_party/wiredtiger/src/support/power8/crc32_wrapper.c b/src/third_party/wiredtiger/src/support/power8/crc32_wrapper.c deleted file mode 100644 index 34ac4150338..00000000000 --- a/src/third_party/wiredtiger/src/support/power8/crc32_wrapper.c +++ /dev/null @@ -1,66 +0,0 @@ -#if defined(__powerpc64__) -#define CRC_TABLE -#include "crc32_constants.h" - -#define VMX_ALIGN 16 -#define VMX_ALIGN_MASK (VMX_ALIGN-1) - -#ifdef REFLECT -static unsigned int crc32_align(unsigned int crc, unsigned char *p, - unsigned long len) -{ - while (len--) - crc = crc_table[(crc ^ *p++) & 0xff] ^ (crc >> 8); - return crc; -} -#else -static unsigned int crc32_align(unsigned int crc, unsigned char *p, - unsigned long len) -{ - while (len--) - crc = crc_table[((crc >> 24) ^ *p++) & 0xff] ^ (crc << 8); - return crc; -} -#endif - -unsigned int __crc32_vpmsum(unsigned int crc, unsigned char *p, - unsigned long len); - -unsigned int crc32_vpmsum(unsigned int crc, unsigned char *p, - unsigned long len) -{ - unsigned int prealign; - unsigned int tail; - -#ifdef CRC_XOR - crc ^= 0xffffffff; -#endif - - if (len < VMX_ALIGN + VMX_ALIGN_MASK) { - crc = crc32_align(crc, p, len); - goto out; - } - - if ((unsigned long)p & VMX_ALIGN_MASK) { - prealign = VMX_ALIGN - ((unsigned long)p & VMX_ALIGN_MASK); - crc = crc32_align(crc, p, prealign); - len -= prealign; - p += prealign; - } - - crc = __crc32_vpmsum(crc, p, len & ~VMX_ALIGN_MASK); - - tail = len & VMX_ALIGN_MASK; - if (tail) { - p += len & ~VMX_ALIGN_MASK; - crc = crc32_align(crc, p, tail); - } - -out: -#ifdef CRC_XOR - crc ^= 0xffffffff; -#endif - - return crc; -} -#endif diff --git a/src/third_party/wiredtiger/src/support/power8/ppc-opcode.h b/src/third_party/wiredtiger/src/support/power8/ppc-opcode.h deleted file mode 100644 index b63feea60a0..00000000000 --- a/src/third_party/wiredtiger/src/support/power8/ppc-opcode.h +++ /dev/null @@ -1,23 +0,0 @@ -#ifndef __OPCODES_H -#define __OPCODES_H - -#define __PPC_RA(a) (((a) & 0x1f) << 16) -#define __PPC_RB(b) (((b) & 0x1f) << 11) -#define __PPC_XA(a) ((((a) & 0x1f) << 16) | (((a) & 0x20) >> 3)) -#define __PPC_XB(b) ((((b) & 0x1f) << 11) | (((b) & 0x20) >> 4)) -#define __PPC_XS(s) ((((s) & 0x1f) << 21) | (((s) & 0x20) >> 5)) -#define __PPC_XT(s) __PPC_XS(s) -#define VSX_XX3(t, a, b) (__PPC_XT(t) | __PPC_XA(a) | __PPC_XB(b)) -#define VSX_XX1(s, a, b) (__PPC_XS(s) | __PPC_RA(a) | __PPC_RB(b)) - -#define PPC_INST_VPMSUMW 0x10000488 -#define PPC_INST_VPMSUMD 0x100004c8 -#define PPC_INST_MFVSRD 0x7c000066 -#define PPC_INST_MTVSRD 0x7c000166 - -#define VPMSUMW(t, a, b) .long PPC_INST_VPMSUMW | VSX_XX3((t), a, b) -#define VPMSUMD(t, a, b) .long PPC_INST_VPMSUMD | VSX_XX3((t), a, b) -#define MFVRD(a, t) .long PPC_INST_MFVSRD | VSX_XX1((t)+32, a, 0) -#define MTVRD(t, a) .long PPC_INST_MTVSRD | VSX_XX1((t)+32, a, 0) - -#endif diff --git a/src/third_party/wiredtiger/src/txn/txn.c b/src/third_party/wiredtiger/src/txn/txn.c index dd4384d9a9a..d6d5760ea15 100644 --- a/src/third_party/wiredtiger/src/txn/txn.c +++ b/src/third_party/wiredtiger/src/txn/txn.c @@ -124,6 +124,7 @@ __wt_txn_get_snapshot(WT_SESSION_IMPL *session) txn = &session->txn; txn_global = &conn->txn_global; txn_state = WT_SESSION_TXN_STATE(session); + n = 0; /* * Spin waiting for the lock: the sleeps in our blocking readlock @@ -137,20 +138,26 @@ __wt_txn_get_snapshot(WT_SESSION_IMPL *session) current_id = snap_min = txn_global->current; prev_oldest_id = txn_global->oldest_id; + /* + * Include the checkpoint transaction, if one is running: we should + * ignore any uncommitted changes the checkpoint has written to the + * metadata. We don't have to keep the checkpoint's changes pinned so + * don't including it in the published snap_min. + */ + if ((id = txn_global->checkpoint_txnid) != WT_TXN_NONE) + txn->snapshot[n++] = id; + /* For pure read-only workloads, avoid scanning. */ if (prev_oldest_id == current_id) { txn_state->snap_min = current_id; - __txn_sort_snapshot(session, 0, current_id); - /* Check that the oldest ID has not moved in the meantime. */ WT_ASSERT(session, prev_oldest_id == txn_global->oldest_id); - WT_RET(__wt_readunlock(session, txn_global->scan_rwlock)); - return (0); + goto done; } /* Walk the array of concurrent transactions. */ WT_ORDERED_READ(session_cnt, conn->session_cnt); - for (i = n = 0, s = txn_global->states; i < session_cnt; i++, s++) { + for (i = 0, s = txn_global->states; i < session_cnt; i++, s++) { /* * Build our snapshot of any concurrent transaction IDs. * @@ -178,8 +185,7 @@ __wt_txn_get_snapshot(WT_SESSION_IMPL *session) WT_ASSERT(session, prev_oldest_id == txn_global->oldest_id); txn_state->snap_min = snap_min; - WT_RET(__wt_readunlock(session, txn_global->scan_rwlock)); - +done: WT_RET(__wt_readunlock(session, txn_global->scan_rwlock)); __txn_sort_snapshot(session, n, current_id); return (0); } @@ -433,18 +439,22 @@ __wt_txn_release(WT_SESSION_IMPL *session) WT_TXN_STATE *txn_state; txn = &session->txn; - WT_ASSERT(session, txn->mod_count == 0); - txn->notify = NULL; - txn_global = &S2C(session)->txn_global; txn_state = WT_SESSION_TXN_STATE(session); + WT_ASSERT(session, txn->mod_count == 0); + txn->notify = NULL; + /* Clear the transaction's ID from the global table. */ if (WT_SESSION_IS_CHECKPOINT(session)) { WT_ASSERT(session, txn_state->id == WT_TXN_NONE); - txn->id = WT_TXN_NONE; + txn->id = txn_global->checkpoint_txnid = WT_TXN_NONE; - /* Clear the global checkpoint transaction IDs. */ + /* + * Be extra careful to cleanup everything for checkpoints: once + * the global checkpoint ID is cleared, we can no longer tell + * if this session is doing a checkpoint. + */ txn_global->checkpoint_id = 0; txn_global->checkpoint_pinned = WT_TXN_NONE; } else if (F_ISSET(txn, WT_TXN_HAS_ID)) { @@ -540,6 +550,16 @@ __wt_txn_commit(WT_SESSION_IMPL *session, const char *cfg[]) WT_TRET(txn->notify->notify(txn->notify, (WT_SESSION *)session, txn->id, 1)); + /* + * We are about to release the snapshot: copy values into any + * positioned cursors so they don't point to updates that could be + * freed once we don't have a snapshot. + */ + if (session->ncursors > 0) { + WT_DIAGNOSTIC_YIELD; + WT_RET(__wt_session_copy_values(session)); + } + /* If we are logging, write a commit log record. */ if (ret == 0 && txn->mod_count > 0 && FLD_ISSET(conn->log_flags, WT_CONN_LOG_ENABLED) && @@ -569,14 +589,6 @@ __wt_txn_commit(WT_SESSION_IMPL *session, const char *cfg[]) __wt_txn_op_free(session, op); txn->mod_count = 0; - /* - * We are about to release the snapshot: copy values into any - * positioned cursors so they don't point to updates that could be - * freed once we don't have a transaction ID pinned. - */ - if (session->ncursors > 0) - WT_RET(__wt_session_copy_values(session)); - __wt_txn_release(session); return (0); } diff --git a/src/third_party/wiredtiger/src/txn/txn_ckpt.c b/src/third_party/wiredtiger/src/txn/txn_ckpt.c index 51d26b9aed6..82c1fe7bdfe 100644 --- a/src/third_party/wiredtiger/src/txn/txn_ckpt.c +++ b/src/third_party/wiredtiger/src/txn/txn_ckpt.c @@ -475,21 +475,22 @@ __txn_checkpoint(WT_SESSION_IMPL *session, const char *cfg[]) WT_ERR(__wt_txn_id_check(session)); /* - * Save the checkpoint session ID. We never do checkpoints in the - * default session (with id zero). + * Save the checkpoint session ID. + * + * We never do checkpoints in the default session (with id zero). */ WT_ASSERT(session, session->id != 0 && txn_global->checkpoint_id == 0); txn_global->checkpoint_id = session->id; - txn_global->checkpoint_pinned = - WT_MIN(txn_state->id, txn_state->snap_min); - /* - * We're about to clear the checkpoint transaction from the global - * state table so the oldest ID can move forward. Make sure everything - * we've done above is scheduled. + * Remove the checkpoint transaction from the global table. + * + * This allows ordinary visibility checks to move forward because + * checkpoints often take a long time and only write to the metadata. */ - WT_FULL_BARRIER(); + WT_ERR(__wt_writelock(session, txn_global->scan_rwlock)); + txn_global->checkpoint_txnid = txn->id; + txn_global->checkpoint_pinned = WT_MIN(txn->id, txn->snap_min); /* * Sanity check that the oldest ID hasn't moved on before we have @@ -507,6 +508,7 @@ __txn_checkpoint(WT_SESSION_IMPL *session, const char *cfg[]) * details). */ txn_state->id = txn_state->snap_min = WT_TXN_NONE; + WT_ERR(__wt_writeunlock(session, txn_global->scan_rwlock)); /* Tell logging that we have started a database checkpoint. */ if (full && logging) diff --git a/src/third_party/wiredtiger/test/suite/test_reconfig01.py b/src/third_party/wiredtiger/test/suite/test_reconfig01.py index 876de1fe5af..419ff876dc2 100644 --- a/src/third_party/wiredtiger/test/suite/test_reconfig01.py +++ b/src/third_party/wiredtiger/test/suite/test_reconfig01.py @@ -95,19 +95,24 @@ class test_reconfig01(wttest.WiredTigerTestCase): self.conn.reconfigure("checkpoint=(wait=0,name=hi)") self.conn.reconfigure("checkpoint=(wait=5,name=hi)") - def test_reconfig_stat_log(self): + # Statistics logging: reconfigure the things we can reconfigure. + def test_reconfig_statistics_log_ok(self): self.conn.reconfigure("statistics=[all],statistics_log=(wait=0)") self.conn.reconfigure("statistics_log=(wait=0)") - self.conn.reconfigure("statistics_log=(wait=2)") + self.conn.reconfigure("statistics_log=(wait=2,json=true)") + self.conn.reconfigure("statistics_log=(wait=0)") + self.conn.reconfigure("statistics_log=(wait=2,on_close=true)") self.conn.reconfigure("statistics_log=(wait=0)") self.conn.reconfigure("statistics_log=(wait=2,sources=[lsm:])") self.conn.reconfigure("statistics_log=(wait=0)") self.conn.reconfigure("statistics_log=(wait=2,timestamp=\"t%b %d\")") self.conn.reconfigure("statistics_log=(wait=0)") - self.conn.reconfigure("statistics_log=(wait=2,path=\"wts.%d.%H\")") - self.conn.reconfigure("statistics_log=(wait=0)") - self.conn.reconfigure( - "statistics_log=(wait=2,sources=[lsm:],timestamp=\"%b\")") + + # Statistics logging: reconfigure the things we can't reconfigure. + def test_reconfig_statistics_log_fail(self): + msg = '/unknown configuration key/' + self.assertRaisesWithMessage(wiredtiger.WiredTigerError, + lambda: self.conn.reconfigure("log=(path=foo)"), msg) def test_file_manager(self): self.conn.reconfigure("file_manager=(close_scan_interval=3)") diff --git a/src/third_party/wiredtiger/test/suite/test_reconfig02.py b/src/third_party/wiredtiger/test/suite/test_reconfig02.py index 85a9ceb2a34..9d9ac220aa7 100644 --- a/src/third_party/wiredtiger/test/suite/test_reconfig02.py +++ b/src/third_party/wiredtiger/test/suite/test_reconfig02.py @@ -41,24 +41,29 @@ class test_reconfig02(wttest.WiredTigerTestCase): self.conn_config = self.init_config return wttest.WiredTigerTestCase.setUpConnectionOpen(self, dir) - # Call reconfigure for zero filling a file. There is nothing - # we can actually look for to confirm it did anything. - # Also changing the log file size is a no-op, but should not fail. + # Logging: reconfigure the things we can reconfigure. def test_reconfig02_simple(self): + self.conn.reconfigure("log=(archive=false)") + self.conn.reconfigure("log=(prealloc=false)") + self.conn.reconfigure("log=(zero_fill=false)") + + self.conn.reconfigure("log=(archive=true)") + self.conn.reconfigure("log=(prealloc=true)") self.conn.reconfigure("log=(zero_fill=true)") - self.conn.reconfigure("log=(file_max=1MB)") - # Test that we get an error if we try to turn logging off. + # Logging: reconfigure the things we can't reconfigure. def test_reconfig02_disable(self): - msg = 'Invalid argument' - gotException = False - try: - self.conn.reconfigure("log=(enabled=false)") - except wiredtiger.WiredTigerError as e: - gotException = True - self.pr('got exception: ' + str(e)) - self.assertTrue(str(e).find(msg) >= 0) - self.assertTrue(gotException) + msg = '/unknown configuration key/' + self.assertRaisesWithMessage(wiredtiger.WiredTigerError, + lambda: self.conn.reconfigure("log=(enabled=true)"), msg) + self.assertRaisesWithMessage(wiredtiger.WiredTigerError, + lambda: self.conn.reconfigure("log=(compressor=foo)"), msg) + self.assertRaisesWithMessage(wiredtiger.WiredTigerError, + lambda: self.conn.reconfigure("log=(file_max=1MB)"), msg) + self.assertRaisesWithMessage(wiredtiger.WiredTigerError, + lambda: self.conn.reconfigure("log=(path=foo)"), msg) + self.assertRaisesWithMessage(wiredtiger.WiredTigerError, + lambda: self.conn.reconfigure("log=(recovery=true)"), msg) # Logging starts on, but prealloc is off. Verify it is off. # Reconfigure it on and run again, making sure that log files diff --git a/src/third_party/wiredtiger/test/suite/test_stat_log01.py b/src/third_party/wiredtiger/test/suite/test_stat_log01.py index f6033d940c5..65ce80dfe7d 100644 --- a/src/third_party/wiredtiger/test/suite/test_stat_log01.py +++ b/src/third_party/wiredtiger/test/suite/test_stat_log01.py @@ -51,9 +51,10 @@ class test_stat_log01(wttest.WiredTigerTestCase): None, "create,statistics=(fast),statistics_log=(wait=1)") # Wait for the default interval, to ensure stats have been written. time.sleep(2) - self.check_stats_file("WiredTigerStat") + self.check_stats_file(".") def test_stats_log_name(self): + os.mkdir("foo") self.conn = self.wiredtiger_open( None, "create,statistics=(fast),statistics_log=(wait=1,path=foo)") # Wait for the default interval, to ensure stats have been written. @@ -66,21 +67,18 @@ class test_stat_log01(wttest.WiredTigerTestCase): # Wait for the default interval, to ensure stats have been written. time.sleep(2) self.close_conn() - self.check_stats_file("WiredTigerStat") + self.check_stats_file(".") def test_stats_log_on_close(self): self.conn = self.wiredtiger_open(None, "create,statistics=(fast),statistics_log=(on_close=true)") # Close the connection to ensure the statistics get generated. self.close_conn() - self.check_stats_file("WiredTigerStat") + self.check_stats_file(".") - def check_stats_file(self, filename): - if filename == "WiredTigerStat": - files = glob.glob(filename + '.[0-9]*') - self.assertTrue(files) - else: - self.assertTrue(os.path.isfile(filename)) + def check_stats_file(self, dir): + files = glob.glob(dir + '/' + 'WiredTigerStat.[0-9]*') + self.assertTrue(files) if __name__ == '__main__': wttest.run() diff --git a/src/third_party/wiredtiger/test/utility/test_util.i b/src/third_party/wiredtiger/test/utility/test_util.i deleted file mode 100644 index 43982d9e4a1..00000000000 --- a/src/third_party/wiredtiger/test/utility/test_util.i +++ /dev/null @@ -1,163 +0,0 @@ -/*- - * Public Domain 2014-2016 MongoDB, Inc. - * Public Domain 2008-2014 WiredTiger, Inc. - * - * This is free and unencumbered software released into the public domain. - * - * Anyone is free to copy, modify, publish, use, compile, sell, or - * distribute this software, either in source code form or as a compiled - * binary, for any purpose, commercial or non-commercial, and by any - * means. - * - * In jurisdictions that recognize copyright laws, the author or authors - * of this software dedicate any and all copyright interest in the - * software to the public domain. We make this dedication for the benefit - * of the public at large and to the detriment of our heirs and - * successors. We intend this dedication to be an overt act of - * relinquishment in perpetuity of all present and future rights to this - * software under copyright law. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, - * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF - * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. - * IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR - * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, - * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR - * OTHER DEALINGS IN THE SOFTWARE. - */ -#include "wt_internal.h" /* For __wt_XXX */ - -#ifdef _WIN32 -#include "windows_shim.h" -#endif - -#ifdef _WIN32 - #define DIR_DELIM '\\' - #define RM_COMMAND "rd /s /q " -#else - #define DIR_DELIM '/' - #define RM_COMMAND "rm -rf " -#endif - -#define DEFAULT_DIR "WT_TEST" -#define MKDIR_COMMAND "mkdir " - -/* Allow tests to add their own death handling. */ -extern void (*custom_die)(void); - -static void testutil_die(int, const char *, ...) -#if defined(__GNUC__) -__attribute__((__noreturn__)) -#endif -; - -/* - * die -- - * Report an error and quit. - */ -static void -testutil_die(int e, const char *fmt, ...) -{ - va_list ap; - - /* Allow test programs to cleanup on fatal error. */ - if (custom_die != NULL) - (*custom_die)(); - - va_start(ap, fmt); - vfprintf(stderr, fmt, ap); - va_end(ap); - if (e != 0) - fprintf(stderr, ": %s", wiredtiger_strerror(e)); - fprintf(stderr, "\n"); - - exit(EXIT_FAILURE); -} - -/* - * testutil_check -- - * Complain and quit if a function call fails. - */ -#define testutil_check(call) do { \ - int __r; \ - if ((__r = (call)) != 0) \ - testutil_die(__r, "%s/%d: %s", __func__, __LINE__, #call);\ -} while (0) - -/* - * testutil_checkfmt -- - * Complain and quit if a function call fails, with additional arguments. - */ -#define testutil_checkfmt(call, fmt, ...) do { \ - int __r; \ - if ((__r = (call)) != 0) \ - testutil_die(__r, "%s/%d: %s: " fmt, \ - __func__, __LINE__, #call, __VA_ARGS__); \ -} while (0) - -/* - * testutil_work_dir_from_path -- - * Takes a buffer, its size and the intended work directory. - * Creates the full intended work directory in buffer. - */ -static inline void -testutil_work_dir_from_path(char *buffer, size_t len, const char *dir) -{ - /* If no directory is provided, use the default. */ - if (dir == NULL) - dir = DEFAULT_DIR; - - if (len < strlen(dir) + 1) - testutil_die(ENOMEM, - "Not enough memory in buffer for directory %s", dir); - - strcpy(buffer, dir); -} - -/* - * testutil_clean_work_dir -- - * Remove the work directory. - */ -static inline void -testutil_clean_work_dir(char *dir) -{ - size_t len; - int ret; - char *buf; - - /* Additional bytes for the Windows rd command. */ - len = strlen(dir) + strlen(RM_COMMAND) + 1; - if ((buf = malloc(len)) == NULL) - testutil_die(ENOMEM, "Failed to allocate memory"); - - snprintf(buf, len, "%s%s", RM_COMMAND, dir); - - if ((ret = system(buf)) != 0 && ret != ENOENT) - testutil_die(ret, "%s", buf); - free(buf); -} - -/* - * testutil_make_work_dir -- - * Delete the existing work directory, then create a new one. - */ -static inline void -testutil_make_work_dir(char *dir) -{ - size_t len; - int ret; - char *buf; - - testutil_clean_work_dir(dir); - - /* Additional bytes for the mkdir command */ - len = strlen(dir) + strlen(MKDIR_COMMAND) + 1; - if ((buf = malloc(len)) == NULL) - testutil_die(ENOMEM, "Failed to allocate memory"); - - /* mkdir shares syntax between Windows and Linux */ - snprintf(buf, len, "%s%s", MKDIR_COMMAND, dir); - if ((ret = system(buf)) != 0) - testutil_die(ret, "%s", buf); - free(buf); -} |