/*! @page upgrading Upgrading WiredTiger applications @section version_252 Upgrading to Version 2.5.2 There are no special upgrade steps required. @section version_251 Upgrading to Version 2.5.1
Statistics cursors on data sources
Opening a statistics cursor on a data source without a \c checkpoint name no longer aggregates statistics for all open checkpoints on that data source. The returned statistics will only be for the in-memory version of the data source. For applications that read and write from ordinary tables (without specifying a \c checkpoint), there will be no change. Applications that opened cursors on checkpoints and relied on their statistics being aggregated into a single statistics cursor will need to open statistics cursors on each checkpoint.
@section version_250 Upgrading to Version 2.5.0
WT_STAT_CONN_LOG_BYTES_USER renamed WT_STAT_CONN_LOG_BYTES_PAYLOAD
The statistic for the number of bytes written to the log minus the overhead of log record headers and padding was renamed to make the intent clearer. Any applications using the old name will need to be updated.
"none" configuration for collators, compressors and extractors
Collators, compressors and extractors can now be disabled with an explicit \c "none" value as an alternative to using an empty string. Any applications using the name \c "none" for a collator, compressor or extractor will need to be updated.
maximum keys and value sizes
The WT_SESSION::create \c internal_item_max and \c leaf_item_max configuration strings are now deprecated in favor of the \c internal_key_max, \c leaf_key_max, and \c leaf_value_max configuration strings. See @ref tune_page_sizes for more information.
@section version_241 Upgrading to Version 2.4.1
WT_DEADLOCK renamed
The \c WT_DEADLOCK error return has been deprecated in favor of ::WT_ROLLBACK to clarify that WT_SESSION::rollback_transaction should be called; no program changes are required.
Statistics keys changed
The names of WiredTiger statistics have been updated to be more consistently named, and simpler to categorize. Any application that was parsing the strings output by statistics will need to be updated.
@section version_240 Upgrading to Version 2.4.0
Default configuration file changes
WiredTiger creates a configuration file when a database is first created. This release adds a version number to that configuration file, and functionality to automatically translate configuration settings between versions moving forward.
Cursors no longer reset on transaction begin or commit
In previous versions of WiredTiger, all cursors in a session were reset at transaction boundaries (WT_SESSION::begin_transaction, WT_SESSION::commit_transaction and WT_SESSION::rollback_transaction). Now they are only reset by WT_SESSION::rollback_transaction. This change means that a cursor can be used to iterate through a table and perform transactional updates based on the visited records without the cursor losing its position. Applications relying on this behavior (for example, to avoid pinning the underlying cursor resources), should reset cursors explicitly when the position is no longer required.
WT_COLLATOR interface changes
Add a new WT_COLLATOR::customize callback that WiredTiger will call (if set) for each data source configured to use the collator. Applications using the existing WT_COLLATOR interface that do not require the new functionality should set this callback to \c NULL.
::wiredtiger_open lsm_merge option changed
The global lsm_merge configuration setting has been moved into the \c lsm_manager option group. To disable merges in all LSM trees pass \c lsm_manger=(merge=false) to ::wiredtiger_open.
Java include path search changes
We have updated the configure logic used to search for Java Native Interface files. This fixes problems building WiredTiger's Java API, particularly on OS X, but may introduce problems if your build procedure relied on the old search behavior.
Default mutex implementation changed
The default mutex implementation has been changed from adaptive pthread mutexes to non-adaptive pthread mutexes. Installations can explicitly select adaptive pthread mutexes by specifying \c --with-spinlock=pthread_adaptive at configuration time.
LSM merge threads option change
The WT_SESSION::create \c lsm=(merge_threads) configuration option has been replaced by the W::wiredtiger_open \c lsm_manager=(worker_thread_max) option. The new version specifies a set of LSM threads that are shared across all LSM trees in a database, the older configuration was per LSM table.
@section version_231 Upgrading to Version 2.3.1
::wiredtiger_open eviction_workers configuration changed
The \c eviction_workers configuration setting has been replaced by \c eviction=(threads_min) and \c eviction=(threads_max) settings. There is also a semantic change because \c eviction_workers used to configure additional threads whereas the new settings configure the total number of threads involved with eviction.
@section version_230 Upgrading to Version 2.3.0 There are no special upgrade steps required. @section version_221 Upgrading to Version 2.2.1
::wiredtiger_open configuration parsing order changed
In the 2.2.1 release, the order that configuration strings are parsed and override earlier values changed. Applications using the \c Wiredtiger.config file or \c WIREDTIGER_CONFIG environment variable may need to change. The old order:
  1. default ::wiredtiger_open configuration
  2. base configuration file, created with the database
  3. user configuration file \c Wiredtiger.config
  4. user environment variable \c WIREDTIGER_CONFIG
  5. configuration string passed in to ::wiredtiger_open

In the new order the user's configuration settings override:

  1. default ::wiredtiger_open configuration
  2. base configuration file, created with the database
  3. configuration string passed in to ::wiredtiger_open
  4. user configuration file \c Wiredtiger.config
  5. user environment variable \c WIREDTIGER_CONFIG
\c os_cache_dirty_max off for LSM
In some earlier versions of WiredTiger, creating an LSM table automatically configured \c os_cache_dirty_max, causing additional system calls that slowed some workloads. Applications that benefit from this setting should set it explicitly in WT_SESSION::create.
\c transaction_sync setting change
In the 2.2.1 release of WiredTiger the ::wiredtiger_open \c transaction_sync configuration setting has changed from a string value to a structure with two fields: \c method, which corresponds to the previous value (\c dsync, \c fsync or \c none), and \c enabled, which determines whether there is a sync on every transaction commit by default. Applications can enable or disable sync for a specific transaction with the \c sync setting to WT_SESSION::begin_transaction.
@section version_220 Upgrading to Version 2.2.0
WT_SESSION::create prefix_compression disabled by default
In the 2.2.0 release, prefix compression default to \c false. Applications that benefit from prefix compression will need to explicitly set \c prefix_compression=true when creating tables.
wiredtiger_open verbose message changes
In the 2.2.0 release it is now necessary to include \c --enable-verbose in the configure command to be able to use verbose messages.
@section version_212 Upgrading to Version 2.1.2
::wiredtiger_open shared_cache configuration changes
In the 2.1.2 release of WiredTiger the ::wiredtiger_open \c shared_cache configuration option group have changed. The option that was named \c enable is no longer available. To enable a \c shared_cache it is compulsory to name the pool being shared. We are now also enforcing that only one of \c cache_size and \c shared_cache are specified in the ::wiredtiger_open configuration string.
@section version_211 Upgrading to Version 2.1.1
WT_EXTENSION_API::config methods
In the 2.1.1 release of WiredTiger the configuration string parsing API has been changed and added to a new public handle. The WT_EXTENSION_API::config_strget, WT_EXTENSION_API::config_scan_begin, WT_EXTENSION_API::config_scan_next and WT_EXTENSION_API::config_scan_end have been removed. They have been replaced by a WT_EXTENSION_API::config_parser_open method, which can be used to parse configuration strings. See the WT_CONFIG_PARSER documentation for examples on how to use the updated API.
@section version_21 Upgrading to Version 2.1
WT_ITEM::size type
In the 2.1 release of WiredTiger WT_ITEM::size type has changed from \c uint32_t to \c size_t. Applications may require modifications to resolve compile-time errors.
WT_COMPRESSOR::compress_raw signature
In the 2.1 release of WiredTiger, the behavior of the compress_raw callback has changed so that it will only be retried if it returns \c EAGAIN. If it returns zero and sets \c result_slots to zero, WiredTiger will assume that raw compression has failed and will fall back to calling WT_COMPRESSOR::compress.
Transaction sync default setting
In the 2.1 release of WiredTiger the ::wiredtiger_open \c transaction_sync configuration setting default value has changed from "dsync" to "fsync". This is due to enhancements to the group commit implementation in WiredTiger - which mean that greater throughput can be achieved with explicit "fsync" calls than by enabling "dsync" on a file handle. Applications that don't execute concurrent transactions may see better throughput with transaction_sync set to "dsync".
@section version_20 Upgrading to Version 2.0
File format changes
The underlying file format is unchanged in 2.0
WT_SESSION::create LSM configuration options
In the 2.0 release of WiredTiger the LSM configuration options have been collected into a configuration option subgroup. All configuration options to WT_SESSION::create that previously had a prefix of \c lsm_ now belong to the \c lsm configuration group. If you are explicitly configuring any of the following options, you should review the WT_SESSION::create documentation for details of the updated syntax: lsm_auto_throttle, lsm_bloom, lsm_bloom_config, lsm_bloom_bit_count, lsm_bloom_hash_count, lsm_bloom_oldest, lsm_chunk_max, lsm_chunk_size, lsm_merge_max and lsm_merge_threads.
@section version_166 Upgrading to Version 1.6.6
File format changes
The underlying file formats changed in the 1.6.6 release; tables and files should be dumped and re-loaded into a new database.
WT_SESSION::compact trigger configuration
In previous releases, the \c trigger configuration string to the WT_SESSION::compact method specified a requirement to initiate compaction; in the 1.6.6 release, this configuration string has been removed, and compaction will be attempted if it seems likely at least 10% of the file can be recovered. Applications may require modifications to resolve run-time errors.
Statistics configuration
In previous releases, the ::wiredtiger_open function took a \c statistics configuration, which defaulted to false; when set to true, additional, generally performance-expensive statistics were maintained by the database, above and beyond a default set of statistics. In version 1.6.6, the \c statistics configuration is a list which may be set to "all", "fast" or "none". When set to "none", no statistics are maintained by the database; when set to "fast", only relatively performance-inexpensive statistics are maintained, and when set to "all", all statistics are maintained, regardless of cost. In previous releases, the ::wiredtiger_open function took a \c statistics_log configuration which logged the performance-inexpensive database statistics to a file. In version 1.6.6, the \c statistics_log configuration logs whatever statistics are configured for the database. If the database is configured with \c statistics to "none", no statistics will be logged to the file; if the database is configured with "all" or "fast", the corresponding statistics will be logged to the file. In previous releases, the WT_SESSION::cursor method took \c statistics_clear and a \c statistics_fast configurations. The \c statistics_clear configuration defaulted to false; when set to true, statistics counters were reset after they were gathered by the cursor. The \c statistics_fast configuration defaulted to true; when set to true, the cursor only gathered performance-inexpensive statistics for the cursor, and when set to false, the cursor gathered all available statistics, regardless of cost. In version 1.6.6, these two configuration booleans have been replaced with a new configuration list \c statistics, which may be set from the values "clear", "fast" and "all". When "fast" is configured, only relatively performance-inexpensive statistics are gathered, and when "all" is configured, all statistics are gathered, regardless of cost. When "clear" is configured, statistics counters are reset after they are gathered. Additionally, in version 1.6.6, statistics cursors must be configured to agree with the database statistics configuration; when the database statistics are configured to "none", attempts to open a statistics cursor will fail; when the database statistics are configured to "fast", a statistics cursor must also be configured to "fast"; when the database statistics are configured to "all", a statistics cursor may be configured to either "fast" or "all". Opening a statistics cursor without configuring either "fast" or "all" will configure the cursor to be the same as the current database configuration. Applications may require modifications to resolve run-time errors; application statistics configuration and cursors should be reviewed to confirm they are configured for the desired behavior;
WT_EVENT_HANDLER interface changes
Add a new WT_EVENT_HANDLER::handle_close callback that WiredTiger will call any time it automatically closes an application session or cursor handle. Additionally add a WT_SESSION parameter into the existing WT_EVENT_HANDLER::handle_error, WT_EVENT_HANDLER::handle_message and WT_EVENT_HANDLER::handle_progress callback functions.
@section version_165 Upgrading to Version 1.6.5
WT_CURSOR::insert behavior
In previous releases, the WT_CURSOR::insert ended positioned at the inserted record. To minimize the cursor resources held by applications inserting many records, the WT_CURSOR::insert method has been changed to end without any position. Application insert cursors should be reviewed to confirm they do not attempt to iterate after an insert.
WT_SESSION::open_cursor statistics_fast configuration
In previous releases, the default \c statistics_fast configuration to the WT_SESSION::open_cursor method was \c false; in the 1.6.5 release, the default statistics_fast configuration is \c true. Applications opening statistics cursors should be reviewed to confirm they have the correct behavior.
Synchronous checkpoint configuration
The \c sync configuration key to ::wiredtiger_open has been renamed \c checkpoint_sync.
@section version_164 Upgrading to Version 1.6.4
File format changes
The underlying file formats changed in the 1.6.4 release; tables and files should be dumped and re-loaded into a new database.
wt utility load command
The default behavior of the \c wt utility's \c load command has been changed to overwrite existing data, by default, and the \c -o flag to the \c load command (overwrite existing data) has been replaced with the \c -n flag (do not overwrite existing data). Applications requiring the previous default behavior of not overwriting existing data should add the \c -n option to their command line configuration; applications previously using the \c -o option on their command line configurations should remove it.
@section version_163 Upgrading to Version 1.6.3
Cursor overwrite configuration
In previous releases, the WT_SESSION::open_cursor \c overwrite configuration string behaved inconsistently across Btree and LSM data sources. In Btree, \c overwrite was \c false by default and was limited to the WT_CURSOR::insert method, changing an insert to succeed regardless of whether or not the record previously existed. In LSM trees, \c overwrite was \c true by default, and applied to the WT_CURSOR::insert, WT_CURSOR::remove and WT_CURSOR::update methods, configuring all three methods to ignore the existing state of the record. In the 1.6.3 release, the \c overwrite configuration is consistent across both Btree and LSM tree data sources. For performance reasons, the default is the behavior previously described for LSM trees: in other words, \c overwrite is \c true by default, causing WT_CURSOR::insert, WT_CURSOR::remove and WT_CURSOR::update to ignore the current state of the record, and these methods will succeed regardless of whether or not the record previously exists. When an application configures \c overwrite to \c false, WT_CURSOR::insert will fail with ::WT_DUPLICATE_KEY if the record previously exists, and WT_CURSOR::update and WT_CURSOR::remove will fail with ::WT_NOTFOUND if the record does not previously exist. This is a potentially serious API change that will not be detected by compilation. Application cursors should be reviewed to confirm they are configured for the desired behavior.
::wiredtiger_open no longer accepts a \c transactional configuration
The \c transactional configuration key has been removed from ::wiredtiger_open. Any application setting it should simply remove it, no change in application behavior is needed.
@section version_162 Upgrading to Version 1.6.2
Table of WiredTiger extension methods
New functionality was added to the list of WiredTiger extension methods; applications using the extension methods will require recompilation.
WT_SESSION::create no longer accepts a "source" configuration
The \c "source" configuration key has been removed from WT_SESSION::create. Normal applications should not have been using it, and there were a number of bugs associated with it.
Default checksum configuration
The default file checksum configuration was changed to \c uncompressed, which means blocks that are compressed will no longer also include a checksum, by default. Applications using compression insufficient for the purposes of corrupted block identification should change their file checksum configuration to \c on.
@section version_161 Upgrading to Version 1.6.1
Default page sizes
In the 1.6.1 release, the default for the WT_SESSION::create configuration string \c allocation_size changed from 512B to 4KB, and the default for the configuration string \c internal_page_max changed from 2KB to 4KB. Applications wanting to create files with smaller allocation or internal page sizes will need to set those configuration values explicitly.
Shared cache configuration
In the 1.6.1 release, an explicit shared_cache=(enable=boolean) option was added to the wiredtiger_open configuration options. Existing applications that use shared cache functionality will need to add the enable option to the configuration string. The default value for the option is false.
WT_COMPRESSOR::compress_raw signature
In the 1.6.1 release, the \c split_pct argument to the WT_COMPRESSOR::compress_raw function changed type from \c u_int to \c int, applications may require modification to avoid compiler warnings.
@section version_160 Upgrading to Version 1.6.0
File format changes
The underlying file formats changed in the 1.6.0 release; tables and files should be dumped and re-loaded into a new database.
@section version_153 Upgrading to Version 1.5.3
Configuration strings
An undocumented feature where configuration string case was ignored has been removed, and all configuration strings are now case-dependent. Applications may require modifications to resolve run-time errors.
Loading extensions and WT_EXTENSION_API
The following changes are only applicable to applications loading extensions and/or using the WiredTiger extension functions described in WT_EXTENSION_API.
WT_DATA_SOURCE
The following changes are only applicable to applications providing new implementations of the WiredTiger WT_DATA_SOURCE class.
@section version_143 Upgrading to Version 1.4.3
Statistics
WiredTiger statistics are no longer maintained by default; to configure statistics, use the \c statistics configuration string to the ::wiredtiger_open function.
@section version_139 Upgrading to Version 1.3.9
Compression
A new member, WT_COMPRESSOR::compress_raw, was added to the WT_COMPRESSOR extension API. Applications using the WT_COMPRESSOR extension API should add a NULL as the second field of that structure.
Checksums
The WT_SESSION::create method's \c checksum configuration string has been changed from a boolean type to a string type. Applications using the checksum configuration string should change a value of \c true to the string \c on, and a value of \c false to the string \c off or the string \c uncompressed.
File format changes
The underlying file formats changed in the 1.3.9 release; tables and files should be dumped and re-loaded into a new database.
@section version_138 Upgrading to Version 1.3.8
Statistics keys
The @ref statistics_keys "statistics key constants" have been renamed to use all capitals, and use consistent prefixes to distinguish between connection statistics and statistics for data sources.
@section version_136 Upgrading to Version 1.3.6
Installed library names
The installed WiredTiger extension library names changed to limit namespace pollution: @hrow{Library, Previous Name, New Name} @row{Bzip2 compression, bzip2_compress.a, libwiredtiger_bzip2.a} @row{, bzip2_compress.la, libwiredtiger_bzip2.la} @row{, bzip2_compress.so, libwiredtiger_bzip2.so} @row{Snappy compression, snappy_compress.a, libwiredtiger_snappy.a} @row{, snappy_compress.la, libwiredtiger_snappy.la} @row{, snappy_compress.so, libwiredtiger_snappy.so} @row{No-op compression, nop_compress.a, No longer installed} @row{, nop_compress.la, No longer installed} @row{, nop_compress.so, No longer installed} @row{Reverse order collator, reverse_collator.a, No longer installed} @row{, reverse_collator.la, No longer installed} @row{, reverse_collator.so, No longer installed}
Built-in compression names
The built-in compression name arguments to the WT_SESSION:create \c block_compressor configuration string changed for consistency: @hrow{Extension, Previous Name, New Name} @row{Bzip2 compression, "bzip2_compress", "bzip2"} @row{Snappy compression, "snappy_compress", "snappy"}
@section version_135 Upgrading to Version 1.3.5
File format changes
The underlying file formats changed in the 1.3.5 release; tables and files should be dumped and re-loaded into a new database.
@section version_13 Upgrading to Version 1.3
Checkpoint and Snapshot
The checkpoint functionality supported by WT_SESSION::checkpoint and the snapshot functionality supported by WT_SESSION::sync have been merged into a single piece of functionality.
WT_SESSION.open_cursor
In releases before 1.3, the WT_SESSION::open_cursor method could duplicate cursors that were not positioned in an object; in 1.3, a cursor must be positioned in order to be duplicated.
Transactional cursors
In releases before 1.3, ending a transaction by calling the WT_SESSION::commit_transaction or WT_SESSION::rollback_transaction methods implicitly closed all open cursors; in 1.3, the cursors remain open, but are reset (discarding their positions and cursor values). This means applications must change to either close cursors explicitly, or rely on an eventual WT_SESSION::close or WT_CONNECTION::close methods to implicitly close open cursors.
Default transactional isolation level
In releases before 1.3, the default isolation level for transaction was \c snapshot, and the default isolation level for non-transaction operations was \c read-uncommitted; in 1.3, the default isolation level for all operations is \c read-committed. The default can be overridden for a session using the \c isolation setting in WT_CONNECTION::open_cursor.
WT_SESSION.truncate
In releases before 1.3, the WT_SESSION::truncate method required cursors used for truncation of a cursor range to reference existing keys in the object; in 1.3, the WT_SESSION::truncate method has been changed to allow cursors to reference any valid key in the object's name space so applications may discard portions of the object name space without knowing exactly what records the object contains.
WT_CURSOR.equals
In releases before 1.3, the WT_CURSOR::equals method returned zero/non-zero to indicate cursor equality; in 1.3, the WT_CURSOR::equals method has been replaced with WT_CURSOR::compare, which compares two cursors and returns a cursor comparison status (less than 0, equal to 0, or greater than 0) depending on the cursors' key order.
File format changes
The underlying file formats changed in the 1.3 release; tables and files should be dumped and re-loaded into a new database.
*/