/*! @page upgrading Upgrading WiredTiger applications
@section version_232 Upgrading to Version 2.3.2
- Cursors no longer reset on transaction begin or commit
-
In previous versions of WiredTiger, all cursors in a session were reset at
transaction boundaries (WT_SESSION::begin_transaction,
WT_SESSION::commit_transaction and WT_SESSION::rollback_transaction).
Now they are only reset by WT_SESSION::rollback_transaction.
This change means that a cursor can be used to iterate through a table and
perform transactional updates based on the visited records without the cursor
losing its position.
Applications relying on this behavior (for example, to avoid pinning the
underlying cursor resources), should reset cursors explicitly when the
position is no longer required.
- WT_COLLATOR interface changes
-
Add a new WT_COLLATOR::customize callback that WiredTiger will call (if set)
for each data source configured to use the collator. Applications using the
existing WT_COLLATOR interface that do not require the new functionality should
set this callback to \c NULL.
- ::wiredtiger_open lsm_merge option changed
-
The global lsm_merge configuration setting has been moved into
the \c lsm_manager option group. To disable merges in all LSM
trees pass \c lsm_manger=(merge=false) to ::wiredtiger_open.
- Java include path search changes
-
We have updated the configure logic used to search for Java Native Interface
files. This fixes problems building WiredTiger's Java API, particularly on
OS X, but may introduce problems if your build procedure relied on the old
search behavior.
- Default mutex implementation changed
-
The default mutex implementation has been changed from adaptive pthread
mutexes to non-adaptive pthread mutexes. Installations can explicitly
select adaptive pthread mutexes by specifying
\c --with-spinlock=pthread_adaptive at configuration time.
- LSM merge threads option change
-
The WT_SESSION::create \c lsm=(merge_threads) configuration option has been
replaced by the W::wiredtiger_open \c lsm_manager=(worker_thread_max) option.
The new version specifies a set of LSM threads that are shared across all
LSM trees in a database, the older configuration was per LSM table.
@section version_231 Upgrading to Version 2.3.1
- ::wiredtiger_open eviction_workers configuration changed
-
The \c eviction_workers configuration setting has been replaced by
\c eviction=(threads_min) and \c eviction=(threads_max) settings.
There is also a semantic change because \c eviction_workers used to
configure additional threads whereas the new settings configure the total
number of threads involved with eviction.
@section version_230 Upgrading to Version 2.3.0
There are no special upgrade steps required.
@section version_221 Upgrading to Version 2.2.1
- ::wiredtiger_open configuration parsing order changed
-
In the 2.2.1 release, the order that configuration strings are
parsed and override earlier values changed. Applications using
the \c Wiredtiger.config file or \c WIREDTIGER_CONFIG environment
variable may need to change. The old order:
- default ::wiredtiger_open configuration
- base configuration file, created with the database
- user configuration file \c Wiredtiger.config
- user environment variable \c WIREDTIGER_CONFIG
- configuration string passed in to ::wiredtiger_open
In the new order the user's configuration settings override:
- default ::wiredtiger_open configuration
- base configuration file, created with the database
- configuration string passed in to ::wiredtiger_open
- user configuration file \c Wiredtiger.config
- user environment variable \c WIREDTIGER_CONFIG
- \c os_cache_dirty_max off for LSM
-
In some earlier versions of WiredTiger, creating an LSM table automatically
configured \c os_cache_dirty_max, causing additional system calls that slowed
some workloads. Applications that benefit from this setting should set it
explicitly in WT_SESSION::create.
- \c transaction_sync setting change
-
In the 2.2.1 release of WiredTiger the ::wiredtiger_open \c transaction_sync
configuration setting has changed from a string value to a structure with two
fields: \c method, which corresponds to the previous value (\c dsync, \c fsync
or \c none), and \c enabled, which determines whether there is a sync on
every transaction commit by default. Applications can enable or disable sync
for a specific transaction with the \c sync setting to
WT_SESSION::begin_transaction.
@section version_220 Upgrading to Version 2.2.0
- WT_SESSION::create prefix_compression disabled by default
-
In the 2.2.0 release, prefix compression default to \c false. Applications
that benefit from prefix compression will need to explicitly set
\c prefix_compression=true when creating tables.
- wiredtiger_open verbose message changes
-
In the 2.2.0 release it is now necessary to include \c --enable-verbose
in the configure command to be able to use verbose messages.
@section version_212 Upgrading to Version 2.1.2
- ::wiredtiger_open shared_cache configuration changes
-
In the 2.1.2 release of WiredTiger the ::wiredtiger_open \c shared_cache
configuration option group have changed. The option that was named \c enable is
no longer available. To enable a \c shared_cache it is compulsory to name
the pool being shared.
We are now also enforcing that only one of \c cache_size and \c shared_cache
are specified in the ::wiredtiger_open configuration string.
@section version_211 Upgrading to Version 2.1.1
- WT_EXTENSION_API::config methods
-
In the 2.1.1 release of WiredTiger the configuration string parsing API
has been changed and added to a new public handle. The
WT_EXTENSION_API::config_strget, WT_EXTENSION_API::config_scan_begin,
WT_EXTENSION_API::config_scan_next and WT_EXTENSION_API::config_scan_end
have been removed. They have been replaced by a
WT_EXTENSION_API::config_parser_open method, which can be used to parse
configuration strings. See the WT_CONFIG_PARSER documentation for
examples on how to use the updated API.
@section version_21 Upgrading to Version 2.1
- WT_ITEM::size type
-
In the 2.1 release of WiredTiger WT_ITEM::size type has changed from
\c uint32_t to \c size_t. Applications may require modifications to
resolve compile-time errors.
- WT_COMPRESSOR::compress_raw signature
-
In the 2.1 release of WiredTiger, the behavior of the compress_raw
callback has changed so that it will only be retried if it returns
\c EAGAIN. If it returns zero and sets \c result_slots to zero,
WiredTiger will assume that raw compression has failed and will fall
back to calling WT_COMPRESSOR::compress.
- Transaction sync default setting
-
In the 2.1 release of WiredTiger the ::wiredtiger_open \c transaction_sync
configuration setting default value has changed from "dsync" to "fsync".
This is due to enhancements to the group commit implementation in
WiredTiger - which mean that greater throughput can be achieved with
explicit "fsync" calls than by enabling "dsync" on a file handle.
Applications that don't execute concurrent transactions may see better
throughput with transaction_sync set to "dsync".
@section version_20 Upgrading to Version 2.0
- File format changes
-
The underlying file format is unchanged in 2.0
- WT_SESSION::create LSM configuration options
-
In the 2.0 release of WiredTiger the LSM configuration options have been
collected into a configuration option subgroup. All configuration options
to WT_SESSION::create that previously had a prefix of \c lsm_ now belong to
the \c lsm configuration group. If you are explicitly configuring any of the
following options, you should review the WT_SESSION::create documentation for
details of the updated syntax: lsm_auto_throttle, lsm_bloom, lsm_bloom_config,
lsm_bloom_bit_count, lsm_bloom_hash_count, lsm_bloom_oldest, lsm_chunk_max,
lsm_chunk_size, lsm_merge_max and lsm_merge_threads.
@section version_166 Upgrading to Version 1.6.6
- File format changes
-
The underlying file formats changed in the 1.6.6 release; tables and files
should be dumped and re-loaded into a new database.
- WT_SESSION::compact trigger configuration
-
In previous releases, the \c trigger configuration string to the
WT_SESSION::compact method specified a requirement to initiate compaction; in
the 1.6.6 release, this configuration string has been removed, and compaction
will be attempted if it seems likely at least 10% of the file can be recovered.
Applications may require modifications to resolve run-time errors.
- Statistics configuration
-
In previous releases, the ::wiredtiger_open function took a \c statistics
configuration, which defaulted to false; when set to true, additional,
generally performance-expensive statistics were maintained by the
database, above and beyond a default set of statistics. In version
1.6.6, the \c statistics configuration is a list which may be set to
"all", "fast" or "none". When set to "none", no statistics are
maintained by the database; when set to "fast", only relatively
performance-inexpensive statistics are maintained, and when set to
"all", all statistics are maintained, regardless of cost.
In previous releases, the ::wiredtiger_open function took a \c statistics_log
configuration which logged the performance-inexpensive database
statistics to a file. In version 1.6.6, the \c statistics_log
configuration logs whatever statistics are configured for the database.
If the database is configured with \c statistics to "none", no
statistics will be logged to the file; if the database is configured
with "all" or "fast", the corresponding statistics will be logged to the
file.
In previous releases, the WT_SESSION::cursor method took
\c statistics_clear and a \c statistics_fast configurations. The
\c statistics_clear configuration defaulted to false; when set to true,
statistics counters were reset after they were gathered by the cursor.
The \c statistics_fast configuration defaulted to true; when set to
true, the cursor only gathered performance-inexpensive statistics for
the cursor, and when set to false, the cursor gathered all available
statistics, regardless of cost.
In version 1.6.6, these two configuration booleans have been replaced
with a new configuration list \c statistics, which may be set from the
values "clear", "fast" and "all". When "fast" is configured, only
relatively performance-inexpensive statistics are gathered, and when
"all" is configured, all statistics are gathered, regardless of cost.
When "clear" is configured, statistics counters are reset after they are
gathered.
Additionally, in version 1.6.6, statistics cursors must be configured
to agree with the database statistics configuration; when the database
statistics are configured to "none", attempts to open a statistics
cursor will fail; when the database statistics are configured to "fast",
a statistics cursor must also be configured to "fast"; when the database
statistics are configured to "all", a statistics cursor may be
configured to either "fast" or "all". Opening a statistics cursor
without configuring either "fast" or "all" will configure the cursor to
be the same as the current database configuration.
Applications may require modifications to resolve run-time errors;
application statistics configuration and cursors should be reviewed to
confirm they are configured for the desired behavior;
- WT_EVENT_HANDLER interface changes
-
Add a new WT_EVENT_HANDLER::handle_close callback that WiredTiger will call
any time it automatically closes an application session or cursor handle.
Additionally add a WT_SESSION parameter into the existing
WT_EVENT_HANDLER::handle_error, WT_EVENT_HANDLER::handle_message and
WT_EVENT_HANDLER::handle_progress callback functions.
@section version_165 Upgrading to Version 1.6.5
- WT_CURSOR::insert behavior
-
In previous releases, the WT_CURSOR::insert ended positioned at the inserted
record. To minimize the cursor resources held by applications inserting many
records, the WT_CURSOR::insert method has been changed to end without any
position. Application insert cursors should be reviewed to confirm they do
not attempt to iterate after an insert.
- WT_SESSION::open_cursor statistics_fast configuration
-
In previous releases, the default \c statistics_fast configuration to
the WT_SESSION::open_cursor method was \c false; in the 1.6.5 release,
the default statistics_fast configuration is \c true. Applications
opening statistics cursors should be reviewed to confirm they have the
correct behavior.
- Synchronous checkpoint configuration
-
The \c sync configuration key to ::wiredtiger_open has been renamed \c checkpoint_sync.
@section version_164 Upgrading to Version 1.6.4
- File format changes
-
The underlying file formats changed in the 1.6.4 release; tables and files
should be dumped and re-loaded into a new database.
- wt utility load command
-
The default behavior of the \c wt utility's \c load command has been
changed to overwrite existing data, by default, and the \c -o flag to
the \c load command (overwrite existing data) has been replaced with the
\c -n flag (do not overwrite existing data). Applications requiring
the previous default behavior of not overwriting existing data should
add the \c -n option to their command line configuration; applications
previously using the \c -o option on their command line configurations
should remove it.
@section version_163 Upgrading to Version 1.6.3
- Cursor overwrite configuration
-
In previous releases, the WT_SESSION::open_cursor \c overwrite configuration
string behaved inconsistently across Btree and LSM data sources. In Btree,
\c overwrite was \c false by default and was limited to the WT_CURSOR::insert
method, changing an insert to succeed regardless of whether or not the record
previously existed. In LSM trees, \c overwrite was \c true by default, and
applied to the WT_CURSOR::insert, WT_CURSOR::remove and WT_CURSOR::update
methods, configuring all three methods to ignore the existing state of the
record.
In the 1.6.3 release, the \c overwrite configuration is consistent across both
Btree and LSM tree data sources. For performance reasons, the default is the
behavior previously described for LSM trees: in other words, \c overwrite is
\c true by default, causing WT_CURSOR::insert, WT_CURSOR::remove and
WT_CURSOR::update to ignore the current state of the record, and these methods
will succeed regardless of whether or not the record previously exists. When
an application configures \c overwrite to \c false, WT_CURSOR::insert will fail
with ::WT_DUPLICATE_KEY if the record previously exists, and WT_CURSOR::update
and WT_CURSOR::remove will fail with ::WT_NOTFOUND if the record does not
previously exist.
This is a potentially serious API change that will not be detected by
compilation. Application cursors should be reviewed to confirm they are
configured for the desired behavior.
- ::wiredtiger_open no longer accepts a \c transactional configuration
-
The \c transactional configuration key has been removed from
::wiredtiger_open. Any application setting it should simply remove it,
no change in application behavior is needed.
@section version_162 Upgrading to Version 1.6.2
- Table of WiredTiger extension methods
-
New functionality was added to the list of WiredTiger extension methods;
applications using the extension methods will require recompilation.
- WT_SESSION::create no longer accepts a "source" configuration
-
The \c "source" configuration key has been removed from WT_SESSION::create.
Normal applications should not have been using it, and there were a number of
bugs associated with it.
- Default checksum configuration
-
The default file checksum configuration was changed to \c uncompressed,
which means blocks that are compressed will no longer also include a
checksum, by default. Applications using compression insufficient for
the purposes of corrupted block identification should change their file
checksum configuration to \c on.
@section version_161 Upgrading to Version 1.6.1
- Default page sizes
-
In the 1.6.1 release, the default for the WT_SESSION::create configuration
string \c allocation_size changed from 512B to 4KB, and the default for the
configuration string \c internal_page_max changed from 2KB to 4KB.
Applications wanting to create files with smaller allocation or internal
page sizes will need to set those configuration values explicitly.
- Shared cache configuration
-
In the 1.6.1 release, an explicit shared_cache=(enable=boolean) option was
added to the wiredtiger_open configuration options. Existing applications that
use shared cache functionality will need to add the enable option to the
configuration string. The default value for the option is false.
- WT_COMPRESSOR::compress_raw signature
-
In the 1.6.1 release, the \c split_pct argument to the
WT_COMPRESSOR::compress_raw function changed type from \c u_int to \c int,
applications may require modification to avoid compiler warnings.
@section version_160 Upgrading to Version 1.6.0
- File format changes
-
The underlying file formats changed in the 1.6.0 release; tables and files
should be dumped and re-loaded into a new database.
@section version_153 Upgrading to Version 1.5.3
- Configuration strings
-
An undocumented feature where configuration string case was ignored has
been removed, and all configuration strings are now case-dependent.
Applications may require modifications to resolve run-time errors.
- Loading extensions and WT_EXTENSION_API
-
The following changes are only applicable to applications loading
extensions and/or using the WiredTiger extension functions described in
WT_EXTENSION_API.
-
The signature of ::wiredtiger_extension_init has changed
from
(WT_SESSION *session, WT_EXTENSION_API *api)
to
(WT_CONNECTION *connection)
. As no WT_EXTENSION_API
handle reference is passed to the function, the
WT_CONNECTION::get_extension_api has been added to support retrieval of
the extension API. Applications may require modifications.
-
The type of all configuration arguments to extension methods has changed
from \c "const char *" to \c "WT_CONFIG_ARG *", and the WT_EXTENSION::config
method added to support configuration parsing; applications may require
modifications.
-
The undocumented \c wiredtiger_XXX defines for WT_EXTENSION_API extension
methods have been removed from the \c wiredtiger_ext.h include file;
applications should instead use the method handles referenced by the
WT_EXTENSION_API handle to call extension functions.
-
The extension API methods have all changed to require an additional
parameter, the WT_EXTENSION_API method handle; applications may require
modifications.
-
The WT_SESSION::msg_printf method was replaced by WT_EXTENSION_API::msg_printf;
applications may require modifications.
- WT_DATA_SOURCE
-
The following changes are only applicable to applications providing new
implementations of the WiredTiger WT_DATA_SOURCE class.
-
The WT_DATA_SOURCE class has three new methods: WT_DATA_SOURCE::compact,
WT_DATA_SOURCE::salvage, and WT_DATA_SOURCE::verify; applications may
require modifications to resolve compile errors.
-
The \c owner argument to the WT_DATA_SOURCE::open_cursor method has been
removed; applications may require modifications to resolve compile errors.
-
The \c exclusive argument to the WT_DATA_SOURCE::create method has been
removed; applications may require modifications to resolve compile errors.
@section version_143 Upgrading to Version 1.4.3
- Statistics
-
WiredTiger statistics are no longer maintained by default; to configure
statistics, use the \c statistics configuration string to the
::wiredtiger_open function.
@section version_139 Upgrading to Version 1.3.9
- Compression
-
A new member, WT_COMPRESSOR::compress_raw, was added to the WT_COMPRESSOR
extension API. Applications using the WT_COMPRESSOR extension API should
add a NULL as the second field of that structure.
- Checksums
-
The WT_SESSION::create method's \c checksum configuration string has
been changed from a boolean type to a string type. Applications using
the checksum configuration string should change a value of \c true to
the string \c on, and a value of \c false to the string \c off or the
string \c uncompressed.
- File format changes
-
The underlying file formats changed in the 1.3.9 release; tables and files
should be dumped and re-loaded into a new database.
@section version_138 Upgrading to Version 1.3.8
- Statistics keys
-
The @ref statistics_keys "statistics key constants" have been renamed to use
all capitals, and use consistent prefixes to distinguish between connection
statistics and statistics for data sources.
@section version_136 Upgrading to Version 1.3.6
- Installed library names
-
The installed WiredTiger extension library names changed to limit
namespace pollution:
@hrow{Library, Previous Name, New Name}
@row{Bzip2 compression, bzip2_compress.a, libwiredtiger_bzip2.a}
@row{, bzip2_compress.la, libwiredtiger_bzip2.la}
@row{, bzip2_compress.so, libwiredtiger_bzip2.so}
@row{Snappy compression, snappy_compress.a, libwiredtiger_snappy.a}
@row{, snappy_compress.la, libwiredtiger_snappy.la}
@row{, snappy_compress.so, libwiredtiger_snappy.so}
@row{No-op compression, nop_compress.a, No longer installed}
@row{, nop_compress.la, No longer installed}
@row{, nop_compress.so, No longer installed}
@row{Reverse order collator, reverse_collator.a, No longer installed}
@row{, reverse_collator.la, No longer installed}
@row{, reverse_collator.so, No longer installed}
- Built-in compression names
-
The built-in compression name arguments to the WT_SESSION:create
\c block_compressor configuration string changed for consistency:
@hrow{Extension, Previous Name, New Name}
@row{Bzip2 compression, "bzip2_compress", "bzip2"}
@row{Snappy compression, "snappy_compress", "snappy"}
@section version_135 Upgrading to Version 1.3.5
- File format changes
-
The underlying file formats changed in the 1.3.5 release; tables and files
should be dumped and re-loaded into a new database.
@section version_13 Upgrading to Version 1.3
- Checkpoint and Snapshot
-
The checkpoint functionality supported by WT_SESSION::checkpoint and the
snapshot functionality supported by WT_SESSION::sync have been merged
into a single piece of functionality.
- WT_SESSION.checkpoint
The WT_SESSION::checkpoint method's \c snapshot configuration string has
been renamed to \c name. The name assigned to checkpoints without a
specified \c name configuration is now \c "WiredTigerCheckpoint".
- WT_SESSION.drop
In releases before 1.3, the WT_SESSION::drop method was used to delete
snapshots. In 1.3, the functionality of deleting snapshots has been
moved to the WT_SESSION::checkpoint method, specifically, snapshots are
discarded using the WT_SESSION::checkpoint method's \c drop configuration
string.
- WT_SESSION.sync
The WT_SESSION::sync method has been removed from the 1.3 release; the
functionality of creating an object snapshot has moved to the
WT_SESSION::checkpoint method, specifically, creating a snapshot of a
one or more objects is done using the WT_SESSION::checkpoint method's
\c target configuration string.
- wt drop -s
The \c -s option to the \c drop command for the \c wt command line
utility has been removed, and object snapshots may no longer be removed
from the command line.
- wt dump, list -s
The \c -s options to the \c dump and \c list commands for the \c wt
command line utility have been renamed to be \c -c.
- WT_SESSION.open_cursor
-
In releases before 1.3, the WT_SESSION::open_cursor method could duplicate
cursors that were not positioned in an object; in 1.3, a cursor must be
positioned in order to be duplicated.
- Transactional cursors
-
In releases before 1.3, ending a transaction by calling the
WT_SESSION::commit_transaction or WT_SESSION::rollback_transaction methods
implicitly closed all open cursors; in 1.3, the cursors remain open, but are
reset (discarding their positions and cursor values). This means applications
must change to either close cursors explicitly, or rely on an eventual
WT_SESSION::close or WT_CONNECTION::close methods to implicitly close open
cursors.
- Default transactional isolation level
-
In releases before 1.3, the default isolation level for transaction was
\c snapshot, and the default isolation level for non-transaction operations
was \c read-uncommitted; in 1.3, the default isolation level for all
operations is \c read-committed.
The default can be overridden for a session using the \c isolation setting
in WT_CONNECTION::open_cursor.
- WT_SESSION.truncate
-
In releases before 1.3, the WT_SESSION::truncate method required cursors used
for truncation of a cursor range to reference existing keys in the object;
in 1.3, the WT_SESSION::truncate method has been changed to allow cursors to
reference any valid key in the object's name space so applications may discard
portions of the object name space without knowing exactly what records the
object contains.
- WT_CURSOR.equals
-
In releases before 1.3, the WT_CURSOR::equals method returned zero/non-zero
to indicate cursor equality; in 1.3, the WT_CURSOR::equals method has
been replaced with WT_CURSOR::compare, which compares two cursors and
returns a cursor comparison status (less than 0, equal to 0, or greater than
0) depending on the cursors' key order.
- File format changes
-
The underlying file formats changed in the 1.3 release; tables and files
should be dumped and re-loaded into a new database.
*/