Library Version 11.2.5.3

Library Version 12.1.6.1

@@ -22,9 +22,7 @@ - +

Prev	Chapter 11. - Berkeley DB Transactional Data Store Applications -	Chapter 11. Berkeley DB Transactional Data Store Applications	Next

@@ -38,32 +36,46 @@

The first reason listed for using transactions was recoverability. Any -logical change to a database may require multiple changes to underlying -data structures. For example, modifying a record in a Btree may require -leaf and internal pages to split, so a single DB->put() method -call can potentially require that multiple physical database pages be -written. If only some of those pages are written and then the system -or application fails, the database is left inconsistent and cannot be -used until it has been recovered; that is, until the partially completed -changes have been undone.

Write-ahead-logging is the term that describes the underlying -implementation that Berkeley DB uses to ensure recoverability. What it means -is that before any change is made to a database, information about the -change is written to a database log. During recovery, the log is read, -and databases are checked to ensure that changes described in the log -for committed transactions appear in the database. Changes that appear -in the database but are related to aborted or unfinished transactions -in the log are undone from the database.

For recoverability after application or system failure, operations that -modify the database must be protected by transactions. More -specifically, operations are not recoverable unless a transaction is -begun and each operation is associated with the transaction via the -Berkeley DB interfaces, and then the transaction successfully committed. This -is true even if logging is turned on in the database environment.

Here is an example function that updates a record in a database in a -transactionally protected manner. The function takes a key and data -items as arguments and then attempts to store them into the database.

+ The first reason listed for using transactions was + recoverability. Any logical change to a database may require + multiple changes to underlying data structures. For example, + modifying a record in a Btree may require leaf and internal + pages to split, so a single DB->put() method call can + potentially require that multiple physical database pages be + written. If only some of those pages are written and then the + system or application fails, the database is left inconsistent + and cannot be used until it has been recovered; that is, until + the partially completed changes have been undone. +

+ Write-ahead-logging is the term that + describes the underlying implementation that Berkeley DB uses + to ensure recoverability. What it means is that before any + change is made to a database, information about the change is + written to a database log. During recovery, the log is read, + and databases are checked to ensure that changes described in + the log for committed transactions appear in the database. + Changes that appear in the database but are related to aborted + or unfinished transactions in the log are undone from the + database. +

+ For recoverability after application or system failure, + operations that modify the database must be protected by + transactions. More specifically, operations are not + recoverable unless a transaction is begun and each operation + is associated with the transaction via the Berkeley DB + interfaces, and then the transaction successfully committed. + This is true even if logging is turned on in the database + environment. +

+ Here is an example function that updates a record in a + database in a transactionally protected manner. The function + takes a key and data items as arguments and then attempts to + store them into the database. +

Berkeley DB also uses transactions to recover from deadlock. Database -operations (that is, any call to a function underlying the handles -returned by DB->open() and DB->cursor()) are usually -performed on behalf of a unique locker. Transactions can be used to -perform multiple calls on behalf of the same locker within a single -thread of control. For example, consider the case in which an -application uses a cursor scan to locate a record and then the -application accesses another other item in the database, based on the -key returned by the cursor, without first closing the cursor. If these -operations are done using default locker IDs, they may conflict. If the -locks are obtained on behalf of a transaction, using the transaction's -locker ID instead of the database handle's locker ID, the operations -will not conflict.

There is a new error return in this function that you may not have seen -before. In transactional (not Concurrent Data Store) applications -supporting both readers and writers, or just multiple writers, Berkeley DB -functions have an additional possible error return: -DB_LOCK_DEADLOCK. This means two threads of control deadlocked, -and the thread receiving the DB_LOCK_DEADLOCK error return has -been selected to discard its locks in order to resolve the problem. -When an application receives a DB_LOCK_DEADLOCK return, the -correct action is to close any cursors involved in the operation and -abort any enclosing transaction. In the sample code, any time the -DB->put() method returns DB_LOCK_DEADLOCK, DB_TXN->abort() is -called (which releases the transaction's Berkeley DB resources and undoes any -partial changes to the databases), and then the transaction is retried -from the beginning.

There is no requirement that the transaction be attempted again, but -that is a common course of action for applications. Applications may -want to set an upper bound on the number of times an operation will be -retried because some operations on some data sets may simply be unable -to succeed. For example, updating all of the pages on a large Web site -during prime business hours may simply be impossible because of the high -access rate to the database.

The DB_TXN->abort() method is called in error cases other than deadlock. -Any time an error occurs, such that a transactionally protected set of -operations cannot complete successfully, the transaction must be -aborted. While deadlock is by far the most common of these errors, -there are other possibilities; for example, running out of disk space -for the filesystem. In Berkeley DB transactional applications, there are -three classes of error returns: "expected" errors, "unexpected but -recoverable" errors, and a single "unrecoverable" error. Expected -errors are errors like -DB_NOTFOUND, -which indicates that a -searched-for key item is not present in the database. Applications may -want to explicitly test for and handle this error, or, in the case where -the absence of a key implies the enclosing transaction should fail, -simply call DB_TXN->abort(). Unexpected but recoverable errors are -errors like -DB_LOCK_DEADLOCK, -which indicates that an operation -has been selected to resolve a deadlock, or a system error such as EIO, -which likely indicates that the filesystem has no available disk space. -Applications must immediately call DB_TXN->abort() when these returns -occur, as it is not possible to proceed otherwise. The only -unrecoverable error is -DB_RUNRECOVERY, -which indicates that the -system must stop and recovery must be run.

The above code can be simplified in the case of a transaction comprised -entirely of a single database put or delete operation, as operations -occurring in transactional databases are implicitly transaction -protected. For example, in a transactional database, the above code -could be more simply written as:

+ Berkeley DB also uses transactions to recover from deadlock. + Database operations (that is, any call to a function + underlying the handles returned by DB->open() and DB->cursor()) + are usually performed on behalf of a unique locker. + Transactions can be used to perform multiple calls on behalf + of the same locker within a single thread of control. For + example, consider the case in which an application uses a + cursor scan to locate a record and then the application + accesses another other item in the database, based on the key + returned by the cursor, without first closing the cursor. If + these operations are done using default locker IDs, they may + conflict. If the locks are obtained on behalf of a + transaction, using the transaction's locker ID instead of the + database handle's locker ID, the operations will not + conflict. +

+ There is a new error return in this function that you may + not have seen before. In transactional (not Concurrent Data + Store) applications supporting both readers and writers, or + just multiple writers, Berkeley DB functions have an + additional possible error return: + DB_LOCK_DEADLOCK. This means two threads of + control deadlocked, and the thread receiving the + DB_LOCK_DEADLOCK error return has been + selected to discard its locks in order to resolve the problem. + When an application receives a + DB_LOCK_DEADLOCK return, the correct + action is to close any cursors involved in the operation and + abort any enclosing transaction. In the sample code, any time + the DB->put() method returns + DB_LOCK_DEADLOCK, DB_TXN->abort() is called + (which releases the transaction's Berkeley DB resources and + undoes any partial changes to the databases), and then the + transaction is retried from the beginning. +

+ There is no requirement that the transaction be attempted + again, but that is a common course of action for applications. + Applications may want to set an upper bound on the number of + times an operation will be retried because some operations on + some data sets may simply be unable to succeed. For example, + updating all of the pages on a large Web site during prime + business hours may simply be impossible because of the high + access rate to the database. +

+ The DB_TXN->abort() method is called in error cases other than + deadlock. Any time an error occurs, such that a + transactionally protected set of operations cannot complete + successfully, the transaction must be aborted. While deadlock + is by far the most common of these errors, there are other + possibilities; for example, running out of disk space for the + filesystem. In Berkeley DB transactional applications, there + are three classes of error returns: "expected" errors, + "unexpected but recoverable" errors, and a single + "unrecoverable" error. Expected errors are errors like DB_NOTFOUND, + which indicates that a searched-for key item is not present in + the database. Applications may want to explicitly test for and + handle this error, or, in the case where the absence of a key + implies the enclosing transaction should fail, simply call + DB_TXN->abort(). Unexpected but recoverable errors are errors like + DB_LOCK_DEADLOCK, + which indicates that an operation has been selected to resolve a + deadlock, or a system error such as EIO, which likely indicates + that the filesystem has no available disk space. Applications must + immediately call DB_TXN->abort() when these returns occur, as it is not + possible to proceed otherwise. The only unrecoverable error is + DB_RUNRECOVERY, + which indicates that the system + must stop and recovery must be run. +

+ The above code can be simplified in the case of a + transaction comprised entirely of a single database put or + delete operation, as operations occurring in transactional + databases are implicitly transaction protected. For example, + in a transactional database, the above code could be more + simply written as: +

Programmers should not attempt to enumerate all possible error returns -in their software. Instead, they should explicitly handle expected -returns and default to aborting the transaction for the rest. It is -entirely the choice of the programmer whether to check for -DB_RUNRECOVERY -explicitly or not — attempting new Berkeley DB -operations after -DB_RUNRECOVERY -is returned does not worsen the -situation. Alternatively, using the DB_ENV->set_event_notify() method to -handle an unrecoverable error and simply doing some number of -abort-and-retry cycles for any unexpected Berkeley DB or system error in the -mainline code often results in the simplest and cleanest application -code.

+ and the underlying transaction would be automatically + handled by Berkeley DB. +

+ Programmers should not attempt to enumerate all possible + error returns in their software. Instead, they should + explicitly handle expected returns and default to aborting the + transaction for the rest. It is entirely the choice of the + programmer whether to check for + DB_RUNRECOVERY explicitly or not — + attempting new Berkeley DB operations after DB_RUNRECOVERY + is returned does not worsen the + situation. Alternatively, using the DB_ENV->set_event_notify() method + to handle an unrecoverable error and simply doing some number + of abort-and-retry cycles for any unexpected Berkeley DB or + system error in the mainline code often results in the + simplest and cleanest application code. +