diff options
| author | Lorry Tar Creator <lorry-tar-importer@baserock.org> | 2015-02-17 17:25:57 +0000 |
|---|---|---|
| committer | <> | 2015-03-17 16:26:24 +0000 |
| commit | 780b92ada9afcf1d58085a83a0b9e6bc982203d1 (patch) | |
| tree | 598f8b9fa431b228d29897e798de4ac0c1d3d970 /docs/programmer_reference | |
| parent | 7a2660ba9cc2dc03a69ddfcfd95369395cc87444 (diff) | |
| download | berkeleydb-780b92ada9afcf1d58085a83a0b9e6bc982203d1.tar.gz | |
Diffstat (limited to 'docs/programmer_reference')
207 files changed, 21697 insertions, 15494 deletions
diff --git a/docs/programmer_reference/BDB_Prog_Reference.pdf b/docs/programmer_reference/BDB_Prog_Reference.pdf Binary files differindex 5e5b59b4..6fce588f 100644 --- a/docs/programmer_reference/BDB_Prog_Reference.pdf +++ b/docs/programmer_reference/BDB_Prog_Reference.pdf diff --git a/docs/programmer_reference/am.html b/docs/programmer_reference/am.html index e866ee8c..394f61a8 100644 --- a/docs/programmer_reference/am.html +++ b/docs/programmer_reference/am.html @@ -14,13 +14,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 3. - Access Method Operations - </th> + <th colspan="3" align="center">Chapter 3. Access Method Operations </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rq_conf.html">Prev</a> </td> @@ -34,9 +32,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="am"></a>Chapter 3. - Access Method Operations - </h2> + <h2 class="title"><a id="am"></a>Chapter 3. Access Method Operations </h2> </div> </div> </div> @@ -52,24 +48,25 @@ </dt> <dt> <span class="sect1"> - <a href="am_opensub.html">Opening multiple databases in a single file</a> + <a href="am_opensub.html">Opening multiple databases in a + single file</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="am_opensub.html#idp724392">Configuring databases sharing a file</a> + <a href="am_opensub.html#idm1925008">Configuring databases sharing a file</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_opensub.html#idp768720">Caching databases sharing a file</a> + <a href="am_opensub.html#idm148416">Caching databases sharing a file</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_opensub.html#idp769416">Locking in databases based on sharing a file</a> + <a href="am_opensub.html#idm548184">Locking in databases based on sharing a file</a> </span> </dt> </dl> @@ -83,17 +80,20 @@ <dl> <dt> <span class="sect2"> - <a href="am_partition.html#am_partition_keys">Specifying partition keys</a> + <a href="am_partition.html#am_partition_keys">Specifying partition + keys</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_partition.html#am_partition_function">Partitioning callback</a> + <a href="am_partition.html#am_partition_function">Partitioning + callback</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_partition.html#partition_file_placement">Placing partition files</a> + <a href="am_partition.html#partition_file_placement">Placing partition + files</a> </span> </dt> </dl> @@ -130,7 +130,8 @@ </dt> <dt> <span class="sect1"> - <a href="am_verify.html">Database verification and salvage</a> + <a href="am_verify.html">Database verification and + salvage</a> </span> </dt> <dt> @@ -152,7 +153,7 @@ <dl> <dt> <span class="sect2"> - <a href="am_second.html#idp863384">Error Handling With Secondary Indexes</a> + <a href="am_second.html#idp382496">Error Handling With Secondary Indexes</a> </span> </dt> </dl> @@ -171,7 +172,8 @@ <dl> <dt> <span class="sect2"> - <a href="am_cursor.html#am_curget">Retrieving records with a cursor</a> + <a href="am_cursor.html#am_curget">Retrieving records with a + cursor</a> </span> </dt> <dt> @@ -208,21 +210,33 @@ </dd> </dl> </div> - <p>Once a database handle has been created using <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a>, there -are several standard access method operations. Each of these operations -is performed using a method referred to by the returned handle. -Generally, the database will be opened using <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a>. If the -database is from an old release of Berkeley DB, it may need to be upgraded to -the current release before it is opened using <a href="../api_reference/C/dbupgrade.html" class="olink">DB->upgrade()</a>.</p> - <p>Once a database has been opened, records may be retrieved (<a href="../api_reference/C/dbget.html" class="olink">DB->get()</a>), -stored (<a href="../api_reference/C/dbput.html" class="olink">DB->put()</a>), and deleted (<a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a>).</p> - <p>Additional operations supported by the database handle include -statistics (<a href="../api_reference/C/dbstat.html" class="olink">DB->stat()</a>), truncation (<a href="../api_reference/C/dbtruncate.html" class="olink">DB->truncate()</a>), -version upgrade (<a href="../api_reference/C/dbupgrade.html" class="olink">DB->upgrade()</a>), verification and salvage -(<a href="../api_reference/C/dbverify.html" class="olink">DB->verify()</a>), flushing to a backing file (<a href="../api_reference/C/dbsync.html" class="olink">DB->sync()</a>), -and association of secondary indices (<a href="../api_reference/C/dbassociate.html" class="olink">DB->associate()</a>). Database -handles are eventually closed using <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a>.</p> - <p>For more information on the access method operations supported by the database handle, see the <a href="../api_reference/C/db.html#dblist" class="olink">Database and Related Methods</a> section in the <em class="citetitle">Berkeley DB C API Reference Guide.</em> </p> + <p> + Once a database handle has been created using <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a>, + there are several standard access method operations. Each of + these operations is performed using a method referred to by + the returned handle. Generally, the database will be opened + using <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a>. If the database is from an old release of + Berkeley DB, it may need to be upgraded to the current release + before it is opened using <a href="../api_reference/C/dbupgrade.html" class="olink">DB->upgrade()</a>. + </p> + <p> + Once a database has been opened, records may be retrieved + (<a href="../api_reference/C/dbget.html" class="olink">DB->get()</a>), stored (<a href="../api_reference/C/dbput.html" class="olink">DB->put()</a>), and deleted (<a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a>). + </p> + <p> + Additional operations supported by the database handle + include statistics (<a href="../api_reference/C/dbstat.html" class="olink">DB->stat()</a>), truncation (<a href="../api_reference/C/dbtruncate.html" class="olink">DB->truncate()</a>), + version upgrade (<a href="../api_reference/C/dbupgrade.html" class="olink">DB->upgrade()</a>), verification and salvage + (<a href="../api_reference/C/dbverify.html" class="olink">DB->verify()</a>), flushing to a backing file (<a href="../api_reference/C/dbsync.html" class="olink">DB->sync()</a>), and + association of secondary indices (<a href="../api_reference/C/dbassociate.html" class="olink">DB->associate()</a>). Database + handles are eventually closed using <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a>. + </p> + <p> + For more information on the access method operations + supported by the database handle, see the <a href="../api_reference/C/db.html#dblist" class="olink">Database and Related + Methods</a> section in the + <em class="citetitle">Berkeley DB C API Reference Guide.</em> + </p> <div class="sect1" lang="en" xml:lang="en"> <div class="titlepage"> <div> @@ -232,7 +246,8 @@ handles are eventually closed using <a href="../api_reference/C/dbclose.html" cl </div> </div> <p> - The <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> method opens a database, and takes five arguments: + The <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> method opens a database, and takes five + arguments: </p> <div class="variablelist"> <dl> @@ -240,7 +255,7 @@ handles are eventually closed using <a href="../api_reference/C/dbclose.html" cl <span class="term">file</span> </dt> <dd> - The name of the file to be opened. + The name of the file to be opened. </dd> <dt> <span class="term">database</span> @@ -251,23 +266,24 @@ handles are eventually closed using <a href="../api_reference/C/dbclose.html" cl <dt> <span class="term">type</span> </dt> - <dd> - The type of database to open. This value will be one of - the five access methods Berkeley DB supports: DB_BTREE, - DB_HASH, DB_HEAP, DB_QUEUE or DB_RECNO, or the special - value DB_UNKNOWN, which allows you to open an existing file - without knowing its type. + <dd> + The type of database to open. This value will + be one of the five access methods Berkeley DB + supports: DB_BTREE, DB_HASH, DB_HEAP, DB_QUEUE or + DB_RECNO, or the special value DB_UNKNOWN, which + allows you to open an existing file without knowing + its type. </dd> <dt> <span class="term">mode</span> </dt> - <dd> + <dd> The permissions to give to any created file. </dd> </dl> </div> - <p> - There are a few flags that you can set to customize open: + <p> + There are a few flags that you can set to customize open: </p> <div class="variablelist"> <dl> @@ -276,8 +292,9 @@ handles are eventually closed using <a href="../api_reference/C/dbclose.html" cl <a href="../api_reference/C/dbopen.html#open_DB_CREATE" class="olink">DB_CREATE</a> </span> </dt> - <dd> - Create the underlying database and any necessary physical files. + <dd> + Create the underlying database and any + necessary physical files. </dd> <dt> <span class="term"> @@ -301,8 +318,9 @@ handles are eventually closed using <a href="../api_reference/C/dbclose.html" cl </span> </dt> <dd> - The returned handle is free-threaded, that is, it can be - used simultaneously by multiple threads within the process. + The returned handle is free-threaded, that is, + it can be used simultaneously by multiple threads + within the process. </dd> <dt> <span class="term"> @@ -310,19 +328,19 @@ handles are eventually closed using <a href="../api_reference/C/dbclose.html" cl </span> </dt> <dd> - Physically truncate the underlying database file, - discarding all databases it contained. Underlying - filesystem primitives are used to implement this flag. For - this reason it is only applicable to the physical file and - cannot be used to discard individual databases from within - physical files. + Physically truncate the underlying database + file, discarding all databases it contained. + Underlying filesystem primitives are used to implement + this flag. For this reason it is only applicable to + the physical file and cannot be used to discard + individual databases from within physical files. </dd> <dt> <span class="term"> <a href="../api_reference/C/dbset_feedback.html#set_feedback_DB_UPGRADE" class="olink">DB_UPGRADE</a> </span> </dt> - <dd> + <dd> Upgrade the database format as necessary. </dd> </dl> @@ -342,7 +360,8 @@ handles are eventually closed using <a href="../api_reference/C/dbclose.html" cl <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Opening multiple databases in a single file</td> + <td width="40%" align="right" valign="top"> Opening multiple databases in a + single file</td> </tr> </table> </div> diff --git a/docs/programmer_reference/am_close.html b/docs/programmer_reference/am_close.html index 3cc0a30e..50a7a1de 100644 --- a/docs/programmer_reference/am_close.html +++ b/docs/programmer_reference/am_close.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_sync.html">Prev</a> </td> - <th width="60%" align="center">Chapter 3. - Access Method Operations - </th> + <th width="60%" align="center">Chapter 3. Access Method Operations </th> <td width="20%" align="right"> <a accesskey="n" href="am_second.html">Next</a></td> </tr> </table> @@ -38,10 +36,15 @@ </div> </div> </div> - <p>The <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> database handle closes the <a href="../api_reference/C/db.html" class="olink">DB</a> handle. By -default, <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> also flushes all modified records from the -database cache to disk.</p> - <p>There is one flag that you can set to customize <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a>:</p> + <p> + The <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> database handle closes the <a href="../api_reference/C/db.html" class="olink">DB</a> handle. By + default, <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> also flushes all modified records from the + database cache to disk. + </p> + <p> + There is one flag that you can set to customize + <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a>: + </p> <div class="variablelist"> <dl> <dt> @@ -49,25 +52,43 @@ database cache to disk.</p> <a href="../api_reference/C/dbclose.html#dbclose_DB_NOSYNC" class="olink">DB_NOSYNC</a> </span> </dt> - <dd>Do not flush cached information to disk.</dd> + <dd> + Do not flush cached information to + disk. + </dd> </dl> </div> <span class="bold"> - <strong>It is important to understand that flushing cached information -to disk only minimizes the window of opportunity for corrupted data, it -does not eliminate the possibility.</strong> + <strong> + It is important to understand that flushing + cached information to disk only minimizes the window of + opportunity for corrupted data, it does not eliminate the + possibility. + </strong> </span> - <p>While unlikely, it is possible for database corruption to happen if a -system or application crash occurs while writing data to the database. To -ensure that database corruption never occurs, applications must either:</p> + <p> + While unlikely, it is possible for database corruption to + happen if a system or application crash occurs while writing + data to the database. To ensure that database corruption never + occurs, applications must either: + </p> <div class="itemizedlist"> <ul type="disc"> - <li>Use transactions and logging with automatic recovery.</li> - <li>Use logging and application-specific recovery.</li> - <li>Edit a copy of the database, and, once all applications -using the database have successfully called <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a>, use -system operations (for example, the POSIX rename system call) to -atomically replace the original database with the updated copy.</li> + <li> + Use transactions and logging with automatic + recovery. + </li> + <li> + Use logging and application-specific + recovery. + </li> + <li> + Edit a copy of the database, and, once all + applications using the database have successfully called + <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a>, use system operations (for example, the POSIX + rename system call) to atomically replace the original + database with the updated copy. + </li> </ul> </div> </div> diff --git a/docs/programmer_reference/am_conf.html b/docs/programmer_reference/am_conf.html index 1e297bd4..c041c004 100644 --- a/docs/programmer_reference/am_conf.html +++ b/docs/programmer_reference/am_conf.html @@ -14,13 +14,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 2. - Access Method Configuration - </th> + <th colspan="3" align="center">Chapter 2. Access Method Configuration </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="intro_products.html">Prev</a> </td> @@ -34,9 +32,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="am_conf"></a>Chapter 2. - Access Method Configuration - </h2> + <h2 class="title"><a id="am_conf"></a>Chapter 2. Access Method Configuration </h2> </div> </div> </div> @@ -47,36 +43,35 @@ <dl> <dt> <span class="sect1"> - <a href="am_conf.html#am_conf_intro"> - What are the available access methods? - </a> + <a href="am_conf.html#am_conf_intro"> What are the available access + methods? </a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="am_conf.html#idm2161896">Btree</a> + <a href="am_conf.html#idm2727912">Btree</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_conf.html#idp32168">Hash</a> + <a href="am_conf.html#idm1674312">Hash</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_conf.html#idm2680320">Heap</a> + <a href="am_conf.html#idm1985432">Heap</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_conf.html#idm2335248">Queue</a> + <a href="am_conf.html#idm1872368">Queue</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_conf.html#idm1801904">Recno</a> + <a href="am_conf.html#idm1924872">Recno</a> </span> </dt> </dl> @@ -90,17 +85,17 @@ <dl> <dt> <span class="sect2"> - <a href="am_conf_select.html#idm1772384">Btree or Heap?</a> + <a href="am_conf_select.html#idm2274400">Btree or Heap?</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_conf_select.html#idm2622384">Hash or Btree?</a> + <a href="am_conf_select.html#idm2915784">Hash or Btree?</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_conf_select.html#idm1789184">Queue or Recno?</a> + <a href="am_conf_select.html#idm2732704">Queue or Recno?</a> </span> </dt> </dl> @@ -139,7 +134,8 @@ </dt> <dt> <span class="sect2"> - <a href="general_am_conf.html#am_conf_malloc">Non-local memory allocation</a> + <a href="general_am_conf.html#am_conf_malloc">Non-local memory + allocation</a> </span> </dt> </dl> @@ -158,7 +154,8 @@ </dt> <dt> <span class="sect2"> - <a href="bt_conf.html#am_conf_bt_prefix">Btree prefix comparison</a> + <a href="bt_conf.html#am_conf_bt_prefix">Btree prefix + comparison</a> </span> </dt> <dt> @@ -180,7 +177,8 @@ </dd> <dt> <span class="sect1"> - <a href="hash_conf.html">Hash access method specific configuration</a> + <a href="hash_conf.html">Hash access method specific + configuration</a> </span> </dt> <dd> @@ -204,7 +202,8 @@ </dd> <dt> <span class="sect1"> - <a href="heap_conf.html">Heap access method specific configuration</a> + <a href="heap_conf.html">Heap access method specific + configuration</a> </span> </dt> <dt> @@ -216,22 +215,26 @@ <dl> <dt> <span class="sect2"> - <a href="rq_conf.html#am_conf_recno">Managing record-based databases</a> + <a href="rq_conf.html#am_conf_recno">Managing record-based + databases</a> </span> </dt> <dt> <span class="sect2"> - <a href="rq_conf.html#am_conf_extentsize">Selecting a Queue extent size</a> + <a href="rq_conf.html#am_conf_extentsize">Selecting a Queue extent + size</a> </span> </dt> <dt> <span class="sect2"> - <a href="rq_conf.html#am_conf_re_source">Flat-text backing files</a> + <a href="rq_conf.html#am_conf_re_source">Flat-text backing + files</a> </span> </dt> <dt> <span class="sect2"> - <a href="rq_conf.html#am_conf_renumber">Logically renumbering records</a> + <a href="rq_conf.html#am_conf_renumber">Logically renumbering + records</a> </span> </dt> </dl> @@ -242,9 +245,8 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="am_conf_intro"></a> - What are the available access methods? - </h2> + <h2 class="title" style="clear: both"><a id="am_conf_intro"></a> What are the available access + methods? </h2> </div> </div> </div> @@ -252,123 +254,128 @@ <dl> <dt> <span class="sect2"> - <a href="am_conf.html#idm2161896">Btree</a> + <a href="am_conf.html#idm2727912">Btree</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_conf.html#idp32168">Hash</a> + <a href="am_conf.html#idm1674312">Hash</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_conf.html#idm2680320">Heap</a> + <a href="am_conf.html#idm1985432">Heap</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_conf.html#idm2335248">Queue</a> + <a href="am_conf.html#idm1872368">Queue</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_conf.html#idm1801904">Recno</a> + <a href="am_conf.html#idm1924872">Recno</a> </span> </dt> </dl> </div> - <p> - Berkeley DB currently offers five access methods: Btree, Hash, Heap, Queue and Recno. + <p> + Berkeley DB currently offers five access methods: Btree, + Hash, Heap, Queue and Recno. </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm2161896"></a>Btree</h3> + <h3 class="title"><a id="idm2727912"></a>Btree</h3> </div> </div> </div> - <p> - The Btree access method is an implementation of a sorted, balanced - tree structure. Searches, insertions, and deletions in the tree - all take <span class="emphasis"><em>O(height)</em></span> time, where - <span class="emphasis"><em>height</em></span> is the number of levels in the - Btree from the root to the leaf pages. The upper bound on the - height is <span class="emphasis"><em>log base_b N</em></span>, where - <span class="emphasis"><em>base_b</em></span> is the smallest number of keys on a - page, and <span class="emphasis"><em>N</em></span> is the total number of keys - stored. + <p> + The Btree access method is an implementation of a + sorted, balanced tree structure. Searches, insertions, and + deletions in the tree all take + <span class="emphasis"><em>O(height)</em></span> time, where + <span class="emphasis"><em>height</em></span> is the number of levels in + the Btree from the root to the leaf pages. The upper bound + on the height is <span class="emphasis"><em>log base_b N</em></span>, where + <span class="emphasis"><em>base_b</em></span> is the smallest number of + keys on a page, and <span class="emphasis"><em>N</em></span> is the total + number of keys stored. </p> - <p> - Inserting unordered data into a Btree can result in pages that - are only half-full. DB makes ordered (or inverse ordered) - insertion the best case, resulting in nearly full-page space - utilization. + <p> + Inserting unordered data into a Btree can result in + pages that are only half-full. DB makes ordered (or + inverse ordered) insertion the best case, resulting in + nearly full-page space utilization. </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp32168"></a>Hash</h3> + <h3 class="title"><a id="idm1674312"></a>Hash</h3> </div> </div> </div> <p> - The Hash access method data structure is an implementation of - Extended Linear Hashing, as described in "Linear Hashing: A New - Tool for File and Table Addressing", Witold Litwin, - <span class="emphasis"><em>Proceedings of the 6th International Conference on Very - Large Databases (VLDB)</em></span>, 1980. + The Hash access method data structure is an + implementation of Extended Linear Hashing, as described in + "Linear Hashing: A New Tool for File and Table + Addressing", Witold Litwin, <span class="emphasis"><em>Proceedings of the + 6th International Conference on Very Large Databases + (VLDB)</em></span>, 1980. </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm2680320"></a>Heap</h3> + <h3 class="title"><a id="idm1985432"></a>Heap</h3> </div> </div> </div> <p> - The Heap access method stores records in a heap file. Records - are referenced solely by the page and offset at which they are - written. Because records are written in a heap file, compaction - is not necessary when deleting records, which allows for more - efficient use of space than if Btree is in use. The Heap access - method is intended for platforms with constrained disk space, - especially if those systems are performing a great many record - creation and deletions. + The Heap access method stores records in a heap file. + Records are referenced solely by the page and offset at + which they are written. Because records are written in a + heap file, compaction is not necessary when deleting + records, which allows for more efficient use of space than + if Btree is in use. The Heap access method is intended for + platforms with constrained disk space, especially if those + systems are performing a great many record creation and + deletions. </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm2335248"></a>Queue</h3> + <h3 class="title"><a id="idm1872368"></a>Queue</h3> </div> </div> </div> - <p> - The Queue access method stores fixed-length records with logical - record numbers as keys. It is designed for fast inserts at the - tail and has a special cursor consume operation that deletes and - returns a record from the head of the queue. The Queue access - method uses record level locking. + <p> + The Queue access method stores fixed-length records + with logical record numbers as keys. It is designed for + fast inserts at the tail and has a special cursor consume + operation that deletes and returns a record from the head + of the queue. The Queue access method uses record level + locking. </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm1801904"></a>Recno</h3> + <h3 class="title"><a id="idm1924872"></a>Recno</h3> </div> </div> </div> <p> - The Recno access method stores both fixed and variable-length - records with logical record numbers as keys, optionally backed by a - flat text (byte stream) file. + The Recno access method stores both fixed and + variable-length records with logical record numbers as + keys, optionally backed by a flat text (byte stream) file. </p> </div> </div> diff --git a/docs/programmer_reference/am_conf_logrec.html b/docs/programmer_reference/am_conf_logrec.html index 9f72bcc5..68906418 100644 --- a/docs/programmer_reference/am_conf_logrec.html +++ b/docs/programmer_reference/am_conf_logrec.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_conf_select.html">Prev</a> </td> - <th width="60%" align="center">Chapter 2. - Access Method Configuration - </th> + <th width="60%" align="center">Chapter 2. Access Method Configuration </th> <td width="20%" align="right"> <a accesskey="n" href="general_am_conf.html">Next</a></td> </tr> </table> @@ -38,49 +36,66 @@ </div> </div> </div> - <p>The Berkeley DB Btree, Queue and Recno access methods can operate on logical -record numbers. Record numbers are 1-based, not 0-based, that is, the -first record in a database is record number 1.</p> - <p>In all cases for the Queue and Recno access methods, and when calling -the Btree access method using the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> and <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> methods -with the <a href="../api_reference/C/dbget.html#dbget_DB_SET_RECNO" class="olink">DB_SET_RECNO</a> flag specified, the <span class="bold"><strong>data</strong></span> field of -the key <a href="../api_reference/C/dbt.html" class="olink">DBT</a> must be a pointer to a memory location of type -<span class="bold"><strong>db_recno_t</strong></span>, as typedef'd in the standard Berkeley DB include file. -The <span class="bold"><strong>size</strong></span> field of the key <a href="../api_reference/C/dbt.html" class="olink">DBT</a> should be the size of that -type (for example, "sizeof(db_recno_t)" in the C programming language). -The <span class="bold"><strong>db_recno_t</strong></span> type is a 32-bit unsigned type, which limits the -number of logical records in a Queue or Recno database, and the maximum -logical record which may be directly retrieved from a Btree database, -to 4,294,967,295.</p> - <p>Record numbers in Recno databases can be configured to run in either -mutable or fixed mode: mutable, where logical record numbers change as -records are deleted or inserted, and fixed, where record numbers never -change regardless of the database operation. Record numbers in Queue -databases are always fixed, and never change regardless of the database -operation. Record numbers in Btree databases are always mutable, and -as records are deleted or inserted, the logical record number for other -records in the database can change. See -<a class="xref" href="rq_conf.html#am_conf_renumber" title="Logically renumbering records">Logically renumbering records</a> -for more information.</p> - <p>When appending new data items into Queue databases, record numbers wrap -around. When the tail of the queue reaches the maximum record number, -the next record appended will be given record number 1. If the head of -the queue ever catches up to the tail of the queue, Berkeley DB will return -the system error EFBIG. Record numbers do not wrap around when appending -new data items into Recno databases.</p> - <p>Configuring Btree databases to support record numbers can severely limit -the throughput of applications with multiple concurrent threads writing -the database, because locations used to store record counts often become -hot spots that many different threads all need to update. In the case -of a Btree supporting duplicate data items, the logical record number -refers to a key and all of its data items, as duplicate data items are -not individually numbered.</p> - <p>The following is an example function that reads records from standard -input and stores them into a Recno database. The function then uses a -cursor to step through the database and display the stored records.</p> + <p> + The Berkeley DB Btree, Queue and Recno access methods can + operate on logical record numbers. Record numbers are 1-based, + not 0-based, that is, the first record in a database is record + number 1. + </p> + <p> + In all cases for the Queue and Recno access methods, and + when calling the Btree access method using the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> and + <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> methods with the <a href="../api_reference/C/dbget.html#dbget_DB_SET_RECNO" class="olink">DB_SET_RECNO</a> flag specified, + the <span class="bold"><strong>data</strong></span> field of the key + <a href="../api_reference/C/dbt.html" class="olink">DBT</a> must be a pointer to a memory location of type <span class="bold"><strong>db_recno_t</strong></span>, as typedef'd in the + standard Berkeley DB include file. The <span class="bold"><strong>size + </strong></span> field of the key <a href="../api_reference/C/dbt.html" class="olink">DBT</a> should be the size + of that type (for example, "sizeof(db_recno_t)" in the C + programming language). The <span class="bold"><strong>db_recno_t</strong></span> + type is a 32-bit unsigned type, which limits the number of logical + records in a Queue or Recno database, and the maximum logical record + which may be directly retrieved from a Btree database, to 4,294,967,295. + </p> + <p> + Record numbers in Recno databases can be configured to run + in either mutable or fixed mode: mutable, where logical record + numbers change as records are deleted or inserted, and fixed, + where record numbers never change regardless of the database + operation. Record numbers in Queue databases are always fixed, + and never change regardless of the database operation. Record + numbers in Btree databases are always mutable, and as records + are deleted or inserted, the logical record number for other + records in the database can change. See <a class="xref" href="rq_conf.html#am_conf_renumber" title="Logically renumbering records">Logically renumbering + records</a> for more + information. + </p> + <p> + When appending new data items into Queue databases, record + numbers wrap around. When the tail of the queue reaches the + maximum record number, the next record appended will be given + record number 1. If the head of the queue ever catches up to + the tail of the queue, Berkeley DB will return the system + error EFBIG. Record numbers do not wrap around when appending + new data items into Recno databases. + </p> + <p> + Configuring Btree databases to support record numbers can + severely limit the throughput of applications with multiple + concurrent threads writing the database, because locations + used to store record counts often become hot spots that many + different threads all need to update. In the case of a Btree + supporting duplicate data items, the logical record number + refers to a key and all of its data items, as duplicate data + items are not individually numbered. + </p> + <p> + The following is an example function that reads records from + standard input and stores them into a Recno database. The + function then uses a cursor to step through the database and + display the stored records. + </p> <a id="prog_am1"></a> - <pre class="programlisting"> -int + <pre class="programlisting">int recno_build(DB *dbp) { DBC *dbcp; diff --git a/docs/programmer_reference/am_conf_select.html b/docs/programmer_reference/am_conf_select.html index 6f764410..02101484 100644 --- a/docs/programmer_reference/am_conf_select.html +++ b/docs/programmer_reference/am_conf_select.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_conf.html">Prev</a> </td> - <th width="60%" align="center">Chapter 2. - Access Method Configuration - </th> + <th width="60%" align="center">Chapter 2. Access Method Configuration </th> <td width="20%" align="right"> <a accesskey="n" href="am_conf_logrec.html">Next</a></td> </tr> </table> @@ -42,248 +40,261 @@ <dl> <dt> <span class="sect2"> - <a href="am_conf_select.html#idm1772384">Btree or Heap?</a> + <a href="am_conf_select.html#idm2274400">Btree or Heap?</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_conf_select.html#idm2622384">Hash or Btree?</a> + <a href="am_conf_select.html#idm2915784">Hash or Btree?</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_conf_select.html#idm1789184">Queue or Recno?</a> + <a href="am_conf_select.html#idm2732704">Queue or Recno?</a> </span> </dt> </dl> </div> - <p> - The Berkeley DB access method implementation unavoidably interacts with - each application's data set, locking requirements and data access - patterns. For this reason, one access method may result in - dramatically better performance for an application than another one. - Applications whose data could be stored using more than one access - method may want to benchmark their performance using the different - candidates. + <p> + The Berkeley DB access method implementation unavoidably + interacts with each application's data set, locking + requirements and data access patterns. For this reason, one + access method may result in dramatically better performance + for an application than another one. Applications whose data + could be stored using more than one access method may want to + benchmark their performance using the different candidates. </p> <p> - One of the strengths of Berkeley DB is that it provides multiple access - methods with nearly identical interfaces to the different access - methods. This means that it is simple to modify an application to use - a different access method. Applications can easily benchmark the - different Berkeley DB access methods against each other for their - particular data set and access pattern. + One of the strengths of Berkeley DB is that it provides + multiple access methods with nearly identical interfaces to + the different access methods. This means that it is simple to + modify an application to use a different access method. + Applications can easily benchmark the different Berkeley DB + access methods against each other for their particular data + set and access pattern. </p> <p> - Most applications choose between using the Btree or Heap access - methods, between Btree or Hash, or between Queue and Recno, because - each of these pairs offer similar functionality. + Most applications choose between using the Btree or Heap + access methods, between Btree or Hash, or between Queue and + Recno, because each of these pairs offer similar + functionality. </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm1772384"></a>Btree or Heap?</h3> + <h3 class="title"><a id="idm2274400"></a>Btree or Heap?</h3> </div> </div> </div> - <p> - Most applications use Btree because it performs well for most - general-purpose database workloads. But there are - circumstances where Heap is the better choice. This section - describes the differences between the two access methods so - that you can better understand when Heap might be the superior - choice for your application. + <p> + Most applications use Btree because it performs well + for most general-purpose database workloads. But there are + circumstances where Heap is the better choice. This + section describes the differences between the two access + methods so that you can better understand when Heap might + be the superior choice for your application. </p> - <p> + <p> Before continuing, it is helpful to have a high level - understanding of the operating differences between Btree and - Heap. + understanding of the operating differences between Btree + and Heap. </p> <div class="sect3" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idm1549752"></a>Disk Space Usage</h4> + <h4 class="title"><a id="idm2436344"></a>Disk Space Usage</h4> </div> </div> </div> <p> - The Heap access method was developed for use in systems - with constrained disk space (such as an embedded system). - Because of the way it reuses page space, for some workloads - it can be much better than Btree on disk space usage - because it will not grow the on-disk database file as fast - as Btree. Of course, this assumes that your application is - characterized by a roughly equal number of record creations - and deletions. + The Heap access method was developed for use in + systems with constrained disk space (such as an + embedded system). Because of the way it reuses page + space, for some workloads it can be much better than + Btree on disk space usage because it will not grow the + on-disk database file as fast as Btree. Of course, + this assumes that your application is characterized by + a roughly equal number of record creations and + deletions. </p> <p> - Also, Heap can actively control the space used by the - database with the use of the <a href="../api_reference/C/dbset_heapsize.html" class="olink">DB->set_heapsize()</a> method. When - the limit specified by that method is reached, no - additional pages will be allocated and existing pages will - be aggressively searched for free space. Also records in - the heap can be split to fill space on two or more pages. + Also, Heap can actively control the space used by + the database with the use of the <a href="../api_reference/C/dbset_heapsize.html" class="olink">DB->set_heapsize()</a> + method. When the limit specified by that method is + reached, no additional pages will be allocated and + existing pages will be aggressively searched for free + space. Also records in the heap can be split to fill + space on two or more pages. </p> </div> <div class="sect3" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idm1572504"></a>Record Access</h4> + <h4 class="title"><a id="idm2289136"></a>Record Access</h4> </div> </div> </div> - <p> - Btree and Heap are fundamentally different because of the - way that you access records in them. In Btree, you access a - record by using the record's key. This lookup occurs fairly - quickly because Btree places records in the database - according to a pre-defined sorting order. Your application - is responsible for constructing the key, which means that - it is relatively easy for your application to know what key - is in use by any given record. + <p> + Btree and Heap are fundamentally different because + of the way that you access records in them. In Btree, + you access a record by using the record's key. This + lookup occurs fairly quickly because Btree places + records in the database according to a pre-defined + sorting order. Your application is responsible for + constructing the key, which means that it is + relatively easy for your application to know what key + is in use by any given record. </p> - <p> - Conversely, Heap accesses records based on their offset - location within the database. You retrieve a record in a - Heap database using the record's Record ID (RID), which is - created when the record is added to the database. The RID - is created for you; you cannot specify this yourself. - Because the RID is created for you, your application does - not have control over the key value. For this reason, - retrieval operations for a Heap database are usually - performed using secondary databases. You can then use this - secondary index to retrieve records stored in your Heap - database. + <p> + Conversely, Heap accesses records based on their + offset location within the database. You retrieve a + record in a Heap database using the record's Record ID + (RID), which is created when the record is added to + the database. The RID is created for you; you cannot + specify this yourself. Because the RID is created for + you, your application does not have control over the + key value. For this reason, retrieval operations for a + Heap database are usually performed using secondary + databases. You can then use this secondary index to + retrieve records stored in your Heap database. </p> <p> Note that an application's data access requirements - grow complex, Btree databases also frequently - require secondary databases. So at a certain level - of complexity you will be using secondary databases - regardless of the access method that you choose. + grow complex, Btree databases also frequently require + secondary databases. So at a certain level of + complexity you will be using secondary databases + regardless of the access method that you choose. </p> <p> - Secondary databases are described in - <a class="xref" href="am_second.html" title="Secondary indexes">Secondary indexes</a>. + Secondary databases are described in <a class="xref" href="am_second.html" title="Secondary indexes">Secondary indexes</a>. </p> </div> <div class="sect3" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idm2352312"></a>Record Creation/Deletion</h4> + <h4 class="title"><a id="idm2734232"></a>Record Creation/Deletion</h4> </div> </div> </div> - <p> - When Btree creates a new record, it places the record on - the database page that is appropriate for the sorting order - in use by the database. If Btree can not find a page to put - the new record on, it locates a page that is in the proper - location for the new record, splits it so that the existing - records are divided between the two pages, and then adds - the new record to the appropriate page. + <p> + When Btree creates a new record, it places the + record on the database page that is appropriate for + the sorting order in use by the database. If Btree can + not find a page to put the new record on, it locates a + page that is in the proper location for the new + record, splits it so that the existing records are + divided between the two pages, and then adds the new + record to the appropriate page. </p> - <p> - On deletion, Btree simply removes the deleted record from - whatever page it is stored on. This leaves some amount of - unused space ("holes") on the page. Only new records that - sort to this page can fill that space. However, once a page - is completely empty, it can be reused to hold records with - a different sort value. + <p> + On deletion, Btree simply removes the deleted + record from whatever page it is stored on. This leaves + some amount of unused space ("holes") on the page. + Only new records that sort to this page can fill that + space. However, once a page is completely empty, it + can be reused to hold records with a different sort + value. </p> - <p> - In order to reclaim unused disk space, you must run the - <a href="../api_reference/C/dbcompact.html" class="olink">DB->compact()</a> method, which attempts to fill holes in - existing pages by moving records from other pages. If it is - successful in moving enough records, it might be left with - entire pages that have no data on them. In this event, the - unused pages might be removed from the database (depending - on the flags that you provide to <a href="../api_reference/C/dbcompact.html" class="olink">DB->compact()</a>), which causes - the database file to be reduced in size. + <p> + In order to reclaim unused disk space, you must run + the <a href="../api_reference/C/dbcompact.html" class="olink">DB->compact()</a> method, which attempts to fill holes + in existing pages by moving records from other pages. + If it is successful in moving enough records, it might + be left with entire pages that have no data on them. + In this event, the unused pages might be removed from + the database (depending on the flags that you provide + to <a href="../api_reference/C/dbcompact.html" class="olink">DB->compact()</a>), which causes the database file to be + reduced in size. </p> <p> - Both tree searching and page compaction are relatively - expensive operations. Heap avoids these operations, and so - is able to perform better under some circumstances. + Both tree searching and page compaction are + relatively expensive operations. Heap avoids these + operations, and so is able to perform better under + some circumstances. </p> <p> - Heap does not care about record order. When a record is - created in a Heap database, it is placed on the first page - that has space to store the record. No sorting is involved, - and so the overhead from record sorting is removed. + Heap does not care about record order. When a + record is created in a Heap database, it is placed on + the first page that has space to store the record. No + sorting is involved, and so the overhead from record + sorting is removed. </p> <p> - On deletion, both Btree and Heap free space within a page - when a record is deleted. However, unlike Btree, Heap has - no compaction operation, nor does it have to wait for a - record with the proper sort order to fill a hole on a page. - Instead, Heap simply reuses empty page space whenever any - record is added that will fit into the space. + On deletion, both Btree and Heap free space within + a page when a record is deleted. However, unlike + Btree, Heap has no compaction operation, nor does it + have to wait for a record with the proper sort order + to fill a hole on a page. Instead, Heap simply reuses + empty page space whenever any record is added that + will fit into the space. </p> </div> <div class="sect3" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idm1962568"></a>Cursor Operations</h4> + <h4 class="title"><a id="idm1207960"></a>Cursor Operations</h4> </div> </div> </div> <p> - When considering Heap, be aware that this access method - does not support the full range of cursor operations that - Btree does. + When considering Heap, be aware that this access + method does not support the full range of cursor + operations that Btree does. </p> <div class="itemizedlist"> <ul type="disc"> <li> <p> - On sequential cursor scans of the database, the - retrieval order of the records is not predictable - for Heap because the records are not sorted. Btree, - of course, sorts its records so the retrieval order - is predictable. + On sequential cursor scans of the database, + the retrieval order of the records is not + predictable for Heap because the records are + not sorted. Btree, of course, sorts its + records so the retrieval order is predictable. </p> </li> <li> - <p> - When using a Heap database, you cannot create new - records using a cursor. Also, this means - that Heap does not support the <a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a> - <code class="literal">DB_AFTER</code> and - <code class="literal">DB_BEFORE</code> flags. - You can, however, update existing records using a - cursor. + <p> + When using a Heap database, you cannot + create new records using a cursor. Also, this + means that Heap does not support the <a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a> + <code class="literal">DB_AFTER</code> and + <code class="literal">DB_BEFORE</code> flags. You + can, however, update existing records using a + cursor. </p> </li> <li> <p> - For concurrent applications, iterating through the records in a - Heap database is not recommended due to performance - considerations. This is because there is a good - chance that there are a lot of empty pages in the - database if you have a concurrent application. + For concurrent applications, iterating + through the records in a Heap database is not + recommended due to performance considerations. + This is because there is a good chance that + there are a lot of empty pages in the database + if you have a concurrent application. </p> <p> - For a Heap database, entire regions are locked when - a lock is acquired for a database page. If there is - then contention for that region, and a new database - page needs to be added, then Berkeley DB simply - creates a whole new region. The locked region is - then padded with empty pages in order to reach the - new region. + For a Heap database, entire regions are + locked when a lock is acquired for a database + page. If there is then contention for that + region, and a new database page needs to be + added, then Berkeley DB simply creates a whole + new region. The locked region is then padded + with empty pages in order to reach the new + region. </p> <p> - The result is that if the last used page in a - region is 10, and a new region is created at page - 100, then there are empty pages from 11 to 99. If - you are iterating with a cursor, then all those - empty pages must be examined by the cursor before - it can reach the data at page 100. + The result is that if the last used page in + a region is 10, and a new region is created at + page 100, then there are empty pages from 11 + to 99. If you are iterating with a cursor, + then all those empty pages must be examined by + the cursor before it can reach the data at + page 100. </p> </li> </ul> @@ -293,81 +304,85 @@ <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idm2036040"></a>Which Access Method Should You Use?</h4> + <h4 class="title"><a id="idm2302168"></a>Which Access Method Should You Use?</h4> </div> </div> </div> <p> - Ultimately, you can only determine which access method is - superior for your application through performance testing - using both access methods. To be effective, this - performance testing must use a production-equivalent - workload. + Ultimately, you can only determine which access + method is superior for your application through + performance testing using both access methods. To be + effective, this performance testing must use a + production-equivalent workload. </p> <p> - That said, there are a few times when you absolutely must - use Btree: + That said, there are a few times when you + absolutely must use Btree: </p> <div class="itemizedlist"> <ul type="disc"> <li> <p> - If you want to use bulk put and get operations. + If you want to use bulk put and get + operations. </p> </li> <li> - <p> - If having your database clustered on sort order is - important to you. + <p> + If having your database clustered on sort + order is important to you. </p> </li> <li> <p> - If you want to be able to create records using - cursors. + If you want to be able to create records + using cursors. </p> </li> <li> - <p> + <p> If you have multiple threads/processes - simultaneously creating new records, and you want - to be able to efficiently iterate over those - records using a cursor. + simultaneously creating new records, and you + want to be able to efficiently iterate over + those records using a cursor. </p> </li> </ul> </div> <p> - But beyond those limitations, there are some application - characteristics that should cause you to suspect that Heap - will work better for your application than Btree. They are: + But beyond those limitations, there are some + application characteristics that should cause you to + suspect that Heap will work better for your + application than Btree. They are: </p> <div class="itemizedlist"> <ul type="disc"> <li> - <p> - Your application will run in an environment with - constrained resources and you want to set a hard - limit on the size of the database file. + <p> + Your application will run in an environment + with constrained resources and you want to set + a hard limit on the size of the database file. </p> </li> <li> - <p> - You want to limit the disk space growth of your - database file, and your application performs a - roughly equivalent number of record creations and - deletions. + <p> + You want to limit the disk space growth of + your database file, and your application + performs a roughly equivalent number of record + creations and deletions. </p> </li> <li> <p> - Inserts into a Btree require sorting the new record - onto its proper page. This operation can require - multiple page reads. A Heap database can simply - reuse whatever empty page space it can find in the - cache. Insert-intensive applications will typically - find that Heap is much more efficient than Btree, - especially as the size of the database increases. + Inserts into a Btree require sorting the + new record onto its proper page. This + operation can require multiple page reads. A + Heap database can simply reuse whatever empty + page space it can find in the cache. + Insert-intensive applications will typically + find that Heap is much more efficient than + Btree, especially as the size of the database + increases. </p> </li> </ul> @@ -378,127 +393,135 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm2622384"></a>Hash or Btree?</h3> + <h3 class="title"><a id="idm2915784"></a>Hash or Btree?</h3> </div> </div> </div> - <p> - The Hash and Btree access methods should be used when logical - record numbers are not the primary key used for data access. - (If logical record numbers are a secondary key used for data - access, the Btree access method is a possible choice, as it - supports simultaneous access by a key and a record number.) + <p> + The Hash and Btree access methods should be used when + logical record numbers are not the primary key used for + data access. (If logical record numbers are a secondary + key used for data access, the Btree access method is a + possible choice, as it supports simultaneous access by a + key and a record number.) </p> - <p> - Keys in Btrees are stored in sorted order and the relationship - between them is defined by that sort order. For this reason, - the Btree access method should be used when there is any - locality of reference among keys. Locality of reference means - that accessing one particular key in the Btree implies that the - application is more likely to access keys near to the key being - accessed, where "near" is defined by the sort order. For - example, if keys are timestamps, and it is likely that a - request for an 8AM timestamp will be followed by a request for - a 9AM timestamp, the Btree access method is generally the right - choice. Or, for example, if the keys are names, and the - application will want to review all entries with the same last - name, the Btree access method is again a good choice. + <p> + Keys in Btrees are stored in sorted order and the + relationship between them is defined by that sort order. + For this reason, the Btree access method should be used + when there is any locality of reference among keys. + Locality of reference means that accessing one particular + key in the Btree implies that the application is more + likely to access keys near to the key being accessed, + where "near" is defined by the sort order. For example, if + keys are timestamps, and it is likely that a request for + an 8AM timestamp will be followed by a request for a 9AM + timestamp, the Btree access method is generally the right + choice. Or, for example, if the keys are names, and the + application will want to review all entries with the same + last name, the Btree access method is again a good choice. </p> <p> - There is little difference in performance between the Hash and - Btree access methods on small data sets, where all, or most of, - the data set fits into the cache. However, when a data set is - large enough that significant numbers of data pages no longer - fit into the cache, then the Btree locality of reference - described previously becomes important for performance reasons. - For example, there is no locality of reference for the Hash - access method, and so key "AAAAA" is as likely to be stored on - the same database page with key "ZZZZZ" as with key "AAAAB". - In the Btree access method, because items are sorted, key - "AAAAA" is far more likely to be near key "AAAAB" than key - "ZZZZZ". So, if the application exhibits locality of reference - in its data requests, then the Btree page read into the cache - to satisfy a request for key "AAAAA" is much more likely to be - useful to satisfy subsequent requests from the application than - the Hash page read into the cache to satisfy the same request. - This means that for applications with locality of reference, - the cache is generally much more effective for the Btree access - method than the Hash access method, and the Btree access method - will make many fewer I/O calls. + There is little difference in performance between the + Hash and Btree access methods on small data sets, where + all, or most of, the data set fits into the cache. + However, when a data set is large enough that significant + numbers of data pages no longer fit into the cache, then + the Btree locality of reference described previously + becomes important for performance reasons. For example, + there is no locality of reference for the Hash access + method, and so key "AAAAA" is as likely to be stored on + the same database page with key "ZZZZZ" as with key + "AAAAB". In the Btree access method, because items are + sorted, key "AAAAA" is far more likely to be near key + "AAAAB" than key "ZZZZZ". So, if the application exhibits + locality of reference in its data requests, then the Btree + page read into the cache to satisfy a request for key + "AAAAA" is much more likely to be useful to satisfy + subsequent requests from the application than the Hash + page read into the cache to satisfy the same request. This + means that for applications with locality of reference, + the cache is generally much more effective for the Btree + access method than the Hash access method, and the Btree + access method will make many fewer I/O calls. </p> - <p> - However, when a data set becomes even larger, the Hash access - method can outperform the Btree access method. The reason for - this is that Btrees contain more metadata pages than Hash - databases. The data set can grow so large that metadata pages - begin to dominate the cache for the Btree access method. If - this happens, the Btree can be forced to do an I/O for each - data request because the probability that any particular data - page is already in the cache becomes quite small. Because the - Hash access method has fewer metadata pages, its cache stays - "hotter" longer in the presence of large data sets. In - addition, once the data set is so large that both the Btree and - Hash access methods are almost certainly doing an I/O for each - random data request, the fact that Hash does not have to walk + <p> + However, when a data set becomes even larger, the Hash + access method can outperform the Btree access method. The + reason for this is that Btrees contain more metadata pages + than Hash databases. The data set can grow so large that + metadata pages begin to dominate the cache for the Btree + access method. If this happens, the Btree can be forced to + do an I/O for each data request because the probability + that any particular data page is already in the cache + becomes quite small. Because the Hash access method has + fewer metadata pages, its cache stays "hotter" longer in + the presence of large data sets. In addition, once the + data set is so large that both the Btree and Hash access + methods are almost certainly doing an I/O for each random + data request, the fact that Hash does not have to walk several internal pages as part of a key search becomes a - performance advantage for the Hash access method as well. + performance advantage for the Hash access method as well. </p> - <p> - Application data access patterns strongly affect all of these - behaviors, for example, accessing the data by walking a cursor - through the database will greatly mitigate the large data set - behavior describe above because each I/O into the cache will - satisfy a fairly large number of subsequent data - requests. + <p> + Application data access patterns strongly affect all of + these behaviors, for example, accessing the data by + walking a cursor through the database will greatly + mitigate the large data set behavior describe above + because each I/O into the cache will satisfy a fairly + large number of subsequent data requests. </p> - <p> - In the absence of information on application data and data - access patterns, for small data sets either the Btree or Hash - access methods will suffice. For data sets larger than the - cache, we normally recommend using the Btree access method. If - you have truly large data, then the Hash access method may be a - better choice. The <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility is a useful tool for monitoring - how well your cache is performing. + <p> + In the absence of information on application data and + data access patterns, for small data sets either the Btree + or Hash access methods will suffice. For data sets larger + than the cache, we normally recommend using the Btree + access method. If you have truly large data, then the Hash + access method may be a better choice. The <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility is a + useful tool for monitoring how well your cache is + performing. </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm1789184"></a>Queue or Recno?</h3> + <h3 class="title"><a id="idm2732704"></a>Queue or Recno?</h3> </div> </div> </div> - <p> - The Queue or Recno access methods should be used when logical - record numbers are the primary key used for data access. The - advantage of the Queue access method is that it performs record - level locking and for this reason supports significantly higher - levels of concurrency than the Recno access method. The - advantage of the Recno access method is that it supports a - number of additional features beyond those supported by the - Queue access method, such as variable-length records and - support for backing flat-text files. + <p> + The Queue or Recno access methods should be used when + logical record numbers are the primary key used for data + access. The advantage of the Queue access method is that + it performs record level locking and for this reason + supports significantly higher levels of concurrency than + the Recno access method. The advantage of the Recno access + method is that it supports a number of additional features + beyond those supported by the Queue access method, such as + variable-length records and support for backing flat-text + files. </p> <p> - Logical record numbers can be mutable or fixed: mutable, where - logical record numbers can change as records are deleted or - inserted, and fixed, where record numbers never change - regardless of the database operation. It is possible to store - and retrieve records based on logical record numbers in the - Btree access method. However, those record numbers are always - mutable, and as records are deleted or inserted, the logical - record number for other records in the database will change. - The Queue access method always runs in fixed mode, and logical + Logical record numbers can be mutable or fixed: + mutable, where logical record numbers can change as + records are deleted or inserted, and fixed, where record + numbers never change regardless of the database operation. + It is possible to store and retrieve records based on + logical record numbers in the Btree access method. + However, those record numbers are always mutable, and as + records are deleted or inserted, the logical record number + for other records in the database will change. The Queue + access method always runs in fixed mode, and logical record numbers never change regardless of the database - operation. The Recno access method can be configured to run in - either mutable or fixed mode. + operation. The Recno access method can be configured to + run in either mutable or fixed mode. </p> <p> - In addition, the Recno access method provides support for - databases whose permanent storage is a flat text file and the - database is used as a fast, temporary storage area while the - data is being read or modified. + In addition, the Recno access method provides support + for databases whose permanent storage is a flat text file + and the database is used as a fast, temporary storage area + while the data is being read or modified. </p> </div> </div> @@ -513,9 +536,7 @@ <td width="40%" align="right"> <a accesskey="n" href="am_conf_logrec.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 2. - Access Method Configuration - </td> + <td width="40%" align="left" valign="top">Chapter 2. Access Method Configuration </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/am_cursor.html b/docs/programmer_reference/am_cursor.html index 0d1fe075..c43d74b3 100644 --- a/docs/programmer_reference/am_cursor.html +++ b/docs/programmer_reference/am_cursor.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_foreign.html">Prev</a> </td> - <th width="60%" align="center">Chapter 3. - Access Method Operations - </th> + <th width="60%" align="center">Chapter 3. Access Method Operations </th> <td width="20%" align="right"> <a accesskey="n" href="am_misc.html">Next</a></td> </tr> </table> @@ -42,7 +40,8 @@ <dl> <dt> <span class="sect2"> - <a href="am_cursor.html#am_curget">Retrieving records with a cursor</a> + <a href="am_cursor.html#am_curget">Retrieving records with a + cursor</a> </span> </dt> <dt> @@ -77,57 +76,61 @@ </dt> </dl> </div> - <p> - A database cursor refers to a single key/data pair in the database. - It supports traversal of the database and is the only way to access - individual duplicate data items. Cursors are used for operating on - collections of records, for iterating over a database, and for - saving handles to individual records, so that they can be modified - after they have been read. + <p> + A database cursor refers to a single key/data pair in the + database. It supports traversal of the database and is the + only way to access individual duplicate data items. Cursors + are used for operating on collections of records, for + iterating over a database, and for saving handles to + individual records, so that they can be modified after they + have been read. </p> <p> - The <a href="../api_reference/C/dbcursor.html" class="olink">DB->cursor()</a> method opens a cursor into a database. Upon return - the cursor is uninitialized, cursor positioning occurs as part of - the first cursor operation. + The <a href="../api_reference/C/dbcursor.html" class="olink">DB->cursor()</a> method opens a cursor into a database. Upon + return the cursor is uninitialized, cursor positioning occurs + as part of the first cursor operation. </p> <p> - Once a database cursor has been opened, records may be retrieved - (<a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a>), stored (<a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a>), and deleted (<a href="../api_reference/C/dbcdel.html" class="olink">DBC->del()</a>). + Once a database cursor has been opened, records may be + retrieved (<a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a>), stored (<a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a>), and deleted + (<a href="../api_reference/C/dbcdel.html" class="olink">DBC->del()</a>). </p> - <p> - Additional operations supported by the cursor handle include - duplication (<a href="../api_reference/C/dbcdup.html" class="olink">DBC->dup()</a>), equality join (<a href="../api_reference/C/dbjoin.html" class="olink">DB->join()</a>), and a count of - duplicate data items (<a href="../api_reference/C/dbccount.html" class="olink">DBC->count()</a>). Cursors are eventually closed - using <a href="../api_reference/C/dbcclose.html" class="olink">DBC->close()</a>. + <p> + Additional operations supported by the cursor handle + include duplication (<a href="../api_reference/C/dbcdup.html" class="olink">DBC->dup()</a>), equality join (<a href="../api_reference/C/dbjoin.html" class="olink">DB->join()</a>), and + a count of duplicate data items (<a href="../api_reference/C/dbccount.html" class="olink">DBC->count()</a>). Cursors are + eventually closed using <a href="../api_reference/C/dbcclose.html" class="olink">DBC->close()</a>. </p> <p> - For more information on the operations supported by the cursor - handle, see the - <a href="../api_reference/C/dbc.html#dbclist" class="olink">Database Cursors and Related Methods</a> - section in the <em class="citetitle">Berkeley DB C API Reference Guide.</em> + For more information on the operations supported by the + cursor handle, see the <a href="../api_reference/C/dbc.html#dbclist" class="olink">Database Cursors and Related + Methods</a> section in the + <em class="citetitle">Berkeley DB C API Reference Guide.</em> </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="am_curget"></a>Retrieving records with a cursor</h3> + <h3 class="title"><a id="am_curget"></a>Retrieving records with a + cursor</h3> </div> </div> </div> <p> - The <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method retrieves records from the database using a - cursor. The <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method takes a flag which controls how the - cursor is positioned within the database and returns the key/data - item associated with that positioning. Similar to <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a>, - <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> may also take a supplied key and retrieve the data - associated with that key from the database. There are several - flags that you can set to customize retrieval. + The <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method retrieves records from the database + using a cursor. The <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method takes a flag which + controls how the cursor is positioned within the database and + returns the key/data item associated with that positioning. + Similar to <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a>, <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> may also take a supplied key and + retrieve the data associated with that key from the database. + There are several flags that you can set to customize + retrieval. </p> <div class="sect3" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idp894136"></a>Cursor position flags</h4> + <h4 class="title"><a id="idp413624"></a>Cursor position flags</h4> </div> </div> </div> @@ -136,43 +139,45 @@ <dt> <span class="term"><a href="../api_reference/C/dbcget.html#dbcget_DB_FIRST" class="olink">DB_FIRST</a>, <a href="../api_reference/C/dbcget.html#dbcget_DB_LAST" class="olink">DB_LAST</a></span> </dt> - <dd> - Return the first (last) record in the database. + <dd> + Return the first (last) record in the + database. </dd> <dt> <span class="term"><a href="../api_reference/C/dbcget.html#dbcget_DB_NEXT" class="olink">DB_NEXT</a>, <a href="../api_reference/C/dbcget.html#dbcget_DB_PREV" class="olink">DB_PREV</a></span> </dt> - <dd> - Return the next (previous) record in the database. + <dd> + Return the next (previous) record in the + database. </dd> <dt> <span class="term"> <a href="../api_reference/C/dbcget.html#dbcget_DB_NEXT_DUP" class="olink">DB_NEXT_DUP</a> </span> </dt> - <dd> - Return the next record in the database, if it is a - duplicate data item for the current key. For Heap - databases, this flag always results in the cursor - returning the <code class="literal">DB_NOTFOUND</code> - error. + <dd> + Return the next record in the database, if + it is a duplicate data item for the current key. + For Heap databases, this flag always results in + the cursor returning the + <code class="literal">DB_NOTFOUND</code> error. </dd> <dt> <span class="term"><a href="../api_reference/C/dbcget.html#dbcget_DB_NEXT_NODUP" class="olink">DB_NEXT_NODUP</a>, <a href="../api_reference/C/dbcget.html#dbcget_DB_PREV_NODUP" class="olink">DB_PREV_NODUP</a></span> </dt> <dd> - Return the next (previous) record in the database that - is not a duplicate data item for the current - key. + Return the next (previous) record in the + database that is not a duplicate data item for the + current key. </dd> <dt> <span class="term"> <a href="../api_reference/C/dbcget.html#dbcget_DB_CURRENT" class="olink">DB_CURRENT</a> </span> </dt> - <dd> - Return the record from the database to which the cursor - currently refers. + <dd> + Return the record from the database to + which the cursor currently refers. </dd> </dl> </div> @@ -181,7 +186,7 @@ <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idp900744"></a>Retrieving specific key/data pairs</h4> + <h4 class="title"><a id="idp419144"></a>Retrieving specific key/data pairs</h4> </div> </div> </div> @@ -193,12 +198,12 @@ </span> </dt> <dd> - Return the record from the database that matches the - supplied key. In the case of duplicates the first - duplicate is returned and the cursor is positioned at - the beginning of the duplicate list. The user can then - traverse the duplicate entries for the - key. + Return the record from the database that + matches the supplied key. In the case of + duplicates the first duplicate is returned and the + cursor is positioned at the beginning of the + duplicate list. The user can then traverse the + duplicate entries for the key. </dd> <dt> <span class="term"> @@ -206,23 +211,24 @@ </span> </dt> <dd> - Return the smallest record in the database greater than - or equal to the supplied key. This functionality - permits partial key matches and range searches in the - Btree access method. + Return the smallest record in the database + greater than or equal to the supplied key. This + functionality permits partial key matches and + range searches in the Btree access method. </dd> <dt> <span class="term"> <a href="../api_reference/C/dbcget.html#dbcget_DB_GET_BOTH" class="olink">DB_GET_BOTH</a> </span> </dt> - <dd> - Return the record from the database that matches both - the supplied key and data items. This is particularly - useful when there are large numbers of duplicate - records for a key, as it allows the cursor to easily be - positioned at the correct place for traversal of some - part of a large set of duplicate records. + <dd> + Return the record from the database that + matches both the supplied key and data items. This + is particularly useful when there are large + numbers of duplicate records for a key, as it + allows the cursor to easily be positioned at the + correct place for traversal of some part of a + large set of duplicate records. </dd> <dt> <span class="term"> @@ -230,11 +236,13 @@ </span> </dt> <dd> - If used on a database configured for sorted duplicates, this - returns the smallest record in the database greater than or equal - to the supplied key and data items. If used on a database that is - <span class="emphasis"><em>not</em></span> configured for sorted duplicates, this - flag behaves identically to <code class="literal">DB_GET_BOTH</code>. + If used on a database configured for sorted + duplicates, this returns the smallest record in + the database greater than or equal to the supplied + key and data items. If used on a database that is + <span class="emphasis"><em>not</em></span> configured for sorted + duplicates, this flag behaves identically to + <code class="literal">DB_GET_BOTH</code>. </dd> </dl> </div> @@ -243,7 +251,7 @@ <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idp889792"></a>Retrieving based on record numbers</h4> + <h4 class="title"><a id="idp409264"></a>Retrieving based on record numbers</h4> </div> </div> </div> @@ -255,21 +263,21 @@ </span> </dt> <dd> - If the underlying database is a Btree, and was - configured so that it is possible to search it by - logical record number, retrieve a specific record based - on a record number argument. + If the underlying database is a Btree, and + was configured so that it is possible to search it + by logical record number, retrieve a specific + record based on a record number argument. </dd> <dt> <span class="term"> <a href="../api_reference/C/dbcget.html#dbcget_DB_GET_RECNO" class="olink">DB_GET_RECNO</a> </span> </dt> - <dd> - If the underlying database is a Btree, and was - configured so that it is possible to search it by - logical record number, return the record number for the - record to which the cursor refers. + <dd> + If the underlying database is a Btree, and + was configured so that it is possible to search it + by logical record number, return the record number + for the record to which the cursor refers. </dd> </dl> </div> @@ -278,7 +286,7 @@ <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idp899384"></a>Special-purpose flags</h4> + <h4 class="title"><a id="idp418264"></a>Special-purpose flags</h4> </div> </div> </div> @@ -289,10 +297,10 @@ <a href="../api_reference/C/dbget.html#dbget_DB_CONSUME" class="olink">DB_CONSUME</a> </span> </dt> - <dd> - Read-and-delete: the first record (the head) of the - queue is returned and deleted. The underlying database - must be a Queue. + <dd> + Read-and-delete: the first record (the + head) of the queue is returned and deleted. The + underlying database must be a Queue. </dd> <dt> <span class="term"> @@ -300,22 +308,22 @@ </span> </dt> <dd> - Read-modify-write: acquire write locks instead of read - locks during retrieval. This can enhance performance in - threaded applications by reducing the chance of - deadlock. + Read-modify-write: acquire write locks + instead of read locks during retrieval. This can + enhance performance in threaded applications by + reducing the chance of deadlock. </dd> </dl> </div> - <p> + <p> In all cases, the cursor is repositioned by a <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> - operation to point to the newly-returned key/data pair in the - database. + operation to point to the newly-returned key/data pair in + the database. </p> <p> - The following is a code example showing a cursor walking - through a database and displaying the records it contains to - the standard output: + The following is a code example showing a cursor + walking through a database and displaying the records it + contains to the standard output: </p> <a id="prog_am19"></a> <pre class="programlisting">int @@ -387,13 +395,14 @@ err: if (close_dbc && (ret = dbcp->close(dbcp)) != 0) </div> </div> <p> - The <a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a> method stores records into the database using a - cursor. In general, <a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a> takes a key and inserts the - associated data into the database, at a location controlled by a - specified flag. + The <a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a> method stores records into the database using + a cursor. In general, <a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a> takes a key and inserts the + associated data into the database, at a location controlled by + a specified flag. </p> <p> - There are several flags that you can set to customize storage: + There are several flags that you can set to customize + storage: </p> <div class="variablelist"> <dl> @@ -402,18 +411,18 @@ err: if (close_dbc && (ret = dbcp->close(dbcp)) != 0) <a href="../api_reference/C/dbcput.html#dbcput_DB_AFTER" class="olink">DB_AFTER</a> </span> </dt> - <dd> - Create a new record, immediately after the record to which - the cursor refers. + <dd> + Create a new record, immediately after the + record to which the cursor refers. </dd> <dt> <span class="term"> <a href="../api_reference/C/dbcput.html#dbcput_DB_BEFORE" class="olink">DB_BEFORE</a> </span> </dt> - <dd> - Create a new record, immediately before the record to - which the cursor refers. + <dd> + Create a new record, immediately before the + record to which the cursor refers. </dd> <dt> <span class="term"> @@ -421,17 +430,17 @@ err: if (close_dbc && (ret = dbcp->close(dbcp)) != 0) </span> </dt> <dd> - Replace the data part of the record to which the cursor - refers. + Replace the data part of the record to which + the cursor refers. </dd> <dt> <span class="term"> <a href="../api_reference/C/dbcput.html#dbcput_DB_KEYFIRST" class="olink">DB_KEYFIRST</a> </span> </dt> - <dd> - Create a new record as the first of the duplicate records - for the supplied key. + <dd> + Create a new record as the first of the + duplicate records for the supplied key. </dd> <dt> <span class="term"> @@ -439,18 +448,20 @@ err: if (close_dbc && (ret = dbcp->close(dbcp)) != 0) </span> </dt> <dd> - Create a new record, as the last of the duplicate records - for the supplied key. + Create a new record, as the last of the + duplicate records for the supplied key. </dd> </dl> </div> - <p> - In all cases, the cursor is repositioned by a <a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a> operation to - point to the newly inserted key/data pair in the database. + <p> + In all cases, the cursor is repositioned by a <a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a> + operation to point to the newly inserted key/data pair in the + database. </p> - <p> - The following is a code example showing a cursor storing two data - items in a database that supports duplicate data items: + <p> + The following is a code example showing a cursor storing + two data items in a database that supports duplicate data + items: </p> <a id="prog_am20"></a> <pre class="programlisting">int @@ -504,12 +515,12 @@ err: if ((ret = dbcp->close(dbcp)) != 0) <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> <h3 class="title">Note</h3> <p> - If you are using the Heap access method and you are creating a - new record in the database, then the key that you provide to - the <a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a> method should be empty. The <a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a> method will - return the record's ID (RID) in the key. The RID is - automatically created for you when Heap database records are - created. + If you are using the Heap access method and you are + creating a new record in the database, then the key that + you provide to the <a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a> method should be empty. The + <a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a> method will return the record's ID (RID) in the + key. The RID is automatically created for you when Heap + database records are created. </p> </div> </div> @@ -521,10 +532,12 @@ err: if ((ret = dbcp->close(dbcp)) != 0) </div> </div> </div> - <p>The <a href="../api_reference/C/dbcdel.html" class="olink">DBC->del()</a> method deletes records from the database using a cursor. -The <a href="../api_reference/C/dbcdel.html" class="olink">DBC->del()</a> method deletes the record to which the cursor currently -refers. In all cases, the cursor position is unchanged after a -delete.</p> + <p> + The <a href="../api_reference/C/dbcdel.html" class="olink">DBC->del()</a> method deletes records from the database using + a cursor. The <a href="../api_reference/C/dbcdel.html" class="olink">DBC->del()</a> method deletes the record to which the + cursor currently refers. In all cases, the cursor position is + unchanged after a delete. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> @@ -534,18 +547,24 @@ delete.</p> </div> </div> </div> - <p>Once a cursor has been initialized (for example, by a call to -<a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a>), it can be thought of as identifying a particular -location in a database. The <a href="../api_reference/C/dbcdup.html" class="olink">DBC->dup()</a> method permits an application to -create a new cursor that has the same locking and transactional -information as the cursor from which it is copied, and which optionally -refers to the same position in the database.</p> - <p>In order to maintain a cursor position when an application is using -locking, locks are maintained on behalf of the cursor until the cursor is -closed. In cases when an application is using locking without -transactions, cursor duplication is often required to avoid -self-deadlocks. For further details, refer to -<a class="xref" href="lock_am_conv.html" title="Berkeley DB Transactional Data Store locking conventions">Berkeley DB Transactional Data Store locking conventions</a>.</p> + <p> + Once a cursor has been initialized (for example, by a call + to <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a>), it can be thought of as identifying a particular + location in a database. The <a href="../api_reference/C/dbcdup.html" class="olink">DBC->dup()</a> method permits an + application to create a new cursor that has the same locking + and transactional information as the cursor from which it is + copied, and which optionally refers to the same position in + the database. + </p> + <p> + In order to maintain a cursor position when an application + is using locking, locks are maintained on behalf of the cursor + until the cursor is closed. In cases when an application is + using locking without transactions, cursor duplication is + often required to avoid self-deadlocks. For further details, + refer to <a class="xref" href="lock_am_conv.html" title="Berkeley DB Transactional Data Store locking conventions">Berkeley DB Transactional Data + Store locking conventions</a>. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> @@ -555,18 +574,25 @@ self-deadlocks. For further details, refer to </div> </div> </div> - <p>Berkeley DB supports "equality" (also known as "natural"), joins on secondary -indices. An equality join is a method of retrieving data from a primary -database using criteria stored in a set of secondary indices. It -requires the data be organized as a primary database which contains the -primary key and primary data field, and a set of secondary indices. -Each of the secondary indices is indexed by a different secondary key, -and, for each key in a secondary index, there is a set of duplicate data -items that match the primary keys in the primary database.</p> - <p>For example, let's assume the need for an application that will return -the names of stores in which one can buy fruit of a given color. We -would first construct a primary database that lists types of fruit as -the key item, and the store where you can buy them as the data item:</p> + <p> + Berkeley DB supports "equality" (also known as "natural"), + joins on secondary indices. An equality join is a method of + retrieving data from a primary database using criteria stored + in a set of secondary indices. It requires the data be + organized as a primary database which contains the primary key + and primary data field, and a set of secondary indices. Each + of the secondary indices is indexed by a different secondary + key, and, for each key in a secondary index, there is a set of + duplicate data items that match the primary keys in the + primary database. + </p> + <p> + For example, let's assume the need for an application that + will return the names of stores in which one can buy fruit of + a given color. We would first construct a primary database + that lists types of fruit as the key item, and the store where + you can buy them as the data item: + </p> <div class="informaltable"> <table border="1" width="80%"> <colgroup> @@ -607,8 +633,11 @@ the key item, and the store where you can buy them as the data item:</p> </tbody> </table> </div> - <p>We would then create a secondary index with the key <span class="bold"><strong>color</strong></span>, and, -as the data items, the names of fruits of different colors.</p> + <p> + We would then create a secondary index with the key + <span class="bold"><strong>color</strong></span>, and, as the data + items, the names of fruits of different colors. + </p> <div class="informaltable"> <table border="1" width="80%"> <colgroup> @@ -649,35 +678,54 @@ as the data items, the names of fruits of different colors.</p> </tbody> </table> </div> - <p>This secondary index would allow an application to look up a color, and -then use the data items to look up the stores where the colored fruit -could be purchased. For example, by first looking up <span class="bold"><strong>blue</strong></span>, -the data item <span class="bold"><strong>blueberry</strong></span> could be used as the lookup key in the -primary database, returning <span class="bold"><strong>Farmer's Market</strong></span>.</p> - <p>Your data must be organized in the following manner in order to use the -<a href="../api_reference/C/dbjoin.html" class="olink">DB->join()</a> method:</p> + <p> + This secondary index would allow an application to look up a + color, and then use the data items to look up the stores where + the colored fruit could be purchased. For example, by first + looking up <span class="bold"><strong>blue</strong></span>, the data + item <span class="bold"><strong>blueberry</strong></span> could be used + as the lookup key in the primary database, returning <span class="bold"><strong>Farmer's Market</strong></span>. + </p> + <p> + Your data must be organized in the following manner in order + to use the <a href="../api_reference/C/dbjoin.html" class="olink">DB->join()</a> method: + </p> <div class="orderedlist"> <ol type="1"> - <li>The actual data should be stored in the database represented by the -<a href="../api_reference/C/db.html" class="olink">DB</a> object used to invoke this method. Generally, this -<a href="../api_reference/C/db.html" class="olink">DB</a> object is called the <span class="emphasis"><em>primary</em></span>.</li> - <li>Secondary indices should be stored in separate databases, whose keys -are the values of the secondary indices and whose data items are the -primary keys corresponding to the records having the designated -secondary key value. It is acceptable (and expected) that there may be -duplicate entries in the secondary indices. -<p>These duplicate entries should be sorted for performance reasons, although -it is not required. For more information see the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_DUPSORT" class="olink">DB_DUPSORT</a> flag -to the <a href="../api_reference/C/dbset_flags.html" class="olink">DB->set_flags()</a> method.</p></li> + <li> + The actual data should be stored in the database + represented by the <a href="../api_reference/C/db.html" class="olink">DB</a> object used to invoke this method. + Generally, this <a href="../api_reference/C/db.html" class="olink">DB</a> object is called the + <span class="emphasis"><em>primary</em></span>. + </li> + <li> + Secondary indices should be stored in separate + databases, whose keys are the values of the secondary + indices and whose data items are the primary keys + corresponding to the records having the designated + secondary key value. It is acceptable (and expected) that + there may be duplicate entries in the secondary indices. + <p> + These duplicate entries should be sorted for + performance reasons, although it is not required. For + more information see the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_DUPSORT" class="olink">DB_DUPSORT</a> flag to the + <a href="../api_reference/C/dbset_flags.html" class="olink">DB->set_flags()</a> method. + </p></li> </ol> </div> - <p>What the <a href="../api_reference/C/dbjoin.html" class="olink">DB->join()</a> method does is review a list of secondary keys, and, -when it finds a data item that appears as a data item for all of the -secondary keys, it uses that data item as a lookup into the primary -database, and returns the associated data item.</p> - <p>If there were another secondary index that had as its key the <span class="bold"><strong>cost</strong></span> -of the fruit, a similar lookup could be done on stores where inexpensive -fruit could be purchased:</p> + <p> + What the <a href="../api_reference/C/dbjoin.html" class="olink">DB->join()</a> method does is review a list of secondary + keys, and, when it finds a data item that appears as a data + item for all of the secondary keys, it uses that data item as + a lookup into the primary database, and returns the associated + data item. + </p> + <p> + If there were another secondary index that had as its key + the <span class="bold"><strong>cost</strong></span> of the fruit, a + similar lookup could be done on stores where inexpensive fruit + could be purchased: + </p> <div class="informaltable"> <table border="1" width="80%"> <colgroup> @@ -722,25 +770,33 @@ fruit could be purchased:</p> </tbody> </table> </div> - <p>The <a href="../api_reference/C/dbjoin.html" class="olink">DB->join()</a> method provides equality join functionality. While not -strictly cursor functionality, in that it is not a method off a cursor -handle, it is more closely related to the cursor operations than to the -standard <a href="../api_reference/C/db.html" class="olink">DB</a> operations.</p> - <p>It is also possible to do lookups based on multiple criteria in a single -operation. For example, it is possible to look up fruits that are both -red and expensive in a single operation. If the same fruit appeared as -a data item in both the color and expense indices, then that fruit name -would be used as the key for retrieval from the primary index, and would -then return the store where expensive, red fruit could be purchased.</p> + <p> + The <a href="../api_reference/C/dbjoin.html" class="olink">DB->join()</a> method provides equality join functionality. + While not strictly cursor functionality, in that it is not a + method off a cursor handle, it is more closely related to the + cursor operations than to the standard <a href="../api_reference/C/db.html" class="olink">DB</a> operations. + </p> + <p> + It is also possible to do lookups based on multiple criteria + in a single operation. For example, it is possible to look up + fruits that are both red and expensive in a single operation. + If the same fruit appeared as a data item in both the color + and expense indices, then that fruit name would be used as the + key for retrieval from the primary index, and would then + return the store where expensive, red fruit could be + purchased. + </p> <div class="sect3" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idp942824"></a>Example</h4> + <h4 class="title"><a id="idp472064"></a>Example</h4> </div> </div> </div> - <p>Consider the following three databases:</p> + <p> + Consider the following three databases: + </p> <div class="variablelist"> <dl> <dt> @@ -750,7 +806,9 @@ then return the store where expensive, red fruit could be purchased.</p> <div class="itemizedlist"> <ul type="disc"> <li>key = SSN</li> - <li>data = record containing name, address, phone number, job title</li> + <li>data = record containing name, + address, phone number, job + title</li> </ul> </div> </dd> @@ -778,15 +836,22 @@ then return the store where expensive, red fruit could be purchased.</p> </dd> </dl> </div> - <p>Consider the following query:</p> + <p> + Consider the following query: + </p> <pre class="programlisting">Return the personnel records of all people named smith with the job title manager.</pre> - <p>This query finds are all the records in the primary database (personnel) -for whom the criteria <span class="bold"><strong>lastname=smith and job title=manager</strong></span> is -true.</p> - <p>Assume that all databases have been properly opened and have the -handles: pers_db, name_db, job_db. We also assume that we have an -active transaction to which the handle txn refers.</p> + <p> + This query finds are all the records in the primary + database (personnel) for whom the criteria <span class="bold"><strong>lastname=smith and job + title=manager</strong></span> is true. + </p> + <p> + Assume that all databases have been properly opened and + have the handles: pers_db, name_db, job_db. We also assume + that we have an active transaction to which the handle txn + refers. + </p> <a id="prog_am21"></a> <pre class="programlisting">DBC *name_curs, *job_curs, *join_curs; DBC *carray[3]; @@ -846,11 +911,15 @@ if (job_curs != NULL && ret = tret; return (ret);</pre> - <p>The name cursor is positioned at the beginning of the duplicate list -for <span class="bold"><strong>smith</strong></span> and the job cursor is placed at the beginning of -the duplicate list for <span class="bold"><strong>manager</strong></span>. The join cursor is returned -from the join method. This code then loops over the join cursor getting -the personnel records of each one until there are no more.</p> + <p> + The name cursor is positioned at the beginning of the + duplicate list for <span class="bold"><strong>smith</strong></span> + and the job cursor is placed at the beginning of the + duplicate list for <span class="bold"><strong>manager</strong></span>. + The join cursor is returned from the join method. This code then + loops over the join cursor getting the personnel records of each one + until there are no more. + </p> </div> </div> <div class="sect2" lang="en" xml:lang="en"> @@ -861,12 +930,15 @@ the personnel records of each one until there are no more.</p> </div> </div> </div> - <p>Once a cursor has been initialized to refer to a particular key in the -database, it can be used to determine the number of data items that are -stored for any particular key. The <a href="../api_reference/C/dbccount.html" class="olink">DBC->count()</a> method returns -this number of data items. The returned value is always one, unless -the database supports duplicate data items, in which case it may be any -number of items.</p> + <p> + Once a cursor has been initialized to refer to a particular + key in the database, it can be used to determine the number of + data items that are stored for any particular key. The + <a href="../api_reference/C/dbccount.html" class="olink">DBC->count()</a> method returns this number of data items. The + returned value is always one, unless the database supports + duplicate data items, in which case it may be any number of + items. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> @@ -876,12 +948,15 @@ number of items.</p> </div> </div> </div> - <p>The <a href="../api_reference/C/dbcclose.html" class="olink">DBC->close()</a> method closes the <a href="../api_reference/C/dbc.html" class="olink">DBC</a> cursor, after which the -cursor may no longer be used. Although cursors are implicitly closed -when the database they point to are closed, it is good programming -practice to explicitly close cursors. In addition, in transactional -systems, cursors may not exist outside of a transaction and so must be -explicitly closed.</p> + <p> + The <a href="../api_reference/C/dbcclose.html" class="olink">DBC->close()</a> method closes the <a href="../api_reference/C/dbc.html" class="olink">DBC</a> cursor, after which + the cursor may no longer be used. Although cursors are + implicitly closed when the database they point to are closed, + it is good programming practice to explicitly close cursors. + In addition, in transactional systems, cursors may not exist + outside of a transaction and so must be explicitly + closed. + </p> </div> </div> <div class="navfooter"> @@ -899,9 +974,7 @@ explicitly closed.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 4. - Access Method Wrapup - </td> + <td width="40%" align="right" valign="top"> Chapter 4. Access Method Wrapup </td> </tr> </table> </div> diff --git a/docs/programmer_reference/am_delete.html b/docs/programmer_reference/am_delete.html index 81bf37c0..108ed33c 100644 --- a/docs/programmer_reference/am_delete.html +++ b/docs/programmer_reference/am_delete.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_put.html">Prev</a> </td> - <th width="60%" align="center">Chapter 3. - Access Method Operations - </th> + <th width="60%" align="center">Chapter 3. Access Method Operations </th> <td width="20%" align="right"> <a accesskey="n" href="am_stat.html">Next</a></td> </tr> </table> @@ -38,12 +36,17 @@ </div> </div> </div> - <p>The <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a> method deletes records from the database. In general, -<a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a> takes a key and deletes the data item associated with -it from the database.</p> - <p>If the database has been configured to support duplicate records, the -<a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a> method will remove all of the duplicate records. To remove -individual duplicate records, you must use a Berkeley DB cursor interface.</p> + <p> + The <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a> method deletes records from the database. In + general, <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a> takes a key and deletes the data item + associated with it from the database. + </p> + <p> + If the database has been configured to support duplicate + records, the <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a> method will remove all of the duplicate + records. To remove individual duplicate records, you must use + a Berkeley DB cursor interface. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/am_foreign.html b/docs/programmer_reference/am_foreign.html index 677f9554..ff175021 100644 --- a/docs/programmer_reference/am_foreign.html +++ b/docs/programmer_reference/am_foreign.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_second.html">Prev</a> </td> - <th width="60%" align="center">Chapter 3. - Access Method Operations - </th> + <th width="60%" align="center">Chapter 3. Access Method Operations </th> <td width="20%" align="right"> <a accesskey="n" href="am_cursor.html">Next</a></td> </tr> </table> @@ -39,124 +37,136 @@ </div> </div> <p> - Foreign keys are used to ensure a level of consistency between - two different databases in terms of the keys that the databases use. - In a foreign key relationship, one database is the - <span class="emphasis"><em>constrained</em></span> database. This database is actually a - secondary database which is associated with a primary database. The - other database in this relationship is the - <span class="emphasis"><em>foreign key</em></span> database. Once this relationship has - been established between a constrained database and a foreign key - database, then: -</p> + Foreign keys are used to ensure a level of consistency + between two different databases in terms of the keys that the + databases use. In a foreign key relationship, one database is + the <span class="emphasis"><em>constrained</em></span> database. This database + is actually a secondary database which is associated with a + primary database. The other database in this relationship is + the <span class="emphasis"><em>foreign key</em></span> database. Once this + relationship has been established between a constrained + database and a foreign key database, then: + </p> <div class="orderedlist"> <ol type="1"> <li> <p> - Key/data items cannot be added to the - constrained database unless that same key - already exists in the foreign key - database. - </p> + Key/data items cannot be added to the constrained + database unless that same key already exists in the + foreign key database. + </p> </li> <li> <p> - A key/data pair cannot be deleted from the foreign - key database unless some action is also taken to - keep the constrained database consistent with - the foreign key database. - </p> + A key/data pair cannot be deleted from the foreign + key database unless some action is also taken to keep + the constrained database consistent with the foreign + key database. + </p> </li> </ol> </div> <p> - Because the constrained database is a secondary database, by ensuring - it's consistency with a foreign key database you are actually ensuring - that a primary database (the one to which the secondary database is - associated) is consistent with the foreign key database. -</p> + Because the constrained database is a secondary database, + by ensuring it's consistency with a foreign key database you + are actually ensuring that a primary database (the one to + which the secondary database is associated) is consistent with + the foreign key database. + </p> <p> - Deletions of keys in the foreign key database affect the constrained database - in one of three ways, as specified by the application: -</p> + Deletions of keys in the foreign key database affect the + constrained database in one of three ways, as specified by the + application: + </p> <div class="itemizedlist"> <ul type="disc"> <li> <p> - <code class="literal">Abort</code> - </p> - <p> - The deletion of a record from the foreign database will not proceed if that key - exists in the constrained primary database. Transactions must be used to prevent - the aborted delete from corrupting either of the databases. - </p> + <code class="literal">Abort</code> + </p> + <p> + The deletion of a record from the foreign database + will not proceed if that key exists in the constrained + primary database. Transactions must be used to prevent + the aborted delete from corrupting either of the + databases. + </p> </li> <li> <p> - <code class="literal">Cascade</code> - </p> - <p> - The deletion of a record from the foreign database will also cause any records - in the constrained primary database that use that key to also be - automatically deleted. - </p> + <code class="literal">Cascade</code> + </p> + <p> + The deletion of a record from the foreign database + will also cause any records in the constrained primary + database that use that key to also be automatically + deleted. + </p> </li> <li> <p> - <code class="literal">Nullify</code> - </p> - <p> - The deletion of a record from the foreign database will cause a user specified - callback function to be called, in order to alter or nullify any records - using that key in the constrained primary database. - </p> + <code class="literal">Nullify</code> + </p> + <p> + The deletion of a record from the foreign database + will cause a user specified callback function to be + called, in order to alter or nullify any records using + that key in the constrained primary database. + </p> </li> </ul> </div> <p> - Note that it is possible to delete a key from the constrained database, - but not from the foreign key database. For this reason, if you want the - keys used in both databases to be 100% accurate, then you will have to - write code to ensure that when a key is removed from the constrained - database, it is also removed from the foreign key database. -</p> + Note that it is possible to delete a key from the + constrained database, but not from the foreign key database. + For this reason, if you want the keys used in both databases + to be 100% accurate, then you will have to write code to + ensure that when a key is removed from the constrained + database, it is also removed from the foreign key database. + </p> + <p> + As an example of how foreign key indexes might be used, + consider a database of customer information and a database of + order information. A typical customer database would use a + customer ID as the key and those keys would also appear in the + order database. To ensure an order is not booked for a + non-existent customer, the customer database can be associated + with the order database as a foreign index. + </p> <p> - As an example of how foreign key indexes might be used, consider a database - of customer information and a database of order information. A typical - customer database would use a customer ID as the key and those keys - would also appear in the order database. To ensure an order is not - booked for a non-existent customer, the customer database can be - associated with the order database as a foreign index. -</p> + In order to do this, you create a secondary index of the + order database, which uses customer IDs as the key for its + key/data pairs. This secondary index is, then, the constrained + database. But because the secondary index is constrained, so + too is the order database because the contents of the + secondary index are programmatically tied to the contents of + the order database. + </p> + <p> + The customer database, then, is the foreign key database. + It is associated to the order database's secondary index using + the <a href="../api_reference/C/dbassociate_foreign.html" class="olink">DB->associate_foreign()</a> method. In this way, an order cannot + be added to the order database unless the customer ID already + exists in the customer database. + </p> + <p> + Note that this relationship can also be configured to + delete any outstanding orders for a customer when that + customer is deleted from the customer database. + </p> <p> - In order to do this, you create a secondary index of the order - database, which uses customer IDs as the key for its key/data pairs. - This secondary index is, then, the constrained database. But because - the secondary index is constrained, so too is the order database - because the contents of the secondary index are programmatically tied - to the contents of the order database. -</p> - <p> - The customer database, then, is the foreign key database. It is - associated to the order database's secondary index using the - <a href="../api_reference/C/dbassociate_foreign.html" class="olink">DB->associate_foreign()</a> method. In this way, - an order cannot be added to the order database unless the customer ID - already exists in the customer database. -</p> - <p> - Note that this relationship can also be configured to delete any - outstanding orders for a customer when that customer is deleted from - the customer database. -</p> - <p>In SQL, this would be done by executing something like the following:</p> + In SQL, this would be done by executing something like the + following: + </p> <pre class="programlisting">CREATE TABLE customers(cust_id CHAR(4) NOT NULL, lastname CHAR(15), firstname CHAR(15), PRIMARY KEY(cust_id)); CREATE TABLE orders(order_id CHAR(4) NOT NULL, order_num int NOT NULL, cust_id CHAR(4), PRIMARY KEY (order_id), FOREIGN KEY (cust_id) REFERENCES customers(cust_id) - ON DELETE CASCADE); -</pre> - <p>In Berkeley DB, this would work as follows:</p> + ON DELETE CASCADE); </pre> + <p> + In Berkeley DB, this would work as follows: + </p> <a id="prog_am18"></a> <pre class="programlisting">struct customer { char cust_id[4]; diff --git a/docs/programmer_reference/am_get.html b/docs/programmer_reference/am_get.html index 8aba222b..491dec69 100644 --- a/docs/programmer_reference/am_get.html +++ b/docs/programmer_reference/am_get.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_partition.html">Prev</a> </td> - <th width="60%" align="center">Chapter 3. - Access Method Operations - </th> + <th width="60%" align="center">Chapter 3. Access Method Operations </th> <td width="20%" align="right"> <a accesskey="n" href="am_put.html">Next</a></td> </tr> </table> @@ -38,10 +36,15 @@ </div> </div> </div> - <p>The <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> method retrieves records from the database. In general, -<a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> takes a key and returns the associated data from the -database.</p> - <p>There are a few flags that you can set to customize retrieval:</p> + <p> + The <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> method retrieves records from the database. In + general, <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> takes a key and returns the associated data + from the database. + </p> + <p> + There are a few flags that you can set to customize + retrieval: + </p> <div class="variablelist"> <dl> <dt> @@ -49,29 +52,40 @@ database.</p> <a href="../api_reference/C/dbget.html#get_DB_GET_BOTH" class="olink">DB_GET_BOTH</a> </span> </dt> - <dd>Search for a matching key and data item, that is, only return success -if both the key and the data items match those stored in the database.</dd> + <dd> + Search for a matching key and data item, that + is, only return success if both the key and the data + items match those stored in the database. + </dd> <dt> <span class="term"> <a href="../api_reference/C/dbcget.html#dbcget_DB_RMW" class="olink">DB_RMW</a> </span> </dt> - <dd>Read-modify-write: acquire write locks instead of read locks during -retrieval. This can enhance performance in threaded applications by -reducing the chance of deadlock.</dd> + <dd> + Read-modify-write: acquire write locks instead + of read locks during retrieval. This can enhance + performance in threaded applications by reducing the + chance of deadlock. + </dd> <dt> <span class="term"> <a href="../api_reference/C/dbget.html#dbget_DB_SET_RECNO" class="olink">DB_SET_RECNO</a> </span> </dt> - <dd>If the underlying database is a Btree, and was configured so that it -is possible to search it by logical record number, retrieve a specific -record.</dd> + <dd> + If the underlying database is a Btree, and was + configured so that it is possible to search it by + logical record number, retrieve a specific + record. + </dd> </dl> </div> - <p>If the database has been configured to support duplicate records, -<a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> will always return the first data item in the duplicate -set.</p> + <p> + If the database has been configured to support duplicate + records, <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> will always return the first data item in the + duplicate set. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/am_misc.html b/docs/programmer_reference/am_misc.html index d2f071ae..d8ed6664 100644 --- a/docs/programmer_reference/am_misc.html +++ b/docs/programmer_reference/am_misc.html @@ -14,13 +14,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 4. - Access Method Wrapup - </th> + <th colspan="3" align="center">Chapter 4. Access Method Wrapup </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_cursor.html">Prev</a> </td> @@ -34,9 +32,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="am_misc"></a>Chapter 4. - Access Method Wrapup - </h2> + <h2 class="title"><a id="am_misc"></a>Chapter 4. Access Method Wrapup </h2> </div> </div> </div> @@ -76,12 +72,14 @@ </dd> <dt> <span class="sect1"> - <a href="am_misc_partial.html">Partial record storage and retrieval</a> + <a href="am_misc_partial.html">Partial record storage and + retrieval</a> </span> </dt> <dt> <span class="sect1"> - <a href="am_misc_struct.html">Storing C/C++ structures/objects</a> + <a href="am_misc_struct.html">Storing C/C++ + structures/objects</a> </span> </dt> <dt> @@ -106,26 +104,62 @@ </dt> <dt> <span class="sect1"> - <a href="am_misc_diskspace.html">Disk space requirements</a> + <a href="am_misc_diskspace.html">Disk space + requirements</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="am_misc_diskspace.html#idp1074008">Btree</a> + <a href="am_misc_diskspace.html#idp595712">Btree</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_misc_diskspace.html#idp1074072">Hash</a> + <a href="am_misc_diskspace.html#idp595776">Hash</a> </span> </dt> </dl> </dd> <dt> <span class="sect1"> - <a href="am_misc_db_sql.html">Specifying a Berkeley DB schema using SQL DDL</a> + <a href="blobs.html">BLOB support</a> + </span> + </dt> + <dd> + <dl> + <dt> + <span class="sect2"> + <a href="blobs.html#blob_threshold">The BLOB threshold</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="blobs.html#blob_create">Creating BLOBs</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="blobs.html#blob_access">BLOB access</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="blobs.html#blob_storage">BLOB storage</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="blobs.html#blob_replication">BLOBs and Replication</a> + </span> + </dt> + </dl> + </dd> + <dt> + <span class="sect1"> + <a href="am_misc_db_sql.html">Specifying a Berkeley DB schema + using SQL DDL</a> </span> </dt> <dt> @@ -148,12 +182,15 @@ </div> </div> </div> - <p>The Berkeley DB access methods provide no guarantees about byte alignment for -returned key/data pairs, or callback functions which take <a href="../api_reference/C/dbt.html" class="olink">DBT</a> -references as arguments, and applications are responsible for arranging -any necessary alignment. The <a href="../api_reference/C/dbt.html#dbt_DB_DBT_MALLOC" class="olink">DB_DBT_MALLOC</a>, <a href="../api_reference/C/dbt.html#dbt_DB_DBT_REALLOC" class="olink">DB_DBT_REALLOC</a>, and -<a href="../api_reference/C/dbt.html#dbt_DB_DBT_USERMEM" class="olink">DB_DBT_USERMEM</a> flags may be used to -store returned items in memory of arbitrary alignment.</p> + <p> + The Berkeley DB access methods provide no guarantees about + byte alignment for returned key/data pairs, or callback + functions which take <a href="../api_reference/C/dbt.html" class="olink">DBT</a> references as arguments, and + applications are responsible for arranging any necessary + alignment. The <a href="../api_reference/C/dbt.html#dbt_DB_DBT_MALLOC" class="olink">DB_DBT_MALLOC</a>, <a href="../api_reference/C/dbt.html#dbt_DB_DBT_REALLOC" class="olink">DB_DBT_REALLOC</a>, and + <a href="../api_reference/C/dbt.html#dbt_DB_DBT_USERMEM" class="olink">DB_DBT_USERMEM</a> flags may be used to store returned items in + memory of arbitrary alignment. + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/am_misc_bulk.html b/docs/programmer_reference/am_misc_bulk.html index fcfd4736..f69e712b 100644 --- a/docs/programmer_reference/am_misc_bulk.html +++ b/docs/programmer_reference/am_misc_bulk.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_misc.html">Prev</a> </td> - <th width="60%" align="center">Chapter 4. - Access Method Wrapup - </th> + <th width="60%" align="center">Chapter 4. Access Method Wrapup </th> <td width="20%" align="right"> <a accesskey="n" href="am_misc_partial.html">Next</a></td> </tr> </table> @@ -57,11 +55,12 @@ </dt> </dl> </div> - <p>When retrieving or modifying large numbers of records, the number -of method calls can often dominate performance. Berkeley DB offers bulk get, -put and delete interfaces which can significantly increase performance for some -applications. -</p> + <p> + When retrieving or modifying large numbers of records, the + number of method calls can often dominate performance. + Berkeley DB offers bulk get, put and delete interfaces which + can significantly increase performance for some applications. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> @@ -70,68 +69,78 @@ applications. </div> </div> </div> + <p> + To retrieve records in bulk, an application buffer must + be specified to the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> methods. This is + done in the C API by setting the <span class="bold"><strong>data</strong></span> + and <span class="bold"><strong>ulen</strong></span> fields of the + <span class="bold"><strong>data</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> to reference an application + buffer, and the <span class="bold"><strong>flags</strong></span> + field of that structure to <a href="../api_reference/C/dbt.html#dbt_DB_DBT_USERMEM" class="olink">DB_DBT_USERMEM</a>. In the + Berkeley DB C++ and Java APIs, the actions are similar, + although there are API-specific methods to set the <a href="../api_reference/C/dbt.html" class="olink">DBT</a> + values. Then, the <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> or <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a> flags + are specified to the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> methods, which + cause multiple records to be returned in the specified + buffer. + </p> <p> - To retrieve records in bulk, an application buffer must be specified to - the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> methods. This is done in the C API by setting - the <span class="bold"><strong>data</strong></span> and <span class="bold"><strong>ulen</strong></span> fields of the <span class="bold"><strong>data</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> to reference an application - buffer, and the - <span class="bold"><strong>flags</strong></span> field of that structure to - <a href="../api_reference/C/dbt.html#dbt_DB_DBT_USERMEM" class="olink">DB_DBT_USERMEM</a>. In the Berkeley DB C++ and Java APIs, the actions - are similar, although there are API-specific methods to set the <a href="../api_reference/C/dbt.html" class="olink">DBT</a> - values. Then, the <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> or <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a> flags are - specified to the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> methods, which cause multiple - records to be returned in the specified buffer. -</p> + The difference between <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> and + <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a> is as follows: <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> returns + multiple data items for a single key. For example, the + <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> flag would be used to retrieve all of the + duplicate data items for a single key in a single call. + The <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a> flag is used to retrieve multiple + key/data pairs, where each returned key may or may not + have duplicate data items. + </p> + <p> + Once the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method has returned, the + application will walk through the buffer handling the + returned records. This is implemented for the C and C++ + APIs using four macros: <a href="../api_reference/C/DB_MULTIPLE_INIT.html" class="olink">DB_MULTIPLE_INIT</a>, + <a href="../api_reference/C/DB_MULTIPLE_NEXT.html" class="olink">DB_MULTIPLE_NEXT</a>, <a href="../api_reference/C/DB_MULTIPLE_KEY_NEXT.html" class="olink">DB_MULTIPLE_KEY_NEXT</a>, and + <a href="../api_reference/C/DB_MULTIPLE_RECNO_NEXT.html" class="olink">DB_MULTIPLE_RECNO_NEXT</a>. For the Java API, this is + implemented as three iterator classes: <a class="ulink" href="../java/com/sleepycat/db/MultipleDataEntry.html" target="_top">MultipleDataEntry</a>, + <a class="ulink" href="../java/com/sleepycat/db/MultipleKeyDataEntry.html" target="_top">MultipleKeyDataEntry</a>, + and <a class="ulink" href="../java/com/sleepycat/db/MultipleRecnoDataEntry.html" target="_top"> + MultipleRecnoDataEntry</a>. + </p> <p> - The difference between <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> and <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a> is as - follows: <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> returns multiple data items for a single key. - For example, the <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> flag would be used to retrieve all of - the duplicate data items for a single key in a single call. The - <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a> flag is used to retrieve multiple key/data pairs, - where each returned key may or may not have duplicate data items. -</p> + The <a href="../api_reference/C/DB_MULTIPLE_INIT.html" class="olink">DB_MULTIPLE_INIT</a> macro is always called first. It + initializes a local application variable and the <span class="bold"><strong>data</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> for stepping through + the set of returned records. Then, the application calls + one of the remaining three macros: <a href="../api_reference/C/DB_MULTIPLE_NEXT.html" class="olink">DB_MULTIPLE_NEXT</a>, + <a href="../api_reference/C/DB_MULTIPLE_KEY_NEXT.html" class="olink">DB_MULTIPLE_KEY_NEXT</a>, and <a href="../api_reference/C/DB_MULTIPLE_RECNO_NEXT.html" class="olink">DB_MULTIPLE_RECNO_NEXT</a>. + </p> <p> - Once the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method has returned, the application will - walk through the buffer handling the returned records. This is - implemented for the C and C++ APIs using four macros: - <a href="../api_reference/C/DB_MULTIPLE_INIT.html" class="olink">DB_MULTIPLE_INIT</a>, <a href="../api_reference/C/DB_MULTIPLE_NEXT.html" class="olink">DB_MULTIPLE_NEXT</a>, <a href="../api_reference/C/DB_MULTIPLE_KEY_NEXT.html" class="olink">DB_MULTIPLE_KEY_NEXT</a>, and - <a href="../api_reference/C/DB_MULTIPLE_RECNO_NEXT.html" class="olink">DB_MULTIPLE_RECNO_NEXT</a>. For the Java API, this is implemented as - three iterator classes: - <a class="ulink" href="../java/com/sleepycat/db/MultipleDataEntry.html" target="_top">MultipleDataEntry</a>, - <a class="ulink" href="../java/com/sleepycat/db/MultipleKeyDataEntry.html" target="_top">MultipleKeyDataEntry</a>, - and <a class="ulink" href="../java/com/sleepycat/db/MultipleRecnoDataEntry.html" target="_top">MultipleRecnoDataEntry</a>. -</p> - <p> - The <a href="../api_reference/C/DB_MULTIPLE_INIT.html" class="olink">DB_MULTIPLE_INIT</a> macro is always called first. It initializes a - local application variable and the <span class="bold"><strong>data</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> for stepping through the set of - returned records. Then, the application calls one of the remaining - three macros: <a href="../api_reference/C/DB_MULTIPLE_NEXT.html" class="olink">DB_MULTIPLE_NEXT</a>, <a href="../api_reference/C/DB_MULTIPLE_KEY_NEXT.html" class="olink">DB_MULTIPLE_KEY_NEXT</a>, and - <a href="../api_reference/C/DB_MULTIPLE_RECNO_NEXT.html" class="olink">DB_MULTIPLE_RECNO_NEXT</a>. -</p> - <p> - If the <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> flag was specified to the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> - method, the application will always call the <a href="../api_reference/C/DB_MULTIPLE_NEXT.html" class="olink">DB_MULTIPLE_NEXT</a> macro. - If the <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a> flag was specified to the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> - method, and the underlying database is a Btree or Hash database, the - application will always call the <a href="../api_reference/C/DB_MULTIPLE_KEY_NEXT.html" class="olink">DB_MULTIPLE_KEY_NEXT</a> macro. If the - <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a> flag was specified to the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method, - and the underlying database is a Queue or Recno database, the - application will always call the <a href="../api_reference/C/DB_MULTIPLE_RECNO_NEXT.html" class="olink">DB_MULTIPLE_RECNO_NEXT</a> macro. The - <a href="../api_reference/C/DB_MULTIPLE_NEXT.html" class="olink">DB_MULTIPLE_NEXT</a>, <a href="../api_reference/C/DB_MULTIPLE_KEY_NEXT.html" class="olink">DB_MULTIPLE_KEY_NEXT</a>, and - <a href="../api_reference/C/DB_MULTIPLE_RECNO_NEXT.html" class="olink">DB_MULTIPLE_RECNO_NEXT</a> macros are called repeatedly, until the end of - the returned records is reached. The end of the returned records is - detected by the application's local pointer variable being set to NULL. -</p> - <p> - Note that if you want to use a cursor for bulk retrieval of records in - a Btree database, you should open the cursor using the - <code class="literal">DB_CURSOR_BULK</code> flag. This optimizes the cursor for - bulk retrieval. -</p> - <p> - The following is an example of a routine that displays the contents of - a Btree database using the bulk return interfaces. -</p> + If the <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> flag was specified to the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> + or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method, the application will always call the + <a href="../api_reference/C/DB_MULTIPLE_NEXT.html" class="olink">DB_MULTIPLE_NEXT</a> macro. If the <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a> flag + was specified to the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method, and the + underlying database is a Btree or Hash database, the + application will always call the <a href="../api_reference/C/DB_MULTIPLE_KEY_NEXT.html" class="olink">DB_MULTIPLE_KEY_NEXT</a> + macro. If the <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a> flag was specified to the + <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method, and the underlying database is + a Queue or Recno database, the application will always + call the <a href="../api_reference/C/DB_MULTIPLE_RECNO_NEXT.html" class="olink">DB_MULTIPLE_RECNO_NEXT</a> macro. The + <a href="../api_reference/C/DB_MULTIPLE_NEXT.html" class="olink">DB_MULTIPLE_NEXT</a>, <a href="../api_reference/C/DB_MULTIPLE_KEY_NEXT.html" class="olink">DB_MULTIPLE_KEY_NEXT</a>, and + <a href="../api_reference/C/DB_MULTIPLE_RECNO_NEXT.html" class="olink">DB_MULTIPLE_RECNO_NEXT</a> macros are called repeatedly, + until the end of the returned records is reached. The end + of the returned records is detected by the application's + local pointer variable being set to NULL. + </p> + <p> + Note that if you want to use a cursor for bulk + retrieval of records in a Btree database, you should open + the cursor using the <code class="literal">DB_CURSOR_BULK</code> + flag. This optimizes the cursor for bulk retrieval. + </p> + <p> + The following is an example of a routine that displays + the contents of a Btree database using the bulk return + interfaces. + </p> <a id="prog_am22"></a> <pre class="programlisting">int rec_display(DB *dbp) @@ -205,13 +214,34 @@ rec_display(DB *dbp) </div> </div> </div> - <p>To put records in bulk with the btree or hash access methods, construct bulk buffers in the <span class="bold"><strong>key</strong></span> and <span class="bold"><strong>data</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> using <a href="../api_reference/C/DB_MULTIPLE_WRITE_INIT.html" class="olink">DB_MULTIPLE_WRITE_INIT</a> and <a href="../api_reference/C/DB_MULTIPLE_WRITE_NEXT.html" class="olink">DB_MULTIPLE_WRITE_NEXT</a>. To put records in bulk with the recno or queue access methods, construct bulk buffers in the <span class="bold"><strong>data</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> as before, but construct the <span class="bold"><strong>key</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> using <a href="../api_reference/C/DB_MULTIPLE_RECNO_WRITE_INIT.html" class="olink">DB_MULTIPLE_RECNO_WRITE_INIT</a> and <a href="../api_reference/C/DB_MULTIPLE_RECNO_WRITE_NEXT.html" class="olink">DB_MULTIPLE_RECNO_WRITE_NEXT</a> with a data size of zero;. In both cases, set the <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> flag to <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a>.</p> - <p>Alternatively, for btree and hash access methods, construct a single bulk buffer in the <span class="bold"><strong>key</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> using <a href="../api_reference/C/DB_MULTIPLE_WRITE_INIT.html" class="olink">DB_MULTIPLE_WRITE_INIT</a> and <a href="../api_reference/C/DB_MULTIPLE_KEY_WRITE_NEXT.html" class="olink">DB_MULTIPLE_KEY_WRITE_NEXT</a>. For recno and queue access methods, construct a bulk buffer in the <span class="bold"><strong>key</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> using <a href="../api_reference/C/DB_MULTIPLE_RECNO_WRITE_INIT.html" class="olink">DB_MULTIPLE_RECNO_WRITE_INIT</a> and <a href="../api_reference/C/DB_MULTIPLE_RECNO_WRITE_NEXT.html" class="olink">DB_MULTIPLE_RECNO_WRITE_NEXT</a>. In both cases, set the <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a> flag to <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a>. - -</p> - <p>A successful bulk operation is logically equivalent to a loop through each key/data pair, performing a <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> for each one.</p> <p> -</p> + To put records in bulk with the btree or hash access + methods, construct bulk buffers in the <span class="bold"><strong>key</strong></span> and <span class="bold"><strong>data</strong></span> + <a href="../api_reference/C/dbt.html" class="olink">DBT</a> using <a href="../api_reference/C/DB_MULTIPLE_WRITE_INIT.html" class="olink">DB_MULTIPLE_WRITE_INIT</a> + and <a href="../api_reference/C/DB_MULTIPLE_WRITE_NEXT.html" class="olink">DB_MULTIPLE_WRITE_NEXT</a>. To put records in bulk with + the recno or queue access methods, construct bulk buffers + in the <span class="bold"><strong>data</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> as + before, but construct the <span class="bold"><strong>key</strong></span> + <a href="../api_reference/C/dbt.html" class="olink">DBT</a> using <a href="../api_reference/C/DB_MULTIPLE_RECNO_WRITE_INIT.html" class="olink">DB_MULTIPLE_RECNO_WRITE_INIT</a> and + <a href="../api_reference/C/DB_MULTIPLE_RECNO_WRITE_NEXT.html" class="olink">DB_MULTIPLE_RECNO_WRITE_NEXT</a> with a data size of zero;. + In both cases, set the <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> flag to + <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a>. + </p> + <p> + Alternatively, for btree and hash access methods, + construct a single bulk buffer in the <span class="bold"><strong>key</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> using + <a href="../api_reference/C/DB_MULTIPLE_WRITE_INIT.html" class="olink">DB_MULTIPLE_WRITE_INIT</a> and <a href="../api_reference/C/DB_MULTIPLE_KEY_WRITE_NEXT.html" class="olink">DB_MULTIPLE_KEY_WRITE_NEXT</a>. + For recno and queue access methods, construct a bulk + buffer in the <span class="bold"><strong>key</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> + using <a href="../api_reference/C/DB_MULTIPLE_RECNO_WRITE_INIT.html" class="olink">DB_MULTIPLE_RECNO_WRITE_INIT</a> and + <a href="../api_reference/C/DB_MULTIPLE_RECNO_WRITE_NEXT.html" class="olink">DB_MULTIPLE_RECNO_WRITE_NEXT</a>. In both cases, set the + <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a> flag to <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a>. + </p> + <p> + A successful bulk operation is logically equivalent to a loop through + each key/data pair, performing a <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> for each + one. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> @@ -221,9 +251,40 @@ rec_display(DB *dbp) </div> </div> </div> - <p>To delete all records with a specified set of keys with the btree or hash access methods, construct a bulk buffer in the <span class="bold"><strong>key</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> using <a href="../api_reference/C/DB_MULTIPLE_WRITE_INIT.html" class="olink">DB_MULTIPLE_WRITE_INIT</a> and <a href="../api_reference/C/DB_MULTIPLE_WRITE_NEXT.html" class="olink">DB_MULTIPLE_WRITE_NEXT</a>. To delete a set of records with the recno or queue access methods, construct the <span class="bold"><strong>key</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> using <a href="../api_reference/C/DB_MULTIPLE_RECNO_WRITE_INIT.html" class="olink">DB_MULTIPLE_RECNO_WRITE_INIT</a> and <a href="../api_reference/C/DB_MULTIPLE_RECNO_WRITE_NEXT.html" class="olink">DB_MULTIPLE_RECNO_WRITE_NEXT</a> with a data size of zero. In both cases, set the <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> flag to <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a>. This is equivalent to calling <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a> for each key in the bulk buffer. In particular, if the database supports duplicates, all records with the matching key are deleted.</p> - <p>Alternatively, to delete a specific set of key/data pairs, which may be items within a set of duplicates, there are also two cases depending on whether the access method uses record numbers for keys. For btree and hash access methods, construct a single bulk buffer in the <span class="bold"><strong>key</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> using <a href="../api_reference/C/DB_MULTIPLE_WRITE_INIT.html" class="olink">DB_MULTIPLE_WRITE_INIT</a> and <a href="../api_reference/C/DB_MULTIPLE_KEY_WRITE_NEXT.html" class="olink">DB_MULTIPLE_KEY_WRITE_NEXT</a>. For recno and queue access methods, construct a bulk buffer in the <span class="bold"><strong>key</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> using <a href="../api_reference/C/DB_MULTIPLE_RECNO_WRITE_INIT.html" class="olink">DB_MULTIPLE_RECNO_WRITE_INIT</a> and <a href="../api_reference/C/DB_MULTIPLE_RECNO_WRITE_NEXT.html" class="olink">DB_MULTIPLE_RECNO_WRITE_NEXT</a>. In both cases, set the <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a> flag to <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a>.</p> - <p>A successful bulk operation is logically equivalent to a loop through each key/data pair, performing a <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a> for each one.</p> + <p> + To delete all records with a specified set of keys with + the btree or hash access methods, construct a bulk buffer + in the <span class="bold"><strong>key</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> using + <a href="../api_reference/C/DB_MULTIPLE_WRITE_INIT.html" class="olink">DB_MULTIPLE_WRITE_INIT</a> and <a href="../api_reference/C/DB_MULTIPLE_WRITE_NEXT.html" class="olink">DB_MULTIPLE_WRITE_NEXT</a>. To + delete a set of records with the recno or queue access + methods, construct the <span class="bold"><strong>key</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> using + <a href="../api_reference/C/DB_MULTIPLE_RECNO_WRITE_INIT.html" class="olink">DB_MULTIPLE_RECNO_WRITE_INIT</a> and + <a href="../api_reference/C/DB_MULTIPLE_RECNO_WRITE_NEXT.html" class="olink">DB_MULTIPLE_RECNO_WRITE_NEXT</a> with a data size of zero. + In both cases, set the <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE" class="olink">DB_MULTIPLE</a> flag to <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a>. This + is equivalent to calling <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a> for each key in the bulk + buffer. In particular, if the database supports + duplicates, all records with the matching key are + deleted. + </p> + <p> + Alternatively, to delete a specific set of key/data + pairs, which may be items within a set of duplicates, + there are also two cases depending on whether the access + method uses record numbers for keys. For btree and hash + access methods, construct a single bulk buffer in the + <span class="bold"><strong>key</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> using + <a href="../api_reference/C/DB_MULTIPLE_WRITE_INIT.html" class="olink">DB_MULTIPLE_WRITE_INIT</a> and <a href="../api_reference/C/DB_MULTIPLE_KEY_WRITE_NEXT.html" class="olink">DB_MULTIPLE_KEY_WRITE_NEXT</a>. + For recno and queue access methods, construct a bulk + buffer in the <span class="bold"><strong>key</strong></span> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> + using <a href="../api_reference/C/DB_MULTIPLE_RECNO_WRITE_INIT.html" class="olink">DB_MULTIPLE_RECNO_WRITE_INIT</a> and + <a href="../api_reference/C/DB_MULTIPLE_RECNO_WRITE_NEXT.html" class="olink">DB_MULTIPLE_RECNO_WRITE_NEXT</a>. In both cases, set the + <a href="../api_reference/C/dbcget.html#dbcget_DB_MULTIPLE_KEY" class="olink">DB_MULTIPLE_KEY</a> flag to <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a>. + </p> + <p> + A successful bulk operation is logically equivalent to a + loop through each key/data pair, performing a <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a> for + each one. + </p> </div> </div> <div class="navfooter"> @@ -237,13 +298,12 @@ rec_display(DB *dbp) <td width="40%" align="right"> <a accesskey="n" href="am_misc_partial.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 4. - Access Method Wrapup - </td> + <td width="40%" align="left" valign="top">Chapter 4. Access Method Wrapup </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Partial record storage and retrieval</td> + <td width="40%" align="right" valign="top"> Partial record storage and + retrieval</td> </tr> </table> </div> diff --git a/docs/programmer_reference/am_misc_db_sql.html b/docs/programmer_reference/am_misc_db_sql.html index 549135d3..1732a538 100644 --- a/docs/programmer_reference/am_misc_db_sql.html +++ b/docs/programmer_reference/am_misc_db_sql.html @@ -8,23 +8,22 @@ <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> <link rel="up" href="am_misc.html" title="Chapter 4. Access Method Wrapup" /> - <link rel="prev" href="am_misc_diskspace.html" title="Disk space requirements" /> + <link rel="prev" href="blobs.html" title="BLOB support" /> <link rel="next" href="am_misc_tune.html" title="Access method tuning" /> </head> <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Specifying a Berkeley DB schema using SQL DDL</th> + <th colspan="3" align="center">Specifying a Berkeley DB schema + using SQL DDL</th> </tr> <tr> - <td width="20%" align="left"><a accesskey="p" href="am_misc_diskspace.html">Prev</a> </td> - <th width="60%" align="center">Chapter 4. - Access Method Wrapup - </th> + <td width="20%" align="left"><a accesskey="p" href="blobs.html">Prev</a> </td> + <th width="60%" align="center">Chapter 4. Access Method Wrapup </th> <td width="20%" align="right"> <a accesskey="n" href="am_misc_tune.html">Next</a></td> </tr> </table> @@ -34,64 +33,70 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="am_misc_db_sql"></a>Specifying a Berkeley DB schema using SQL DDL</h2> + <h2 class="title" style="clear: both"><a id="am_misc_db_sql"></a>Specifying a Berkeley DB schema + using SQL DDL</h2> </div> </div> </div> - <p> - When starting a new Berkeley DB project, much of the code that you - must write is dedicated to defining the BDB environment: what - databases it contains, the types of the databases, and so forth. - Also, since records in BDB are just byte arrays, you must write - code that assembles and interprets these byte arrays. + <p> + When starting a new Berkeley DB project, much of the code + that you must write is dedicated to defining the BDB + environment: what databases it contains, the types of the + databases, and so forth. Also, since records in BDB are just + byte arrays, you must write code that assembles and interprets + these byte arrays. </p> - <p> - Much of this code can be written automatically (in C) by the - db_sql_codegen utility. To use it, you first specify the schema of your Berkeley - DB environment in SQL Data Definition Language (DDL). Then you - invoke the db_sql_codegen command, giving the DDL as input. - <span class="command"><strong>db_sql_codegen</strong></span> reads the DDL, and writes C code that - implements a storage-layer API suggested by the DDL. + <p> + Much of this code can be written automatically (in C) by + the db_sql_codegen utility. To use it, you first specify the + schema of your Berkeley DB environment in SQL Data Definition + Language (DDL). Then you invoke the db_sql_codegen command, + giving the DDL as input. <span class="command"><strong>db_sql_codegen</strong></span> + reads the DDL, and writes C code that implements a + storage-layer API suggested by the DDL. </p> <p> The generated API includes a general-purpose initialization - function, which sets up the environment and the databases (creating - them if they don't already exist). It also includes C structure - declarations for each record type, and numerous specialized - functions for storing and retrieving those records. + function, which sets up the environment and the databases + (creating them if they don't already exist). It also includes + C structure declarations for each record type, and numerous + specialized functions for storing and retrieving those + records. </p> <p> - <span class="command"><strong>db_sql_codegen</strong></span> can also produce a simple test program - that exercises the generated API. This program is useful as an - example of how to use the API. It contains calls to all of the - interface functions, along with commentary explaining what the code - is doing. + <span class="command"><strong>db_sql_codegen</strong></span> can also produce a simple + test program that exercises the generated API. This program is + useful as an example of how to use the API. It contains calls + to all of the interface functions, along with commentary + explaining what the code is doing. </p> - <p> - Once the storage layer API is produced, your application may use it - as is, or you may customize it as much as you like by editing the - generated source code. Be warned, however: - <span class="command"><strong>db_sql_codegen</strong></span> is a one-way process; there is no way to - automatically incorporate customizations into newly generated code, - if you decide to run <span class="command"><strong>db_sql_codegen</strong></span> again. + <p> + Once the storage layer API is produced, your application + may use it as is, or you may customize it as much as you like + by editing the generated source code. Be warned, however: + <span class="command"><strong>db_sql_codegen</strong></span> is a one-way process; + there is no way to automatically incorporate customizations + into newly generated code, if you decide to run + <span class="command"><strong>db_sql_codegen</strong></span> again. </p> <p> - To learn more about <span class="command"><strong>db_sql_codegen</strong></span>, please consult the - db_sql_codegen utility manual page in the Berkeley DB C API Reference Guide. + To learn more about <span class="command"><strong>db_sql_codegen</strong></span>, + please consult the db_sql_codegen utility manual page in the + Berkeley DB C API Reference Guide. </p> </div> <div class="navfooter"> <hr /> <table width="100%" summary="Navigation footer"> <tr> - <td width="40%" align="left"><a accesskey="p" href="am_misc_diskspace.html">Prev</a> </td> + <td width="40%" align="left"><a accesskey="p" href="blobs.html">Prev</a> </td> <td width="20%" align="center"> <a accesskey="u" href="am_misc.html">Up</a> </td> <td width="40%" align="right"> <a accesskey="n" href="am_misc_tune.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Disk space requirements </td> + <td width="40%" align="left" valign="top">BLOB support </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/am_misc_dbsizes.html b/docs/programmer_reference/am_misc_dbsizes.html index 2cae3130..dac7a08e 100644 --- a/docs/programmer_reference/am_misc_dbsizes.html +++ b/docs/programmer_reference/am_misc_dbsizes.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_misc_stability.html">Prev</a> </td> - <th width="60%" align="center">Chapter 4. - Access Method Wrapup - </th> + <th width="60%" align="center">Chapter 4. Access Method Wrapup </th> <td width="20%" align="right"> <a accesskey="n" href="am_misc_diskspace.html">Next</a></td> </tr> </table> @@ -38,30 +36,52 @@ </div> </div> </div> - <p>The largest database file that Berkeley DB can handle depends on the page size -selected by the application. Berkeley DB stores database file page numbers as -unsigned 32-bit numbers and database file page sizes as unsigned 16-bit -numbers. Using the maximum database page size of 65536, this results in -a maximum database file size of 2<sup>48</sup> (256 terabytes). The -minimum database page size is 512 bytes, which results in a minimum -maximum database size of 2<sup>41</sup> (2 terabytes).</p> - <p>The largest database file Berkeley DB can support is potentially further limited -if the host system does not have filesystem support for files larger than -2<sup>32</sup>, including the ability to seek to absolute offsets within -those files.</p> - <p>The largest key or data item that Berkeley DB can support is 2<sup>32</sup>, -or more likely limited by available memory. Specifically, while key and -data byte strings may be of essentially unlimited length, any one of -them must fit into available memory so that it can be returned to the -application. As some of the Berkeley DB interfaces return both key and data -items to the application, those interfaces will require that any -key/data pair fit simultaneously into memory. Further, as the access -methods may need to compare key and data items with other key and data -items, it may be a requirement that any two key or two data items fit -into available memory. Finally, when writing applications supporting -transactions, it may be necessary to have an additional copy of any data -item in memory for logging purposes.</p> - <p>The maximum Btree depth is 255.</p> + <p> + The largest database file that Berkeley DB can handle + depends on the page size selected by the application. Berkeley + DB stores database file page numbers as unsigned 32-bit + numbers and database file page sizes as unsigned 16-bit + numbers. Using the maximum database page size of 65536, this + results in a maximum database file size of + 2<sup>48</sup> (256 terabytes). The + minimum database page size is 512 bytes, which results in a + minimum maximum database size of + 2<sup>41</sup> (2 terabytes). + </p> + <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> + <h3 class="title">Note</h3> + <p> + In order to store petabytes of data you can use + multiple database files. + </p> + </div> + <p> + The largest database file Berkeley DB can support is + potentially further limited if the host system does not have + filesystem support for files larger than + 2<sup>32</sup>, including the ability to + seek to absolute offsets within those files. + </p> + <p> + The largest key or data item that Berkeley DB can support + is 2<sup>32</sup>, or more likely limited by + available memory. Specifically, while key and data byte + strings may be of essentially unlimited length, any one of + them must fit into available memory so that it can be returned + to the application. As some of the Berkeley DB interfaces + return both key and data items to the application, those + interfaces will require that any key/data pair fit + simultaneously into memory. Further, as the access methods may + need to compare key and data items with other key and data + items, it may be a requirement that any two key or two data + items fit into available memory. Finally, when writing + applications supporting transactions, it may be necessary to + have an additional copy of any data item in memory for logging + purposes. + </p> + <p> + The maximum Btree depth is 255. + </p> </div> <div class="navfooter"> <hr /> @@ -78,7 +98,8 @@ item in memory for logging purposes.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Disk space requirements</td> + <td width="40%" align="right" valign="top"> Disk space + requirements</td> </tr> </table> </div> diff --git a/docs/programmer_reference/am_misc_diskspace.html b/docs/programmer_reference/am_misc_diskspace.html index 8d33bd15..2f565f81 100644 --- a/docs/programmer_reference/am_misc_diskspace.html +++ b/docs/programmer_reference/am_misc_diskspace.html @@ -9,23 +9,22 @@ <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> <link rel="up" href="am_misc.html" title="Chapter 4. Access Method Wrapup" /> <link rel="prev" href="am_misc_dbsizes.html" title="Database limits" /> - <link rel="next" href="am_misc_db_sql.html" title="Specifying a Berkeley DB schema using SQL DDL" /> + <link rel="next" href="blobs.html" title="BLOB support" /> </head> <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Disk space requirements</th> + <th colspan="3" align="center">Disk space + requirements</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_misc_dbsizes.html">Prev</a> </td> - <th width="60%" align="center">Chapter 4. - Access Method Wrapup - </th> - <td width="20%" align="right"> <a accesskey="n" href="am_misc_db_sql.html">Next</a></td> + <th width="60%" align="center">Chapter 4. Access Method Wrapup </th> + <td width="20%" align="right"> <a accesskey="n" href="blobs.html">Next</a></td> </tr> </table> <hr /> @@ -34,7 +33,8 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="am_misc_diskspace"></a>Disk space requirements</h2> + <h2 class="title" style="clear: both"><a id="am_misc_diskspace"></a>Disk space + requirements</h2> </div> </div> </div> @@ -42,158 +42,224 @@ <dl> <dt> <span class="sect2"> - <a href="am_misc_diskspace.html#idp1074008">Btree</a> + <a href="am_misc_diskspace.html#idp595712">Btree</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_misc_diskspace.html#idp1074072">Hash</a> + <a href="am_misc_diskspace.html#idp595776">Hash</a> </span> </dt> </dl> </div> - <p>It is possible to estimate the total database size based on the size of -the data. The following calculations are an estimate of how many bytes -you will need to hold a set of data and then how many pages it will take -to actually store it on disk.</p> - <p>Space freed by deleting key/data pairs from a Btree or Hash database is -never returned to the filesystem, although it is reused where possible. -This means that the Btree and Hash databases are grow-only. If enough -keys are deleted from a database that shrinking the underlying file is -desirable, you should use the <a href="../api_reference/C/dbcompact.html" class="olink">DB->compact()</a> method to reclaim disk space. Alternatively, -you can create a new database and copy the records from -the old one into it.</p> - <p>These are rough estimates at best. For example, they do not take into -account overflow records, filesystem metadata information, large sets -of duplicate data items (where the key is only stored once), or -real-life situations where the sizes of key and data items are wildly -variable, and the page-fill factor changes over time.</p> + <p> + It is possible to estimate the total database size based on + the size of the data. The following calculations are an + estimate of how many bytes you will need to hold a set of data + and then how many pages it will take to actually store it on + disk. + </p> + <p> + Space freed by deleting key/data pairs from a Btree or Hash + database is never returned to the filesystem, although it is + reused where possible. This means that the Btree and Hash + databases are grow-only. If enough keys are deleted from a + database that shrinking the underlying file is desirable, you + should use the <a href="../api_reference/C/dbcompact.html" class="olink">DB->compact()</a> method to reclaim disk space. + Alternatively, you can create a new database and copy the + records from the old one into it. + </p> + <p> + These are rough estimates at best. For example, they do not + take into account overflow records, filesystem metadata + information, large sets of duplicate data items (where the key + is only stored once), or real-life situations where the sizes + of key and data items are wildly variable, and the page-fill + factor changes over time. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1074008"></a>Btree</h3> + <h3 class="title"><a id="idp595712"></a>Btree</h3> </div> </div> </div> - <p>The formulas for the Btree access method are as follows:</p> + <p> + The formulas for the Btree access method are as + follows: + </p> <pre class="programlisting">useful-bytes-per-page = (page-size - page-overhead) * page-fill-factor -<p></p> + bytes-of-data = n-records * - (bytes-per-entry + page-overhead-for-two-entries) -<p></p> -n-pages-of-data = bytes-of-data / useful-bytes-per-page -<p></p> -total-bytes-on-disk = n-pages-of-data * page-size -</pre> - <p>The <span class="bold"><strong>useful-bytes-per-page</strong></span> is a measure of the bytes on each page -that will actually hold the application data. It is computed as the total -number of bytes on the page that are available to hold application data, -corrected by the percentage of the page that is likely to contain data. -The reason for this correction is that the percentage of a page that -contains application data can vary from close to 50% after a page split -to almost 100% if the entries in the database were inserted in sorted -order. Obviously, the <span class="bold"><strong>page-fill-factor</strong></span> can drastically alter -the amount of disk space required to hold any particular data set. The -page-fill factor of any existing database can be displayed using the -<a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility.</p> - <p>The page-overhead for Btree databases is 26 bytes. As an example, using -an 8K page size, with an 85% page-fill factor, there are 6941 bytes of -useful space on each page:</p> +(bytes-per-entry + page-overhead-for-two-entries) + +n-pages-of-data = bytes-of-data / useful-bytes-per-page + +total-bytes-on-disk = n-pages-of-data * page-size</pre> + <p> + The <span class="bold"><strong>useful-bytes-per-page</strong></span> is a + measure of the bytes on each page that will actually hold the application + data. It is computed as the total number of bytes on the + page that are available to hold application data, + corrected by the percentage of the page that is likely to + contain data. The reason for this correction is that the + percentage of a page that contains application data can + vary from close to 50% after a page split to almost 100% + if the entries in the database were inserted in sorted + order. Obviously, the <span class="bold"><strong>page-fill-factor</strong></span> + can drastically alter the amount of disk space required to hold any + particular data set. The page-fill factor of any existing database can be + displayed using the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility. + </p> + <p> + The page-overhead for Btree databases is 26 bytes. As an + example, using an 8K page size, with an 85% page-fill + factor, there are 6941 bytes of useful space on each + page: + </p> <pre class="programlisting">6941 = (8192 - 26) * .85</pre> - <p>The total <span class="bold"><strong>bytes-of-data</strong></span> is an easy calculation: It is the -number of key or data items plus the overhead required to store each -item on a page. The overhead to store a key or data item on a Btree -page is 5 bytes. So, it would take 1560000000 bytes, or roughly 1.34GB -of total data to store 60,000,000 key/data pairs, assuming each key or -data item was 8 bytes long:</p> + <p> + The total <span class="bold"><strong>bytes-of-data</strong></span> + is an easy calculation: It is the number of key or data + items plus the overhead required to store each item on a + page. The overhead to store a key or data item on a Btree + page is 5 bytes. So, it would take 1560000000 bytes, or + roughly 1.34GB of total data to store 60,000,000 key/data + pairs, assuming each key or data item was 8 bytes + long: + </p> <pre class="programlisting">1560000000 = 60000000 * ((8 + 5) * 2)</pre> - <p>The total pages of data, <span class="bold"><strong>n-pages-of-data</strong></span>, is the -<span class="bold"><strong>bytes-of-data</strong></span> divided by the <span class="bold"><strong>useful-bytes-per-page</strong></span>. In -the example, there are 224751 pages of data.</p> + <p> + The total pages of data, <span class="bold"><strong>n-pages-of-data</strong></span>, + is the <span class="bold"><strong>bytes-of-data</strong></span> divided by the + <span class="bold"><strong>useful-bytes-per-page</strong></span>. In the example, + there are 224751 pages of data. + </p> <pre class="programlisting">224751 = 1560000000 / 6941</pre> - <p>The total bytes of disk space for the database is <span class="bold"><strong>n-pages-of-data</strong></span> -multiplied by the <span class="bold"><strong>page-size</strong></span>. In the example, the result is -1841160192 bytes, or roughly 1.71GB.</p> + <p> + The total bytes of disk space for the database is + <span class="bold"><strong>n-pages-of-data</strong></span> + multiplied by the <span class="bold"><strong>page-size</strong></span>. In the + example, the result is 1841160192 bytes, or roughly 1.71GB. + </p> <pre class="programlisting">1841160192 = 224751 * 8192</pre> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1074072"></a>Hash</h3> + <h3 class="title"><a id="idp595776"></a>Hash</h3> </div> </div> </div> - <p>The formulas for the Hash access method are as follows:</p> + <p> + The formulas for the Hash access method are as + follows: + </p> <pre class="programlisting">useful-bytes-per-page = (page-size - page-overhead) -<p></p> + bytes-of-data = n-records * - (bytes-per-entry + page-overhead-for-two-entries) -<p></p> +(bytes-per-entry + page-overhead-for-two-entries) + n-pages-of-data = bytes-of-data / useful-bytes-per-page -<p></p> -total-bytes-on-disk = n-pages-of-data * page-size -</pre> - <p>The <span class="bold"><strong>useful-bytes-per-page</strong></span> is a measure of the bytes on each page -that will actually hold the application data. It is computed as the total -number of bytes on the page that are available to hold application data. -If the application has explicitly set a page-fill factor, pages will -not necessarily be kept full. For databases with a preset fill factor, -see the calculation below. The page-overhead for Hash databases is 26 -bytes and the page-overhead-for-two-entries is 6 bytes.</p> - <p>As an example, using an 8K page size, there are 8166 bytes of useful space -on each page:</p> + +total-bytes-on-disk = n-pages-of-data * page-size</pre> + <p> + The <span class="bold"><strong>useful-bytes-per-page</strong></span> is a measure + of the bytes on each page that will actually hold the application + data. It is computed as the total number of bytes on the + page that are available to hold application data. If the + application has explicitly set a page-fill factor, pages + will not necessarily be kept full. For databases with a + preset fill factor, see the calculation below. The + page-overhead for Hash databases is 26 bytes and the + page-overhead-for-two-entries is 6 bytes. + </p> + <p> + As an example, using an 8K page size, there are 8166 + bytes of useful space on each page: + </p> <pre class="programlisting">8166 = (8192 - 26)</pre> - <p>The total <span class="bold"><strong>bytes-of-data</strong></span> is an easy calculation: it is the number -of key/data pairs plus the overhead required to store each pair on a page. -In this case that's 6 bytes per pair. So, assuming 60,000,000 key/data -pairs, each of which is 8 bytes long, there are 1320000000 bytes, or -roughly 1.23GB of total data:</p> + <p> + The total <span class="bold"><strong>bytes-of-data</strong></span> + is an easy calculation: it is the number of key/data pairs + plus the overhead required to store each pair on a page. + In this case that's 6 bytes per pair. So, assuming + 60,000,000 key/data pairs, each of which is 8 bytes long, + there are 1320000000 bytes, or roughly 1.23GB of total + data: + </p> <pre class="programlisting">1320000000 = 60000000 * (16 + 6)</pre> - <p>The total pages of data, <span class="bold"><strong>n-pages-of-data</strong></span>, is the -<span class="bold"><strong>bytes-of-data</strong></span> divided by the <span class="bold"><strong>useful-bytes-per-page</strong></span>. In -this example, there are 161646 pages of data.</p> + <p> + The total pages of data, <span class="bold"><strong>n-pages-of-data</strong></span>, + is the <span class="bold"><strong>bytes-of-data</strong></span> divided by the + <span class="bold"><strong>useful-bytes-per-page</strong></span>. In this example, + there are 161646 pages of data. + </p> <pre class="programlisting">161646 = 1320000000 / 8166</pre> - <p>The total bytes of disk space for the database is <span class="bold"><strong>n-pages-of-data</strong></span> -multiplied by the <span class="bold"><strong>page-size</strong></span>. In the example, the result is -1324204032 bytes, or roughly 1.23GB.</p> + <p> + The total bytes of disk space for the database is + <span class="bold"><strong>n-pages-of-data</strong></span> + multiplied by the <span class="bold"><strong>page-size</strong></span>. In the + example, the result is 1324204032 bytes, or roughly 1.23GB. + </p> <pre class="programlisting">1324204032 = 161646 * 8192</pre> - <p>Now, let's assume that the application specified a fill factor explicitly. -The fill factor indicates the target number of items to place on a single -page (a fill factor might reduce the utilization of each page, but it can -be useful in avoiding splits and preventing buckets from becoming too -large). Using our estimates above, each item is 22 bytes (16 + 6), and -there are 8166 useful bytes on a page (8192 - 26). That means that, on -average, you can fit 371 pairs per page.</p> + <p> + Now, let's assume that the application specified a fill + factor explicitly. The fill factor indicates the target + number of items to place on a single page (a fill factor + might reduce the utilization of each page, but it can be + useful in avoiding splits and preventing buckets from + becoming too large). Using our estimates above, each item + is 22 bytes (16 + 6), and there are 8166 useful bytes on a + page (8192 - 26). That means that, on average, you can fit + 371 pairs per page. + </p> <pre class="programlisting">371 = 8166 / 22</pre> - <p>However, let's assume that the application designer knows that although -most items are 8 bytes, they can sometimes be as large as 10, and it's -very important to avoid overflowing buckets and splitting. Then, the -application might specify a fill factor of 314.</p> + <p> + However, let's assume that the application designer + knows that although most items are 8 bytes, they can + sometimes be as large as 10, and it's very important to + avoid overflowing buckets and splitting. Then, the + application might specify a fill factor of 314. + </p> <pre class="programlisting">314 = 8166 / 26</pre> - <p>With a fill factor of 314, then the formula for computing database size -is</p> + <p> + With a fill factor of 314, then the formula for + computing database size is + </p> <pre class="programlisting">n-pages-of-data = npairs / pairs-per-page</pre> - <p>or 191082.</p> + <p> + or 191082. + </p> <pre class="programlisting">191082 = 60000000 / 314</pre> - <p>At 191082 pages, the total database size would be 1565343744, or 1.46GB.</p> + <p> + At 191082 pages, the total database size would be + 1565343744, or 1.46GB. + </p> <pre class="programlisting">1565343744 = 191082 * 8192</pre> - <p>There are a few additional caveats with respect to Hash databases. This -discussion assumes that the hash function does a good job of evenly -distributing keys among hash buckets. If the function does not do this, -you may find your table growing significantly larger than you expected. -Secondly, in order to provide support for Hash databases coexisting with -other databases in a single file, pages within a Hash database are -allocated in power-of-two chunks. That means that a Hash database with 65 -buckets will take up as much space as a Hash database with 128 buckets; -each time the Hash database grows beyond its current power-of-two number -of buckets, it allocates space for the next power-of-two buckets. This -space may be sparsely allocated in the file system, but the files will -appear to be their full size. Finally, because of this need for -contiguous allocation, overflow pages and duplicate pages can be allocated -only at specific points in the file, and this too can lead to sparse hash -tables.</p> + <p> + There are a few additional caveats with respect to Hash + databases. This discussion assumes that the hash function + does a good job of evenly distributing keys among hash + buckets. If the function does not do this, you may find + your table growing significantly larger than you expected. + Secondly, in order to provide support for Hash databases + coexisting with other databases in a single file, pages + within a Hash database are allocated in power-of-two + chunks. That means that a Hash database with 65 buckets + will take up as much space as a Hash database with 128 + buckets; each time the Hash database grows beyond its + current power-of-two number of buckets, it allocates space + for the next power-of-two buckets. This space may be + sparsely allocated in the file system, but the files will + appear to be their full size. Finally, because of this + need for contiguous allocation, overflow pages and + duplicate pages can be allocated only at specific points + in the file, and this too can lead to sparse hash + tables. + </p> </div> </div> <div class="navfooter"> @@ -204,14 +270,14 @@ tables.</p> <td width="20%" align="center"> <a accesskey="u" href="am_misc.html">Up</a> </td> - <td width="40%" align="right"> <a accesskey="n" href="am_misc_db_sql.html">Next</a></td> + <td width="40%" align="right"> <a accesskey="n" href="blobs.html">Next</a></td> </tr> <tr> <td width="40%" align="left" valign="top">Database limits </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Specifying a Berkeley DB schema using SQL DDL</td> + <td width="40%" align="right" valign="top"> BLOB support</td> </tr> </table> </div> diff --git a/docs/programmer_reference/am_misc_error.html b/docs/programmer_reference/am_misc_error.html index 1739d9a0..df4e5149 100644 --- a/docs/programmer_reference/am_misc_error.html +++ b/docs/programmer_reference/am_misc_error.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_misc_perm.html">Prev</a> </td> - <th width="60%" align="center">Chapter 4. - Access Method Wrapup - </th> + <th width="60%" align="center">Chapter 4. Access Method Wrapup </th> <td width="20%" align="right"> <a accesskey="n" href="am_misc_stability.html">Next</a></td> </tr> </table> @@ -38,12 +36,20 @@ </div> </div> </div> - <p>Berkeley DB offers programmatic support for displaying error return values.</p> - <p>The <a href="../api_reference/C/envstrerror.html" class="olink">db_strerror()</a> function returns a pointer to the error -message corresponding to any Berkeley DB error return, similar to the ANSI C -strerror function, but is able to handle both system error returns and -Berkeley DB specific return values.</p> - <p>For example:</p> + <p> + Berkeley DB offers programmatic support for displaying error + return values. + </p> + <p> + The <a href="../api_reference/C/envstrerror.html" class="olink">db_strerror()</a> function returns a pointer to the error + message corresponding to any Berkeley DB error return, similar + to the ANSI C strerror function, but is able to handle both + system error returns and Berkeley DB specific return + values. + </p> + <p> + For example: + </p> <a id="prog_am25"></a> <pre class="programlisting">int ret; ... @@ -51,17 +57,28 @@ if ((ret = dbp->put(dbp, NULL, &key, &data, 0)) != 0) { fprintf(stderr, "put failed: %s\n", db_strerror(ret)); return (1); }</pre> - <p>There are also two additional error methods, <a href="../api_reference/C/dberr.html" class="olink">DB->err()</a> and -<code class="methodname">DB->errx()</code>. These methods work like the ANSI C X3.159-1989 (ANSI C) printf -function, taking a printf-style format string and argument list, and -writing a message constructed from the format string and arguments.</p> - <p>The <a href="../api_reference/C/dberr.html" class="olink">DB->err()</a> method appends the standard error string to the -constructed message; the <code class="methodname">DB->errx()</code> method does not. These methods -provide simpler ways of displaying Berkeley DB error messages. For example, -if your application tracks session IDs in a variable called session_id, -it can include that information in its error messages:</p> - <p>Error messages can additionally be configured to always include a prefix -(for example, the program name) using the <a href="../api_reference/C/dbset_errpfx.html" class="olink">DB->set_errpfx()</a> method.</p> + <p> + There are also two additional error methods, <a href="../api_reference/C/dberr.html" class="olink">DB->err()</a> and + <code class="methodname">DB->errx()</code>. These methods work + like the ANSI C X3.159-1989 (ANSI C) printf function, taking a + printf-style format string and argument list, and writing a + message constructed from the format string and + arguments. + </p> + <p> + The <a href="../api_reference/C/dberr.html" class="olink">DB->err()</a> method appends the standard error string to the + constructed message; the + <code class="methodname">DB->errx()</code> method does not. + These methods provide simpler ways of displaying Berkeley DB + error messages. For example, if your application tracks + session IDs in a variable called session_id, it can include + that information in its error messages: + </p> + <p> + Error messages can additionally be configured to always + include a prefix (for example, the program name) using the + <a href="../api_reference/C/dbset_errpfx.html" class="olink">DB->set_errpfx()</a> method. + </p> <a id="prog_am26"></a> <pre class="programlisting">#define DATABASE "access.db" @@ -77,8 +94,11 @@ if ((ret = dbp->open(dbp, session_id); return (1); }</pre> - <p>For example, if the program were called my_app and the open call returned -an EACCESS system error, the error messages shown would appear as follows:</p> + <p> + For example, if the program were called my_app and the open + call returned an EACCESS system error, the error messages + shown would appear as follows: + </p> <pre class="programlisting">my_app: access.db: Permission denied. my_app: contact your system administrator: session ID was 14</pre> </div> diff --git a/docs/programmer_reference/am_misc_faq.html b/docs/programmer_reference/am_misc_faq.html index b93e873d..3ddffcca 100644 --- a/docs/programmer_reference/am_misc_faq.html +++ b/docs/programmer_reference/am_misc_faq.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_misc_tune.html">Prev</a> </td> - <th width="60%" align="center">Chapter 4. - Access Method Wrapup - </th> + <th width="60%" align="center">Chapter 4. Access Method Wrapup </th> <td width="20%" align="right"> <a accesskey="n" href="java.html">Next</a></td> </tr> </table> @@ -42,133 +40,195 @@ <ol type="1"> <li> <span class="bold"> - <strong>Is a Berkeley DB database the same as a "table"?</strong> + <strong>Is a Berkeley DB database the same + as a "table"?</strong> </span> - <p>Yes; "tables" are databases, "rows" are key/data pairs, and "columns" -are application-encapsulated fields within a data item (to which Berkeley DB -does not directly provide access).</p> + <p> + Yes; "tables" are databases, "rows" are key/data + pairs, and "columns" are application-encapsulated + fields within a data item (to which Berkeley DB does + not directly provide access). + </p> </li> <li> <span class="bold"> - <strong>I'm getting an error return in my application, but I can't -figure out what the library is complaining about.</strong> + <strong>I'm getting an error return in my + application, but I can't figure out what the library + is complaining about.</strong> </span> - <p>See <a href="../api_reference/C/envset_errcall.html" class="olink">DB_ENV->set_errcall()</a>, <a href="../api_reference/C/envset_errfile.html" class="olink">DB_ENV->set_errfile()</a> and -<a href="../api_reference/C/dbset_errfile.html" class="olink">DB->set_errfile()</a> for ways to get additional information about -error returns from Berkeley DB.</p> + <p> + See <a href="../api_reference/C/envset_errcall.html" class="olink">DB_ENV->set_errcall()</a>, <a href="../api_reference/C/envset_errfile.html" class="olink">DB_ENV->set_errfile()</a> and + <a href="../api_reference/C/dbset_errfile.html" class="olink">DB->set_errfile()</a> for ways to get additional information + about error returns from Berkeley DB. + </p> </li> <li> <span class="bold"> - <strong>Are Berkeley DB databases portable between architectures with -different integer sizes and different byte orders ?</strong> + <strong>Are Berkeley DB databases portable + between architectures with different integer sizes and + different byte orders ?</strong> </span> - <p>Yes. Specifically, databases can be moved between 32- and 64-bit -machines, as well as between little- and big-endian machines. See -<a class="xref" href="general_am_conf.html#am_conf_byteorder" title="Selecting a byte order">Selecting a byte order</a> for -more information.</p> + <p> + Yes. Specifically, databases can be moved between + 32- and 64-bit machines, as well as between little- + and big-endian machines. See <a class="xref" href="general_am_conf.html#am_conf_byteorder" title="Selecting a byte order">Selecting a byte order</a> for more + information. + </p> </li> <li> <span class="bold"> - <strong>I'm seeing database corruption when creating multiple databases -in a single physical file.</strong> + <strong>I'm seeing database corruption when + creating multiple databases in a single physical + file.</strong> </span> - <p>This problem is usually the result of <a href="../api_reference/C/db.html" class="olink">DB</a> handles not sharing an -underlying database environment. See <a class="xref" href="am_opensub.html" title="Opening multiple databases in a single file">Opening multiple databases in a single file</a> for more information.</p> + <p> + This problem is usually the result of <a href="../api_reference/C/db.html" class="olink">DB</a> handles + not sharing an underlying database environment. See + <a class="xref" href="am_opensub.html" title="Opening multiple databases in a single file">Opening multiple databases in a + single file</a> for more + information. + </p> </li> <li> <span class="bold"> - <strong>I'm using integers as keys for a Btree database, and even -though the key/data pairs are entered in sorted order, the page-fill -factor is low.</strong> + <strong>I'm using integers as keys for a + Btree database, and even though the key/data pairs are + entered in sorted order, the page-fill factor is + low.</strong> </span> - <p>This is usually the result of using integer keys on little-endian -architectures such as the x86. Berkeley DB sorts keys as byte strings, and -little-endian integers don't sort well when viewed as byte strings. -For example, take the numbers 254 through 257. Their byte patterns on -a little-endian system are:</p> + <p> + This is usually the result of using integer keys on + little-endian architectures such as the x86. Berkeley + DB sorts keys as byte strings, and little-endian + integers don't sort well when viewed as byte strings. + For example, take the numbers 254 through 257. Their + byte patterns on a little-endian system are: + </p> <pre class="programlisting">254 fe 0 0 0 255 ff 0 0 0 256 0 1 0 0 257 1 1 0 0</pre> - <p>If you treat them as strings, then they sort badly:</p> + <p> + If you treat them as strings, then they sort + badly: + </p> <pre class="programlisting">256 257 254 255</pre> - <p>On a big-endian system, their byte patterns are:</p> + <p> + On a big-endian system, their byte patterns + are: + </p> <pre class="programlisting">254 0 0 0 fe 255 0 0 0 ff 256 0 0 1 0 257 0 0 1 1</pre> - <p>and so, if you treat them as strings they sort nicely. Which means, if -you use steadily increasing integers as keys on a big-endian system -Berkeley DB behaves well and you get compact trees, but on a little-endian -system Berkeley DB produces much less compact trees. To avoid this problem, -you may want to convert the keys to flat text or big-endian -representations, or provide your own -<a class="xref" href="bt_conf.html#am_conf_bt_compare" title="Btree comparison">Btree comparison</a></p> + <p> + and so, if you treat them as strings they sort + nicely. Which means, if you use steadily increasing + integers as keys on a big-endian system Berkeley DB + behaves well and you get compact trees, but on a + little-endian system Berkeley DB produces much less + compact trees. To avoid this problem, you may want to + convert the keys to flat text or big-endian + representations, or provide your own <a class="xref" href="bt_conf.html#am_conf_bt_compare" title="Btree comparison">Btree comparison</a> + </p> </li> <li> <span class="bold"> - <strong>Is there any way to avoid double buffering in the Berkeley DB system?</strong> + <strong>Is there any way to avoid double + buffering in the Berkeley DB system?</strong> </span> - <p>While you cannot avoid double buffering entirely, there are a few things -you can do to address this issue:</p> - <p>First, the Berkeley DB cache size can be explicitly set. Rather than allocate -additional space in the Berkeley DB cache to cover unexpectedly heavy load or -large table sizes, double buffering may suggest you size the cache to -function well under normal conditions, and then depend on the file -buffer cache to cover abnormal conditions. Obviously, this is a -trade-off, as Berkeley DB may not then perform as well as usual under abnormal -conditions.</p> - <p>Second, depending on the underlying operating system you're using, you -may be able to alter the amount of physical memory devoted to the -system's file buffer cache. Altering this type of resource -configuration may require appropriate privileges, or even operating -system reboots and/or rebuilds, on some systems.</p> - <p>Third, changing the size of the Berkeley DB environment regions can change -the amount of space the operating system makes available for the file -buffer cache, and it's often worth considering exactly how the operating -system is dividing up its available memory. Further, moving the Berkeley DB -database environment regions from filesystem backed memory into system -memory (or heap memory), can often make additional system memory -available for the file buffer cache, especially on systems without a -unified buffer cache and VM system.</p> - <p>Finally, for operating systems that allow buffering to be turned off, -specifying the <a href="../api_reference/C/envset_flags.html#set_flags_DB_DIRECT_DB" class="olink">DB_DIRECT_DB</a> and <a href="../api_reference/C/envlog_set_config.html#log_set_config_DB_LOG_DIRECT" class="olink">DB_LOG_DIRECT</a> flags -will attempt to do so.</p> + <p> + While you cannot avoid double buffering entirely, + there are a few things you can do to address this + issue: + </p> + <p> + First, the Berkeley DB cache size can be explicitly + set. Rather than allocate additional space in the + Berkeley DB cache to cover unexpectedly heavy load or + large table sizes, double buffering may suggest you + size the cache to function well under normal + conditions, and then depend on the file buffer cache + to cover abnormal conditions. Obviously, this is a + trade-off, as Berkeley DB may not then perform as well + as usual under abnormal conditions. + </p> + <p> + Second, depending on the underlying operating system + you're using, you may be able to alter the amount of + physical memory devoted to the system's file buffer + cache. Altering this type of resource configuration + may require appropriate privileges, or even operating + system reboots and/or rebuilds, on some + systems. + </p> + <p> + Third, changing the size of the Berkeley DB + environment regions can change the amount of space the + operating system makes available for the file buffer + cache, and it's often worth considering exactly how + the operating system is dividing up its available + memory. Further, moving the Berkeley DB database + environment regions from filesystem backed memory into + system memory (or heap memory), can often make + additional system memory available for the file buffer + cache, especially on systems without a unified buffer + cache and VM system. + </p> + <p> + Finally, for operating systems that allow buffering + to be turned off, specifying the <a href="../api_reference/C/envset_flags.html#set_flags_DB_DIRECT_DB" class="olink">DB_DIRECT_DB</a> and + <a href="../api_reference/C/envlog_set_config.html#log_set_config_DB_LOG_DIRECT" class="olink">DB_LOG_DIRECT</a> flags will attempt to do so. + </p> </li> <li> <span class="bold"> - <strong>I'm seeing database corruption when I run out of disk space.</strong> + <strong>I'm seeing database corruption when + I run out of disk space.</strong> </span> - <p>Berkeley DB can continue to run when when out-of-disk-space errors occur, but -it requires the application to be transaction protected. Applications -which do not enclose update operations in transactions cannot recover -from out-of-disk-space errors, and the result of running out of disk -space may be database corruption.</p> + <p> + Berkeley DB can continue to run when when + out-of-disk-space errors occur, but it requires the + application to be transaction protected. Applications + which do not enclose update operations in transactions + cannot recover from out-of-disk-space errors, and the + result of running out of disk space may be database + corruption. + </p> </li> <li> <span class="bold"> - <strong>How can I associate application information with a <a href="../api_reference/C/db.html" class="olink">DB</a> -or <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handle?</strong> + <strong>How can I associate application + information with a <a href="../api_reference/C/db.html" class="olink">DB</a> or <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handle?</strong> </span> - <p>In the C API, the <a href="../api_reference/C/db.html" class="olink">DB</a> and <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> structures each contain -an "app_private" field intended to be used to reference -application-specific information. See the <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a> and -<a href="../api_reference/C/envcreate.html" class="olink">db_env_create()</a> documentation for more information.</p> - <p>In the C++ or Java APIs, the easiest way to associate -application-specific data with a handle is to subclass the <a href="../api_reference/CXX/db.html" class="olink">Db</a> -or <a href="../api_reference/CXX/env.html" class="olink">DbEnv</a>, for example subclassing <a href="../api_reference/CXX/db.html" class="olink">Db</a> to get MyDb. -Objects of type MyDb will still have the Berkeley DB API methods available on -them, and you can put any extra data or methods you want into the MyDb -class. If you are using "callback" APIs that take <a href="../api_reference/CXX/db.html" class="olink">Db</a> or -<a href="../api_reference/CXX/env.html" class="olink">DbEnv</a> arguments (for example, <a href="../api_reference/C/dbset_bt_compare.html" class="olink">DB->set_bt_compare()</a>) -these will always be called with the <a href="../api_reference/CXX/db.html" class="olink">Db</a> or <a href="../api_reference/CXX/env.html" class="olink">DbEnv</a> -objects you create. So if you always use MyDb objects, you will be able -to take the first argument to the callback function and cast it to a -MyDb (in C++, cast it to (MyDb*)). That will allow you to access your -data members or methods.</p> + <p> + In the C API, the <a href="../api_reference/C/db.html" class="olink">DB</a> and <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> structures each + contain an "app_private" field intended to be used to + reference application-specific information. See the + <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a> and <a href="../api_reference/C/envcreate.html" class="olink">db_env_create()</a> documentation for more + information. + </p> + <p> + In the C++ or Java APIs, the easiest way to + associate application-specific data with a handle is + to subclass the <a href="../api_reference/CXX/db.html" class="olink">Db</a> or <a href="../api_reference/CXX/env.html" class="olink">DbEnv</a>, for + example subclassing <a href="../api_reference/CXX/db.html" class="olink">Db</a> to get MyDb. + Objects of type MyDb will still have the Berkeley DB + API methods available on them, and you can put any + extra data or methods you want into the MyDb class. If + you are using "callback" APIs that take <a href="../api_reference/CXX/db.html" class="olink">Db</a> + or <a href="../api_reference/CXX/env.html" class="olink">DbEnv</a> arguments (for example, + <a href="../api_reference/C/dbset_bt_compare.html" class="olink">DB->set_bt_compare()</a>) these will always be called with + the <a href="../api_reference/CXX/db.html" class="olink">Db</a> or <a href="../api_reference/CXX/env.html" class="olink">DbEnv</a> objects you + create. So if you always use MyDb objects, you will be + able to take the first argument to the callback + function and cast it to a MyDb (in C++, cast it to + (MyDb*)). That will allow you to access your data + members or methods. + </p> </li> </ol> </div> @@ -188,9 +248,7 @@ data members or methods.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 5. - Java API - </td> + <td width="40%" align="right" valign="top"> Chapter 5. Java API </td> </tr> </table> </div> diff --git a/docs/programmer_reference/am_misc_partial.html b/docs/programmer_reference/am_misc_partial.html index 1169f460..ab128bf8 100644 --- a/docs/programmer_reference/am_misc_partial.html +++ b/docs/programmer_reference/am_misc_partial.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Partial record storage and retrieval</th> + <th colspan="3" align="center">Partial record storage and + retrieval</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_misc_bulk.html">Prev</a> </td> - <th width="60%" align="center">Chapter 4. - Access Method Wrapup - </th> + <th width="60%" align="center">Chapter 4. Access Method Wrapup </th> <td width="20%" align="right"> <a accesskey="n" href="am_misc_struct.html">Next</a></td> </tr> </table> @@ -34,44 +33,57 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="am_misc_partial"></a>Partial record storage and retrieval</h2> + <h2 class="title" style="clear: both"><a id="am_misc_partial"></a>Partial record storage and + retrieval</h2> </div> </div> </div> - <p>It is possible to both store and retrieve parts of data items in all -Berkeley DB access methods. This is done by setting the -<a href="../api_reference/C/dbt.html#dbt_DB_DBT_PARTIAL" class="olink">DB_DBT_PARTIAL</a> flag <a href="../api_reference/C/dbt.html" class="olink">DBT</a> structure passed to the -Berkeley DB method.</p> - <p>The <a href="../api_reference/C/dbt.html#dbt_DB_DBT_PARTIAL" class="olink">DB_DBT_PARTIAL</a> flag is based on the values of two fields -of the <a href="../api_reference/C/dbt.html" class="olink">DBT</a> structure: <span class="bold"><strong>dlen</strong></span> and <span class="bold"><strong>doff</strong></span>. The value -of <span class="bold"><strong>dlen</strong></span> is the number of bytes of the record in which the -application is interested. The value of <span class="bold"><strong>doff</strong></span> is the offset from -the beginning of the data item where those bytes start.</p> - <p>For example, if the data item were <span class="bold"><strong>ABCDEFGHIJKL</strong></span>, a <span class="bold"><strong>doff</strong></span> -value of 3 would indicate that the bytes of interest started at -<span class="bold"><strong>D</strong></span>, and a <span class="bold"><strong>dlen</strong></span> value of 4 would indicate that the bytes -of interest were <span class="bold"><strong>DEFG</strong></span>.</p> - <p>When retrieving a data item from a database, the <span class="bold"><strong>dlen</strong></span> bytes -starting <span class="bold"><strong>doff</strong></span> bytes from the beginning of the record are -returned, as if they comprised the entire record. If any or all of the -specified bytes do not exist in the record, the retrieval is still -successful and any existing bytes are returned.</p> - <p>When storing a data item into the database, the <span class="bold"><strong>dlen</strong></span> bytes -starting <span class="bold"><strong>doff</strong></span> bytes from the beginning of the specified key's -data record are replaced by the data specified by the <span class="bold"><strong>data</strong></span> and -<span class="bold"><strong>size</strong></span> fields. If <span class="bold"><strong>dlen</strong></span> is smaller than <span class="bold"><strong>size</strong></span>, the -record will grow, and if <span class="bold"><strong>dlen</strong></span> is larger than <span class="bold"><strong>size</strong></span>, the -record will shrink. If the specified bytes do not exist, the record will -be extended using nul bytes as necessary, and the store call will still -succeed.</p> - <p>The following are various examples of the put case for the -<a href="../api_reference/C/dbt.html#dbt_DB_DBT_PARTIAL" class="olink">DB_DBT_PARTIAL</a> flag. In all examples, the initial data item is 20 -bytes in length:</p> <p> - <span class="bold"> - <strong>ABCDEFGHIJ0123456789</strong> - </span> - </p> + It is possible to both store and retrieve parts of data + items in all Berkeley DB access methods. This is done by + setting the <a href="../api_reference/C/dbt.html#dbt_DB_DBT_PARTIAL" class="olink">DB_DBT_PARTIAL</a> flag <a href="../api_reference/C/dbt.html" class="olink">DBT</a> structure passed to + the Berkeley DB method. + </p> + <p> + The <a href="../api_reference/C/dbt.html#dbt_DB_DBT_PARTIAL" class="olink">DB_DBT_PARTIAL</a> flag is based on the values of two + fields of the <a href="../api_reference/C/dbt.html" class="olink">DBT</a> structure: <span class="bold"><strong>dlen</strong></span> + and <span class="bold"><strong>doff</strong></span>. The value of <span class="bold"><strong> + dlen</strong></span> is the number of bytes of the record in + which the application is interested. The value of <span class="bold"><strong>doff</strong></span> is the offset from the + beginning of the data item where those bytes start. + </p> + <p> + For example, if the data item were <span class="bold"><strong> + ABCDEFGHIJKL</strong></span>, a <span class="bold"><strong>doff</strong></span> + value of 3 would indicate that the bytes + of interest started at <span class="bold"><strong>D</strong></span>, and + a <span class="bold"><strong>dlen</strong></span> value of 4 would + indicate that the bytes of interest were <span class="bold"><strong> + DEFG</strong></span>. + </p> + <p> + When retrieving a data item from a database, the <span class="bold"><strong>dlen</strong></span> bytes starting <span class="bold"><strong>doff</strong></span> bytes from the beginning of + the record are returned, as if they comprised the entire + record. If any or all of the specified bytes do not exist in + the record, the retrieval is still successful and any existing + bytes are returned. + </p> + <p> + When storing a data item into the database, the <span class="bold"><strong>dlen</strong></span> bytes starting <span class="bold"><strong>doff</strong></span> bytes from the beginning of + the specified key's data record are replaced by the data + specified by the <span class="bold"><strong>data</strong></span> and + <span class="bold"><strong>size</strong></span> fields. If <span class="bold"><strong>dlen</strong></span> is smaller than <span class="bold"><strong>size</strong></span>, the record will grow, and if + <span class="bold"><strong>dlen</strong></span> is larger than + <span class="bold"><strong>size</strong></span>, the record will + shrink. If the specified bytes do not exist, the record will + be extended using nul bytes as necessary, and the store call + will still succeed. + </p> + <p> + The following are various examples of the put case for the + <a href="../api_reference/C/dbt.html#dbt_DB_DBT_PARTIAL" class="olink">DB_DBT_PARTIAL</a> flag. In all examples, the initial data item + is 20 bytes in length: <span class="bold"><strong>ABCDEFGHIJ0123456789</strong></span> + </p> <div class="orderedlist"> <ol type="1"> <li> @@ -182,7 +194,8 @@ ABCDEFGHIJ0123456789 -> ABCDEFGHIJ0123456789\0\0\0\0\0abcdefghij </pre> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Storing C/C++ structures/objects</td> + <td width="40%" align="right" valign="top"> Storing C/C++ + structures/objects</td> </tr> </table> </div> diff --git a/docs/programmer_reference/am_misc_perm.html b/docs/programmer_reference/am_misc_perm.html index de9c1164..f8631afc 100644 --- a/docs/programmer_reference/am_misc_perm.html +++ b/docs/programmer_reference/am_misc_perm.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_misc_struct.html">Prev</a> </td> - <th width="60%" align="center">Chapter 4. - Access Method Wrapup - </th> + <th width="60%" align="center">Chapter 4. Access Method Wrapup </th> <td width="20%" align="right"> <a accesskey="n" href="am_misc_error.html">Next</a></td> </tr> </table> @@ -38,20 +36,26 @@ </div> </div> </div> - <p>When using the non-cursor Berkeley DB calls to retrieve key/data items under -the C/C++ APIs (for example, <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a>), the memory to which the -pointer stored into the <a href="../api_reference/C/dbt.html" class="olink">DBT</a> refers is only valid until the next -call to Berkeley DB using the <a href="../api_reference/C/db.html" class="olink">DB</a> handle. (This includes <span class="bold"><strong>any</strong></span> -use of the returned <a href="../api_reference/C/db.html" class="olink">DB</a> handle, including by another thread of -control within the process. For this reason, when multiple threads are -using the returned <a href="../api_reference/C/db.html" class="olink">DB</a> handle concurrently, one of the -<a href="../api_reference/C/dbt.html#dbt_DB_DBT_MALLOC" class="olink">DB_DBT_MALLOC</a>, <a href="../api_reference/C/dbt.html#dbt_DB_DBT_REALLOC" class="olink">DB_DBT_REALLOC</a> or <a href="../api_reference/C/dbt.html#dbt_DB_DBT_USERMEM" class="olink">DB_DBT_USERMEM</a> -flags must be specified with any non-cursor <a href="../api_reference/C/dbt.html" class="olink">DBT</a> used for key or -data retrieval.)</p> - <p>When using the cursor Berkeley DB calls to retrieve key/data items under the -C/C++ APIs (for example, <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a>), the memory to which the -pointer stored into the <a href="../api_reference/C/dbt.html" class="olink">DBT</a> refers is only valid until the next -call to Berkeley DB using the <a href="../api_reference/C/dbc.html" class="olink">DBC</a> returned by <a href="../api_reference/C/dbcursor.html" class="olink">DB->cursor()</a>.</p> + <p> + When using the non-cursor Berkeley DB calls to retrieve + key/data items under the C/C++ APIs (for example, <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a>), + the memory to which the pointer stored into the <a href="../api_reference/C/dbt.html" class="olink">DBT</a> refers + is only valid until the next call to Berkeley DB using the + <a href="../api_reference/C/db.html" class="olink">DB</a> handle. (This includes <span class="bold"><strong>any</strong></span> use of the returned <a href="../api_reference/C/db.html" class="olink">DB</a> handle, including + by another thread of control within the process. For this + reason, when multiple threads are using the returned <a href="../api_reference/C/db.html" class="olink">DB</a> + handle concurrently, one of the <a href="../api_reference/C/dbt.html#dbt_DB_DBT_MALLOC" class="olink">DB_DBT_MALLOC</a>, + <a href="../api_reference/C/dbt.html#dbt_DB_DBT_REALLOC" class="olink">DB_DBT_REALLOC</a> or <a href="../api_reference/C/dbt.html#dbt_DB_DBT_USERMEM" class="olink">DB_DBT_USERMEM</a> flags must be specified + with any non-cursor <a href="../api_reference/C/dbt.html" class="olink">DBT</a> used for key or data + retrieval.) + </p> + <p> + When using the cursor Berkeley DB calls to retrieve key/data + items under the C/C++ APIs (for example, <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a>), the memory + to which the pointer stored into the <a href="../api_reference/C/dbt.html" class="olink">DBT</a> refers is only + valid until the next call to Berkeley DB using the <a href="../api_reference/C/dbc.html" class="olink">DBC</a> + returned by <a href="../api_reference/C/dbcursor.html" class="olink">DB->cursor()</a>. + </p> </div> <div class="navfooter"> <hr /> @@ -64,7 +68,8 @@ call to Berkeley DB using the <a href="../api_reference/C/dbc.html" class="olink <td width="40%" align="right"> <a accesskey="n" href="am_misc_error.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Storing C/C++ structures/objects </td> + <td width="40%" align="left" valign="top">Storing C/C++ + structures/objects </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/am_misc_stability.html b/docs/programmer_reference/am_misc_stability.html index ea91bfeb..4ed6e148 100644 --- a/docs/programmer_reference/am_misc_stability.html +++ b/docs/programmer_reference/am_misc_stability.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_misc_error.html">Prev</a> </td> - <th width="60%" align="center">Chapter 4. - Access Method Wrapup - </th> + <th width="60%" align="center">Chapter 4. Access Method Wrapup </th> <td width="20%" align="right"> <a accesskey="n" href="am_misc_dbsizes.html">Next</a></td> </tr> </table> @@ -38,38 +36,53 @@ </div> </div> </div> - <p>In the absence of locking, no guarantees are made about the stability -of cursors in different threads of control. However, the Btree, Queue -and Recno access methods guarantee that cursor operations, interspersed -with any other operation in the same thread of control will always -return keys in order and will return each non-deleted key/data pair -exactly once. Because the Hash access method uses a dynamic hashing -algorithm, it cannot guarantee any form of stability in the presence of -inserts and deletes unless transactional locking is performed.</p> - <p>If locking was specified when the Berkeley DB environment was opened, but -transactions are not in effect, the access methods provide repeatable -reads with respect to the cursor. That is, a <a href="../api_reference/C/dbcget.html#dbcget_DB_CURRENT" class="olink">DB_CURRENT</a> call -on the cursor is guaranteed to return the same record as was returned -on the last call to the cursor.</p> - <p>In the presence of transactions, the Btree, Hash and Recno access -methods provide degree 3 isolation (serializable transactions). The -Queue access method provides degree 3 isolation with the exception that -it permits phantom records to appear between calls. That is, deleted -records are not locked, therefore another transaction may replace a -deleted record between two calls to retrieve it. The record would not -appear in the first call but would be seen by the second call. For -readers not enclosed in transactions, all access method calls provide -degree 2 isolation, that is, reads are not repeatable. A transaction -may be declared to run with degree 2 isolation by specifying the -<a href="../api_reference/C/dbcget.html#dbcget_DB_READ_COMMITTED" class="olink">DB_READ_COMMITTED</a> flag. Finally, Berkeley DB provides degree 1 isolation -when the <a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a> flag is specified; that is, reads -may see data modified in transactions which have not yet committed.</p> - <p>For all access methods, a cursor scan of the database performed within -the context of a transaction is guaranteed to return each key/data pair -once and only once, except in the following case. If, while performing -a cursor scan using the Hash access method, the transaction performing -the scan inserts a new pair into the database, it is possible that -duplicate key/data pairs will be returned.</p> + <p> + In the absence of locking, no guarantees are made about the + stability of cursors in different threads of control. However, + the Btree, Queue and Recno access methods guarantee that + cursor operations, interspersed with any other operation in + the same thread of control will always return keys in order + and will return each non-deleted key/data pair exactly once. + Because the Hash access method uses a dynamic hashing + algorithm, it cannot guarantee any form of stability in the + presence of inserts and deletes unless transactional locking + is performed. + </p> + <p> + If locking was specified when the Berkeley DB environment + was opened, but transactions are not in effect, the access + methods provide repeatable reads with respect to the cursor. + That is, a <a href="../api_reference/C/dbcget.html#dbcget_DB_CURRENT" class="olink">DB_CURRENT</a> call on the cursor is guaranteed to + return the same record as was returned on the last call to the + cursor. + </p> + <p> + In the presence of transactions, the Btree, Hash and Recno + access methods provide degree 3 isolation (serializable + transactions). The Queue access method provides degree 3 + isolation with the exception that it permits phantom records + to appear between calls. That is, deleted records are not + locked, therefore another transaction may replace a deleted + record between two calls to retrieve it. The record would not + appear in the first call but would be seen by the second call. + For readers not enclosed in transactions, all access method + calls provide degree 2 isolation, that is, reads are not + repeatable. A transaction may be declared to run with degree 2 + isolation by specifying the <a href="../api_reference/C/dbcget.html#dbcget_DB_READ_COMMITTED" class="olink">DB_READ_COMMITTED</a> flag. Finally, + Berkeley DB provides degree 1 isolation when the + <a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a> flag is specified; that is, reads may + see data modified in transactions which have not yet + committed. + </p> + <p> + For all access methods, a cursor scan of the database + performed within the context of a transaction is guaranteed to + return each key/data pair once and only once, except in the + following case. If, while performing a cursor scan using the + Hash access method, the transaction performing the scan + inserts a new pair into the database, it is possible that + duplicate key/data pairs will be returned. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/am_misc_struct.html b/docs/programmer_reference/am_misc_struct.html index 25128a75..77d135c5 100644 --- a/docs/programmer_reference/am_misc_struct.html +++ b/docs/programmer_reference/am_misc_struct.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Storing C/C++ structures/objects</th> + <th colspan="3" align="center">Storing C/C++ + structures/objects</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_misc_partial.html">Prev</a> </td> - <th width="60%" align="center">Chapter 4. - Access Method Wrapup - </th> + <th width="60%" align="center">Chapter 4. Access Method Wrapup </th> <td width="20%" align="right"> <a accesskey="n" href="am_misc_perm.html">Next</a></td> </tr> </table> @@ -34,16 +33,23 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="am_misc_struct"></a>Storing C/C++ structures/objects</h2> + <h2 class="title" style="clear: both"><a id="am_misc_struct"></a>Storing C/C++ + structures/objects</h2> </div> </div> </div> - <p>Berkeley DB can store any kind of data, that is, it is entirely 8-bit clean. -How you use this depends, to some extent, on the application language -you are using. In the C/C++ languages, there are a couple of different -ways to store structures and objects.</p> - <p>First, you can do some form of run-length encoding and copy your -structure into another piece of memory before storing it:</p> + <p> + Berkeley DB can store any kind of data, that is, it is + entirely 8-bit clean. How you use this depends, to some + extent, on the application language you are using. In the + C/C++ languages, there are a couple of different ways to store + structures and objects. + </p> + <p> + First, you can do some form of run-length encoding and copy + your structure into another piece of memory before storing + it: + </p> <a id="prog_am23"></a> <pre class="programlisting">struct { char *data1; @@ -63,11 +69,15 @@ memcpy(p, &info.data2, sizeof(info.data2)); p += sizeof(info.data2); ... </pre> - <p>and so on, until all the fields of the structure have been loaded into -the byte array. If you want more examples, see the Berkeley DB logging -routines (for example, btree/btree_auto.c:__bam_split_log()). This -technique is generally known as "marshalling". If you use this -technique, you must then un-marshall the data when you read it back:</p> + <p> + and so on, until all the fields of the structure have been + loaded into the byte array. If you want more examples, see the + Berkeley DB logging routines (for example, + btree/btree_auto.c:__bam_split_log()). This technique is + generally known as "marshalling". If you use this technique, + you must then un-marshall the data when you read it + back: + </p> <a id="prog_am24"></a> <pre class="programlisting">struct { char *data1; @@ -87,27 +97,41 @@ memcpy(&info.data2, p, sizeof(info.data2)); p += sizeof(info.data2); ... </pre> - <p>and so on.</p> - <p>The second way to solve this problem only works if you have just one -variable length field in the structure. In that case, you can declare -the structure as follows:</p> + <p> + and so on. + </p> + <p> + The second way to solve this problem only works if you have + just one variable length field in the structure. In that case, + you can declare the structure as follows: + </p> <pre class="programlisting">struct { int a, b, c; u_int8_t buf[1]; } info;</pre> - <p>Then, let's say you have a string you want to store in this structure. -When you allocate the structure, you allocate it as:</p> + <p> + Then, let's say you have a string you want to store in this + structure. When you allocate the structure, you allocate it + as: + </p> <pre class="programlisting">malloc(sizeof(struct info) + strlen(string));</pre> - <p>Since the allocated memory is contiguous, you can the initialize the -structure as:</p> + <p> + Since the allocated memory is contiguous, you can the + initialize the structure as: + </p> <pre class="programlisting">info.a = 1; info.b = 2; info.c = 3; memcpy(&info.buf[0], string, strlen(string) + 1);</pre> - <p>and give it to Berkeley DB to store, with a length of:</p> + <p> + and give it to Berkeley DB to store, with a length + of: + </p> <pre class="programlisting">sizeof(struct info) + strlen(string);</pre> - <p>In this case, the structure can be copied out of the database and used -without any additional work.</p> + <p> + In this case, the structure can be copied out of the + database and used without any additional work. + </p> </div> <div class="navfooter"> <hr /> @@ -120,7 +144,8 @@ without any additional work.</p> <td width="40%" align="right"> <a accesskey="n" href="am_misc_perm.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Partial record storage and retrieval </td> + <td width="40%" align="left" valign="top">Partial record storage and + retrieval </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/am_misc_tune.html b/docs/programmer_reference/am_misc_tune.html index 8e22aa03..b5f9ad51 100644 --- a/docs/programmer_reference/am_misc_tune.html +++ b/docs/programmer_reference/am_misc_tune.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_misc_db_sql.html">Prev</a> </td> - <th width="60%" align="center">Chapter 4. - Access Method Wrapup - </th> + <th width="60%" align="center">Chapter 4. Access Method Wrapup </th> <td width="20%" align="right"> <a accesskey="n" href="am_misc_faq.html">Next</a></td> </tr> </table> @@ -38,119 +36,174 @@ </div> </div> </div> - <p>There are a few different issues to consider when tuning the performance -of Berkeley DB access method applications.</p> + <p> + There are a few different issues to consider when tuning the + performance of Berkeley DB access method applications. + </p> <div class="variablelist"> <dl> <dt> <span class="term">access method</span> </dt> - <dd>An application's choice of a database access method can significantly -affect performance. Applications using fixed-length records and integer -keys are likely to get better performance from the Queue access method. -Applications using variable-length records are likely to get better -performance from the Btree access method, as it tends to be faster for -most applications than either the Hash or Recno access methods. Because -the access method APIs are largely identical between the Berkeley DB access -methods, it is easy for applications to benchmark the different access -methods against each other. See <a class="xref" href="am_conf_select.html" title="Selecting an access method">Selecting an access method</a> for more information.</dd> + <dd> + An application's choice of a database access + method can significantly affect performance. + Applications using fixed-length records and integer + keys are likely to get better performance from the + Queue access method. Applications using + variable-length records are likely to get better + performance from the Btree access method, as it tends + to be faster for most applications than either the + Hash or Recno access methods. Because the access + method APIs are largely identical between the Berkeley + DB access methods, it is easy for applications to + benchmark the different access methods against each + other. See <a class="xref" href="am_conf_select.html" title="Selecting an access method">Selecting an access method</a> for more + information. + </dd> <dt> <span class="term">cache size</span> </dt> - <dd>The Berkeley DB database cache defaults to a fairly small size, and most -applications concerned with performance will want to set it explicitly. -Using a too-small cache will result in horrible performance. The first -step in tuning the cache size is to use the db_stat utility (or the -statistics returned by the <a href="../api_reference/C/dbstat.html" class="olink">DB->stat()</a> function) to measure the -effectiveness of the cache. The goal is to maximize the cache's hit -rate. Typically, increasing the size of the cache until the hit rate -reaches 100% or levels off will yield the best performance. However, -if your working set is sufficiently large, you will be limited by the -system's available physical memory. Depending on the virtual memory -and file system buffering policies of your system, and the requirements -of other applications, the maximum cache size will be some amount -smaller than the size of physical memory. If you find that -the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility shows that increasing the cache size improves your hit -rate, but performance is not improving (or is getting worse), then it's -likely you've hit other system limitations. At this point, you should -review the system's swapping/paging activity and limit the size of the -cache to the maximum size possible without triggering paging activity. -Finally, always remember to make your measurements under conditions as -close as possible to the conditions your deployed application will run -under, and to test your final choices under worst-case conditions.</dd> + <dd> + The Berkeley DB database cache defaults to a + fairly small size, and most applications concerned + with performance will want to set it explicitly. Using + a too-small cache will result in horrible performance. + The first step in tuning the cache size is to use the + db_stat utility (or the statistics returned by the + <a href="../api_reference/C/dbstat.html" class="olink">DB->stat()</a> function) to measure the effectiveness of the + cache. The goal is to maximize the cache's hit rate. + Typically, increasing the size of the cache until the + hit rate reaches 100% or levels off will yield the + best performance. However, if your working set is + sufficiently large, you will be limited by the + system's available physical memory. Depending on the + virtual memory and file system buffering policies of + your system, and the requirements of other + applications, the maximum cache size will be some + amount smaller than the size of physical memory. If + you find that the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility shows that increasing the + cache size improves your hit rate, but performance is + not improving (or is getting worse), then it's likely + you've hit other system limitations. At this point, + you should review the system's swapping/paging + activity and limit the size of the cache to the + maximum size possible without triggering paging + activity. Finally, always remember to make your + measurements under conditions as close as possible to + the conditions your deployed application will run + under, and to test your final choices under worst-case + conditions. + </dd> <dt> <span class="term">shared memory</span> </dt> - <dd>By default, Berkeley DB creates its database environment shared regions in -filesystem backed memory. Some systems do not distinguish between -regular filesystem pages and memory-mapped pages backed by the -filesystem, when selecting dirty pages to be flushed back to disk. For -this reason, dirtying pages in the Berkeley DB cache may cause intense -filesystem activity, typically when the filesystem sync thread or -process is run. In some cases, this can dramatically affect application -throughput. The workaround to this problem is to create the shared -regions in system shared memory (<a href="../api_reference/C/envopen.html#envopen_DB_SYSTEM_MEM" class="olink">DB_SYSTEM_MEM</a>) or application -private memory (<a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a>), or, in cases where this behavior -is configurable, to turn off the operating system's flushing of -memory-mapped pages.</dd> + <dd> + By default, Berkeley DB creates its database + environment shared regions in filesystem backed + memory. Some systems do not distinguish between + regular filesystem pages and memory-mapped pages + backed by the filesystem, when selecting dirty pages + to be flushed back to disk. For this reason, dirtying + pages in the Berkeley DB cache may cause intense + filesystem activity, typically when the filesystem + sync thread or process is run. In some cases, this can + dramatically affect application throughput. The + workaround to this problem is to create the shared + regions in system shared memory (<a href="../api_reference/C/envopen.html#envopen_DB_SYSTEM_MEM" class="olink">DB_SYSTEM_MEM</a>) or + application private memory (<a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a>), or, in + cases where this behavior is configurable, to turn off + the operating system's flushing of memory-mapped + pages. + </dd> <dt> <span class="term">large key/data items</span> </dt> - <dd>Storing large key/data items in a database can alter the performance -characteristics of Btree, Hash and Recno databases. The first parameter -to consider is the database page size. When a key/data item is too -large to be placed on a database page, it is stored on "overflow" pages -that are maintained outside of the normal database structure (typically, -items that are larger than one-quarter of the page size are deemed to -be too large). Accessing these overflow pages requires at least one -additional page reference over a normal access, so it is usually better -to increase the page size than to create a database with a large number -of overflow pages. Use the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility (or the statistics -returned by the <a href="../api_reference/C/dbstat.html" class="olink">DB->stat()</a> method) to review the number of overflow -pages in the database. -<p>The second issue is using large key/data items instead of duplicate data -items. While this can offer performance gains to some applications -(because it is possible to retrieve several data items in a single get -call), once the key/data items are large enough to be pushed off-page, -they will slow the application down. Using duplicate data items is -usually the better choice in the long run.</p></dd> + <dd> + Storing large key/data items in a database can + alter the performance characteristics of Btree, Hash + and Recno databases. The first parameter to consider + is the database page size. When a key/data item is too + large to be placed on a database page, it is stored on + "overflow" pages that are maintained outside of the + normal database structure (typically, items that are + larger than one-quarter of the page size are deemed to + be too large). Accessing these overflow pages requires + at least one additional page reference over a normal + access, so it is usually better to increase the page + size than to create a database with a large number of + overflow pages. Use the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility (or the statistics + returned by the <a href="../api_reference/C/dbstat.html" class="olink">DB->stat()</a> method) to review the number + of overflow pages in the database. + <p> + The second + issue is using large key/data items instead of + duplicate data items. While this can offer + performance gains to some applications (because it + is possible to retrieve several data items in a + single get call), once the key/data items are + large enough to be pushed off-page, they will slow + the application down. Using duplicate data items + is usually the better choice in the long + run. + </p></dd> </dl> </div> - <p>A common question when tuning Berkeley DB applications is scalability. For -example, people will ask why, when adding additional threads or -processes to an application, the overall database throughput decreases, -even when all of the operations are read-only queries.</p> - <p>First, while read-only operations are logically concurrent, they still -have to acquire mutexes on internal Berkeley DB data structures. For example, -when searching a linked list and looking for a database page, the linked -list has to be locked against other threads of control attempting to add -or remove pages from the linked list. The more threads of control you -add, the more contention there will be for those shared data structure -resources.</p> - <p>Second, once contention starts happening, applications will also start -to see threads of control convoy behind locks (especially on -architectures supporting only test-and-set spin mutexes, rather than -blocking mutexes). On test-and-set architectures, threads of control -waiting for locks must attempt to acquire the mutex, sleep, check the -mutex again, and so on. Each failed check of the mutex and subsequent -sleep wastes CPU and decreases the overall throughput of the system.</p> - <p>Third, every time a thread acquires a shared mutex, it has to shoot down -other references to that memory in every other CPU on the system. Many -modern snoopy cache architectures have slow shoot down characteristics.</p> - <p>Fourth, schedulers don't care what application-specific mutexes a thread -of control might hold when de-scheduling a thread. If a thread of -control is descheduled while holding a shared data structure mutex, -other threads of control will be blocked until the scheduler decides to -run the blocking thread of control again. The more threads of control -that are running, the smaller their quanta of CPU time, and the more -likely they will be descheduled while holding a Berkeley DB mutex.</p> - <p>The results of adding new threads of control to an application, on the -application's throughput, is application and hardware specific and -almost entirely dependent on the application's data access pattern and -hardware. In general, using operating systems that support blocking -mutexes will often make a tremendous difference, and limiting threads -of control to to some small multiple of the number of CPUs is usually -the right choice to make.</p> + <p> + A common question when tuning Berkeley DB applications is + scalability. For example, people will ask why, when adding + additional threads or processes to an application, the overall + database throughput decreases, even when all of the operations + are read-only queries. + </p> + <p> + First, while read-only operations are logically concurrent, + they still have to acquire mutexes on internal Berkeley DB + data structures. For example, when searching a linked list and + looking for a database page, the linked list has to be locked + against other threads of control attempting to add or remove + pages from the linked list. The more threads of control you + add, the more contention there will be for those shared data + structure resources. + </p> + <p> + Second, once contention starts happening, applications will + also start to see threads of control convoy behind locks + (especially on architectures supporting only test-and-set spin + mutexes, rather than blocking mutexes). On test-and-set + architectures, threads of control waiting for locks must + attempt to acquire the mutex, sleep, check the mutex again, + and so on. Each failed check of the mutex and subsequent sleep + wastes CPU and decreases the overall throughput of the + system. + </p> + <p> + Third, every time a thread acquires a shared mutex, it has + to shoot down other references to that memory in every other + CPU on the system. Many modern snoopy cache architectures have + slow shoot down characteristics. + </p> + <p> + Fourth, schedulers don't care what application-specific + mutexes a thread of control might hold when de-scheduling a + thread. If a thread of control is descheduled while holding a + shared data structure mutex, other threads of control will be + blocked until the scheduler decides to run the blocking thread + of control again. The more threads of control that are + running, the smaller their quanta of CPU time, and the more + likely they will be descheduled while holding a Berkeley DB + mutex. + </p> + <p> + The results of adding new threads of control to an + application, on the application's throughput, is application + and hardware specific and almost entirely dependent on the + application's data access pattern and hardware. In general, + using operating systems that support blocking mutexes will + often make a tremendous difference, and limiting threads of + control to to some small multiple of the number of CPUs is + usually the right choice to make. + </p> </div> <div class="navfooter"> <hr /> @@ -163,7 +216,8 @@ the right choice to make.</p> <td width="40%" align="right"> <a accesskey="n" href="am_misc_faq.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Specifying a Berkeley DB schema using SQL DDL </td> + <td width="40%" align="left" valign="top">Specifying a Berkeley DB schema + using SQL DDL </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/am_opensub.html b/docs/programmer_reference/am_opensub.html index e071976d..0fb6d8be 100644 --- a/docs/programmer_reference/am_opensub.html +++ b/docs/programmer_reference/am_opensub.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Opening multiple databases in a single file</th> + <th colspan="3" align="center">Opening multiple databases in a + single file</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am.html">Prev</a> </td> - <th width="60%" align="center">Chapter 3. - Access Method Operations - </th> + <th width="60%" align="center">Chapter 3. Access Method Operations </th> <td width="20%" align="right"> <a accesskey="n" href="am_partition.html">Next</a></td> </tr> </table> @@ -34,7 +33,8 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="am_opensub"></a>Opening multiple databases in a single file</h2> + <h2 class="title" style="clear: both"><a id="am_opensub"></a>Opening multiple databases in a + single file</h2> </div> </div> </div> @@ -42,103 +42,137 @@ <dl> <dt> <span class="sect2"> - <a href="am_opensub.html#idp724392">Configuring databases sharing a file</a> + <a href="am_opensub.html#idm1925008">Configuring databases sharing a file</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_opensub.html#idp768720">Caching databases sharing a file</a> + <a href="am_opensub.html#idm148416">Caching databases sharing a file</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_opensub.html#idp769416">Locking in databases based on sharing a file</a> + <a href="am_opensub.html#idm548184">Locking in databases based on sharing a file</a> </span> </dt> </dl> </div> - <p>Applications may create multiple databases within a single physical -file. This is useful when the databases are both numerous and -reasonably small, in order to avoid creating a large number of -underlying files, or when it is desirable to include secondary index -databases in the same file as the primary index database. Putting -multiple databases in a single physical file is an administrative -convenience and unlikely to affect database performance.</p> - <p>To open or create a file that will include more than a single database, -specify a database name when calling the <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> method.</p> - <p>Physical files do not need to be comprised of a single type of database, -and databases in a file may be of any mixture of types, except for Queue and Heap -databases. Queue and Heap databases must be created one per file and cannot -share a file with any other database type. There is no limit on the -number of databases that may be created in a single file other than the -standard Berkeley DB file size and disk space limitations.</p> - <p>It is an error to attempt to open a second database in a file that was -not initially created using a database name, that is, the file must -initially be specified as capable of containing multiple databases for a -second database to be created in it.</p> - <p>It is not an error to open a file that contains multiple databases -without specifying a database name, however the database type should be -specified as DB_UNKNOWN and the database must be opened read-only. The -handle that is returned from such a call is a handle on a database whose -key values are the names of the databases stored in the database file -and whose data values are opaque objects. No keys or data values may be -modified or stored using this database handle.</p> + <p> + Applications may create multiple databases within a single + physical file. This is useful when the databases are both + numerous and reasonably small, in order to avoid creating a + large number of underlying files, or when it is desirable to + include secondary index databases in the same file as the + primary index database. Putting multiple databases in a single + physical file is an administrative convenience and unlikely to + affect database performance. + </p> + <p> + To open or create a file that will include more than a + single database, specify a database name when calling the + <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> method. + </p> + <p> + Physical files do not need to be comprised of a single type + of database, and databases in a file may be of any mixture of + types, except for Queue and Heap databases. Queue and Heap + databases must be created one per file and cannot share a file + with any other database type. There is no limit on the number + of databases that may be created in a single file other than + the standard Berkeley DB file size and disk space + limitations. + </p> + <p> + It is an error to attempt to open a second database in a + file that was not initially created using a database name, + that is, the file must initially be specified as capable of + containing multiple databases for a second database to be + created in it. + </p> + <p> + It is not an error to open a file that contains multiple + databases without specifying a database name, however the + database type should be specified as DB_UNKNOWN and the + database must be opened read-only. The handle that is returned + from such a call is a handle on a database whose key values + are the names of the databases stored in the database file and + whose data values are opaque objects. No keys or data values + may be modified or stored using this database handle. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp724392"></a>Configuring databases sharing a file</h3> + <h3 class="title"><a id="idm1925008"></a>Configuring databases sharing a file</h3> </div> </div> </div> - <p>There are four pieces of configuration information which must be -specified consistently for all databases in a file, rather than -differing on a per-database basis. They are: byte order, checksum and -encryption behavior, and page size. When creating additional databases -in a file, any of these configuration values specified must be -consistent with the existing databases in the file or an error will be -returned.</p> + <p> + There are four pieces of configuration information which + must be specified consistently for all databases in a + file, rather than differing on a per-database basis. They + are: byte order, checksum and encryption behavior, and + page size. When creating additional databases in a file, + any of these configuration values specified must be + consistent with the existing databases in the file or an + error will be returned. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp768720"></a>Caching databases sharing a file</h3> + <h3 class="title"><a id="idm148416"></a>Caching databases sharing a file</h3> </div> </div> </div> - <p>When storing multiple databases in a single physical file rather than -in separate files, if any of the databases in a file is opened for -update, all of the databases in the file must share a memory pool. In -other words, they must be opened in the same database environment. This -is so per-physical-file information common between the two databases is -updated correctly.</p> + <p> + When storing multiple databases in a single physical + file rather than in separate files, if any of the + databases in a file is opened for update, all of the + databases in the file must share a memory pool. In other + words, they must be opened in the same database + environment. This is so per-physical-file information + common between the two databases is updated + correctly. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp769416"></a>Locking in databases based on sharing a file</h3> + <h3 class="title"><a id="idm548184"></a>Locking in databases based on sharing a file</h3> </div> </div> </div> - <p>If databases are in separate files (and access to each separate database -is single-threaded), there is no reason to perform any locking of any -kind, and the two databases may be read and written simultaneously. -Further, there would be no requirement to create a shared database -environment in which to open those two databases.</p> - <p>However, since multiple databases in a file exist in a single physical -file, opening two databases in the same file simultaneously requires -locking be enabled, unless all of the databases are read-only. As the -locks for the two databases can only conflict during page allocation, -this additional locking is unlikely to affect performance. The -exception is when Berkeley DB Concurrent Data Store is configured; a single lock is used for all -databases in the file when Berkeley DB Concurrent Data Store is configured, and a write to one -database will block all accesses to all databases.</p> - <p>In summary, programmers writing applications that open multiple -databases in a single file will almost certainly need to create a shared -database environment in the application as well. For more information -on database environments, see <a class="xref" href="env.html#env_intro" title="Database environment introduction">Database environment introduction</a></p> + <p> + If databases are in separate files (and access to each + separate database is single-threaded), there is no reason + to perform any locking of any kind, and the two databases + may be read and written simultaneously. Further, there + would be no requirement to create a shared database + environment in which to open those two databases. + </p> + <p> + However, since multiple databases in a file exist in a + single physical file, opening two databases in the same + file simultaneously requires locking be enabled, unless + all of the databases are read-only. As the locks for the + two databases can only conflict during page allocation, + this additional locking is unlikely to affect performance. + The exception is when Berkeley DB Concurrent Data Store is + configured; a single lock is used for all databases in the + file when Berkeley DB Concurrent Data Store is configured, + and a write to one database will block all accesses to all + databases. + </p> + <p> + In summary, programmers writing applications that open + multiple databases in a single file will almost certainly + need to create a shared database environment in the + application as well. For more information on database + environments, see <a class="xref" href="env.html#env_intro" title="Database environment introduction">Database environment introduction</a> + </p> </div> </div> <div class="navfooter"> @@ -152,9 +186,7 @@ on database environments, see <a class="xref" href="env.html#env_intro" title="D <td width="40%" align="right"> <a accesskey="n" href="am_partition.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 3. - Access Method Operations - </td> + <td width="40%" align="left" valign="top">Chapter 3. Access Method Operations </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/am_partition.html b/docs/programmer_reference/am_partition.html index e6782175..9bc8879d 100644 --- a/docs/programmer_reference/am_partition.html +++ b/docs/programmer_reference/am_partition.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_opensub.html">Prev</a> </td> - <th width="60%" align="center">Chapter 3. - Access Method Operations - </th> + <th width="60%" align="center">Chapter 3. Access Method Operations </th> <td width="20%" align="right"> <a accesskey="n" href="am_get.html">Next</a></td> </tr> </table> @@ -42,136 +40,133 @@ <dl> <dt> <span class="sect2"> - <a href="am_partition.html#am_partition_keys">Specifying partition keys</a> + <a href="am_partition.html#am_partition_keys">Specifying partition + keys</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_partition.html#am_partition_function">Partitioning callback</a> + <a href="am_partition.html#am_partition_function">Partitioning + callback</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_partition.html#partition_file_placement">Placing partition files</a> + <a href="am_partition.html#partition_file_placement">Placing partition + files</a> </span> </dt> </dl> </div> - <p> - You can improve concurrency on your database reads and writes by - splitting access to a single database into multiple databases. This - helps to avoid contention for internal database pages, as well as - allowing you to spread your databases across multiple disks, - which can help to improve disk I/O. -</p> - <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> - <h3 class="title">Note</h3> - <p> - Database partitions are not supported by the C# and Java APIs at - this time. + <p> + You can improve concurrency on your database reads and + writes by splitting access to a single database into multiple + databases. This helps to avoid contention for internal + database pages, as well as allowing you to spread your + databases across multiple disks, which can help to improve + disk I/O. + </p> + <p> + While you can manually do this by creating and using more + than one database for your data, DB is capable of + partitioning your database for you. When you use DB's + built-in database partitioning feature, your access to your + data is performed in exactly the same way as if you were only + using one database; all the work of knowing which database to + use to access a particular record is handled for you under the + hood. + </p> + <p> + Only the BTree and Hash access methods are supported for + partitioned databases. + </p> + <p> + You indicate that you want your database to be partitioned + by calling <a href="../api_reference/C/dbset_partition.html" class="olink">DB->set_partition()</a> before opening your database the + first time. You can indicate the directory in which each + partition is contained using the <a href="../api_reference/C/dbset_partition_dirs.html" class="olink">DB->set_partition_dirs()</a> + method. + </p> + <p> + Once you have partitioned a database, you cannot change + your partitioning scheme. + </p> + <p> + There are two ways to indicate what key/data pairs should + go on which partition. The first is by specifying an array of + <a href="../api_reference/C/dbt.html" class="olink">DBT</a>s that indicate the minimum key value for a given + partition. The second is by providing a callback that returns + the number of the partition on which a specified key is + placed. </p> - </div> - <p> - While you can manually do this by creating and using more than one - database for your data, DB is capable of partitioning your - database for you. When you use DB's built-in database partitioning - feature, your access to your data is performed in exactly the same way - as if you were only using one database; all the work of knowing which - database to use to access a particular record is handled for you under - the hood. -</p> - <p> - Only the BTree and Hash access methods are supported for partitioned - databases. -</p> - <p> - You indicate that you want your database to be partitioned by calling - <a href="../api_reference/C/dbset_partition.html" class="olink">DB->set_partition()</a> before opening your database the first time. You can - indicate the directory in which each partition is contained using the - <a href="../api_reference/C/dbset_partition_dirs.html" class="olink">DB->set_partition_dirs()</a> method. -</p> - <p> - Once you have partitioned a database, you cannot change your - partitioning scheme. -</p> - <p> - There are two ways to indicate what key/data pairs should go on which - partition. The first is by specifying an array of <a href="../api_reference/C/dbt.html" class="olink">DBT</a>s that indicate - the minimum key value for a given partition. The second is by providing - a callback that returns the number of the partition on which a specified - key is placed. -</p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="am_partition_keys"></a>Specifying partition keys</h3> + <h3 class="title"><a id="am_partition_keys"></a>Specifying partition + keys</h3> </div> </div> </div> <p> - For simple cases, you can partition your database by providing - an array of <a href="../api_reference/C/dbt.html" class="olink">DBT</a>s, each element of which provides the minimum - key value to be placed on a partition. There must be one fewer - elements in this array than you have partitions. The first - element of the array indicates the minimum key value for the - second partition in your database. Key values that are less - than the first key value provided in this array are placed on - the first partition (partition 0). + For simple cases, you can partition your database by + providing an array of <a href="../api_reference/C/dbt.html" class="olink">DBT</a>s, each element of which + provides the minimum key value to be placed on a + partition. There must be one fewer elements in this array + than you have partitions. The first element of the array + indicates the minimum key value for the second partition + in your database. Key values that are less than the first + key value provided in this array are placed on the first + partition (partition 0). </p> <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> <h3 class="title">Note</h3> - <p> - You can use partition keys only if you are using the Btree - access method. + <p> + You can use partition keys only if you are using + the Btree access method. </p> </div> <p> - For example, suppose you had a database of fruit, and you want - three partitions for your database. Then you need a <a href="../api_reference/C/dbt.html" class="olink">DBT</a> array - of size two. The first element in this array indicates the - minimum keys that should be placed on partition 1. The second - element in this array indicates the minimum key value placed on - partition 2. Keys that compare less than the first <a href="../api_reference/C/dbt.html" class="olink">DBT</a> in the - array are placed on partition 0. + For example, suppose you had a database of fruit, and + you want three partitions for your database. Then you need + a <a href="../api_reference/C/dbt.html" class="olink">DBT</a> array of size two. The first element in this array + indicates the minimum keys that should be placed on + partition 1. The second element in this array indicates + the minimum key value placed on partition 2. Keys that + compare less than the first <a href="../api_reference/C/dbt.html" class="olink">DBT</a> in the array are placed + on partition 0. </p> - <p> - All comparisons are performed according to the lexicographic - comparison used by your platform. + <p> + All comparisons are performed according to the + lexicographic comparison used by your platform. </p> - <p> - For example, suppose you want all fruits whose names begin - with: + <p> + For example, suppose you want all fruits whose names + begin with: </p> <div class="itemizedlist"> <ul type="disc"> <li> - <p> - 'a' - 'f' to go on partition 0 - </p> + <p> 'a' - 'f' to go on partition 0 </p> </li> <li> - <p> - 'g' - 'p' to go on partition 1 - </p> + <p> 'g' - 'p' to go on partition 1 </p> </li> <li> - <p> - 'q' - 'z' to go on partition 2. - </p> + <p> 'q' - 'z' to go on partition 2. </p> </li> </ul> </div> - <p> + <p> Then you would accomplish this with the following code - fragment: + fragment: </p> <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> <h3 class="title">Note</h3> - <p> - The <a href="../api_reference/C/dbset_partition.html" class="olink">DB->set_partition()</a> partition callback parameter must - be <code class="literal">NULL</code> if you are using an array of - <a href="../api_reference/C/dbt.html" class="olink">DBT</a>s to partition your database. + <p> + The <a href="../api_reference/C/dbset_partition.html" class="olink">DB->set_partition()</a> partition callback parameter + must be <code class="literal">NULL</code> if you are using an + array of <a href="../api_reference/C/dbt.html" class="olink">DBT</a>s to partition your database. </p> </div> <a id="prog_am10"></a> @@ -225,17 +220,18 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="am_partition_function"></a>Partitioning callback</h3> + <h3 class="title"><a id="am_partition_function"></a>Partitioning + callback</h3> </div> </div> </div> <p> - In some cases, a simple lexicographical comparison of key data - will not sufficiently support a partitioning scheme. For - those situations, you should write a partitioning function. - This function accepts a pointer to the <a href="../api_reference/C/db.html" class="olink">DB</a> and the <a href="../api_reference/C/dbt.html" class="olink">DBT</a>, and - it returns the number of the partition on which the key - belongs. + In some cases, a simple lexicographical comparison of + key data will not sufficiently support a partitioning + scheme. For those situations, you should write a + partitioning function. This function accepts a pointer to + the <a href="../api_reference/C/db.html" class="olink">DB</a> and the <a href="../api_reference/C/dbt.html" class="olink">DBT</a>, and it returns the number of the + partition on which the key belongs. </p> <p> Note that <a href="../api_reference/C/db.html" class="olink">DB</a> actually places the key on the partition @@ -243,12 +239,14 @@ </p> <pre class="programlisting">returned_partition modulo number_of_partitions</pre> <p> - Also, remember that if you use a partitioning function when you - create your database, then you must use the same partitioning - function every time you open that database in the future. + Also, remember that if you use a partitioning function + when you create your database, then you must use the same + partitioning function every time you open that database in + the future. </p> <p> - The following code fragment illustrates a partition callback: + The following code fragment illustrates a partition + callback: </p> <a id="prog_am11"></a> <pre class="programlisting">u_int32_t db_partition_fn(DB *db, DBT *key) { @@ -272,17 +270,17 @@ return ret_number; } </pre> - <p> - You then cause your partition callback to be used by providing it - to the <a href="../api_reference/C/dbset_partition.html" class="olink">DB->set_partition()</a> method, as illustrated by the following - code fragment. - </p> + <p> + You then cause your partition callback to be used by + providing it to the <a href="../api_reference/C/dbset_partition.html" class="olink">DB->set_partition()</a> method, as + illustrated by the following code fragment. + </p> <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> <h3 class="title">Note</h3> <p> - The <a href="../api_reference/C/dbset_partition.html" class="olink">DB->set_partition()</a> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> array parameter must - be <code class="literal">NULL</code> if you are using a partition - call back to partition your database. + The <a href="../api_reference/C/dbset_partition.html" class="olink">DB->set_partition()</a> <a href="../api_reference/C/dbt.html" class="olink">DBT</a> array parameter must be + <code class="literal">NULL</code> if you are using a + partition call back to partition your database. </p> </div> <a id="prog_am12"></a> @@ -326,66 +324,73 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="partition_file_placement"></a>Placing partition files</h3> + <h3 class="title"><a id="partition_file_placement"></a>Placing partition + files</h3> </div> </div> </div> <p> - When you partition a database, a database file is created on - disk in the same way as if you were not partitioning the - database. That is, this file uses the name you provide to the - <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> <code class="literal">file</code> parameter. + When you partition a database, a database file is + created on disk in the same way as if you were not + partitioning the database. That is, this file uses the + name you provide to the <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> <code class="literal">file</code> + parameter. </p> - <p> - However, DB then also creates a series of database files on - disk, one for each partition that you want to use. These - partition files share the same name as the database file name, - but are also number sequentially. So if you create a database - named <code class="filename">mydb.db</code>, and you create 3 partitions - for it, then you will see the following database files on disk: + <p> + However, DB then also creates a series of database + files on disk, one for each partition that you want to + use. These partition files share the same name as the + database file name, but are also number sequentially. So + if you create a database named + <code class="filename">mydb.db</code>, and you create 3 + partitions for it, then you will see the following + database files on disk: </p> <pre class="programlisting"> mydb.db __dbp.mydb.db.000 __dbp.mydb.db.001 __dbp.mydb.db.002 </pre> - <p> - All of the database's contents go into the numbered database - files. You can cause these files to be placed in different - directories (and, hence, different disk partitions or even - disks) by using the <a href="../api_reference/C/dbset_partition_dirs.html" class="olink">DB->set_partition_dirs()</a> method. + <p> + All of the database's contents go into the numbered + database files. You can cause these files to be placed in + different directories (and, hence, different disk + partitions or even disks) by using the + <a href="../api_reference/C/dbset_partition_dirs.html" class="olink">DB->set_partition_dirs()</a> method. </p> - <p> + <p> <a href="../api_reference/C/dbset_partition_dirs.html" class="olink">DB->set_partition_dirs()</a> takes a NULL-terminated array of strings, each one of which should represent an existing filesystem directory. </p> - <p> - If you are using an environment, the directories specified - using <a href="../api_reference/C/dbset_partition_dirs.html" class="olink">DB->set_partition_dirs()</a> must also be included in the - environment list specified by <a href="../api_reference/C/envadd_data_dir.html" class="olink">DB_ENV->add_data_dir()</a>. + <p> + If you are using an environment, the directories + specified using <a href="../api_reference/C/dbset_partition_dirs.html" class="olink">DB->set_partition_dirs()</a> must also be + included in the environment list specified by + <a href="../api_reference/C/envadd_data_dir.html" class="olink">DB_ENV->add_data_dir()</a>. </p> <p> - If you are not using an environment, then the the directories - specified to <a href="../api_reference/C/dbset_partition_dirs.html" class="olink">DB->set_partition_dirs()</a> can be either complete - paths to currently existing directories, or paths relative to - the application's current working directory. + If you are not using an environment, then the the + directories specified to <a href="../api_reference/C/dbset_partition_dirs.html" class="olink">DB->set_partition_dirs()</a> can be + either complete paths to currently existing directories, + or paths relative to the application's current working + directory. </p> - <p> - Ideally, you will provide <a href="../api_reference/C/dbset_partition_dirs.html" class="olink">DB->set_partition_dirs()</a> with an array - that is the same size as the number of partitions you are - creating for your database. Partition files are then placed - according to the order that directories are contained in the - array; partition 0 is placed in directory_array[0], partition 1 - in directory_array[1], and so forth. However, if you provide an - array of directories that is smaller than the number of - database partitions, then the directories are used on a - round-robin fashion. + <p> + Ideally, you will provide <a href="../api_reference/C/dbset_partition_dirs.html" class="olink">DB->set_partition_dirs()</a> with + an array that is the same size as the number of partitions + you are creating for your database. Partition files are + then placed according to the order that directories are + contained in the array; partition 0 is placed in + directory_array[0], partition 1 in directory_array[1], and + so forth. However, if you provide an array of directories + that is smaller than the number of database partitions, + then the directories are used on a round-robin fashion. </p> <p> - You must call <a href="../api_reference/C/dbset_partition_dirs.html" class="olink">DB->set_partition_dirs()</a> before you create your - database, and before you open your database each time - thereafter. The array provided to <a href="../api_reference/C/dbset_partition_dirs.html" class="olink">DB->set_partition_dirs()</a> must not - change after the database has been created. + You must call <a href="../api_reference/C/dbset_partition_dirs.html" class="olink">DB->set_partition_dirs()</a> before you create + your database, and before you open your database each time + thereafter. The array provided to <a href="../api_reference/C/dbset_partition_dirs.html" class="olink">DB->set_partition_dirs()</a> + must not change after the database has been created. </p> </div> </div> @@ -400,7 +405,8 @@ <td width="40%" align="right"> <a accesskey="n" href="am_get.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Opening multiple databases in a single file </td> + <td width="40%" align="left" valign="top">Opening multiple databases in a + single file </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/am_put.html b/docs/programmer_reference/am_put.html index 1797eac0..9cd323c7 100644 --- a/docs/programmer_reference/am_put.html +++ b/docs/programmer_reference/am_put.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_get.html">Prev</a> </td> - <th width="60%" align="center">Chapter 3. - Access Method Operations - </th> + <th width="60%" align="center">Chapter 3. Access Method Operations </th> <td width="20%" align="right"> <a accesskey="n" href="am_delete.html">Next</a></td> </tr> </table> @@ -39,12 +37,13 @@ </div> </div> <p> - The <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> method stores records into the database. In general, - <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> takes a key and stores the associated data into the - database. + The <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> method stores records into the database. In + general, <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> takes a key and stores the associated data + into the database. </p> <p> - There are a few flags that you can set to customize storage: + There are a few flags that you can set to customize + storage: </p> <div class="variablelist"> <dl> @@ -53,39 +52,40 @@ <a href="../api_reference/C/dbput.html#dbput_DB_APPEND" class="olink">DB_APPEND</a> </span> </dt> - <dd> - Simply append the data to the end of the database, treating - the database much like a simple log. This flag is only - valid for the Heap, Queue and Recno access methods. - This flag is required if you are creating a new record in a Heap - database. + <dd> + Simply append the data to the end of the + database, treating the database much like a simple + log. This flag is only valid for the Heap, Queue and + Recno access methods. This flag is required if you are + creating a new record in a Heap database. </dd> <dt> <span class="term"> <a href="../api_reference/C/dbput.html#put_DB_NOOVERWRITE" class="olink">DB_NOOVERWRITE</a> </span> </dt> - <dd> - Only store the data item if the key does not already appear - in the database. + <dd> + Only store the data item if the key does not + already appear in the database. </dd> </dl> </div> - <p> - If the database has been configured to support duplicate records, - the <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> method will add the new data value at the end of the - duplicate set. If the database supports sorted duplicates, the new - data value is inserted at the correct sorted location. + <p> + If the database has been configured to support duplicate + records, the <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> method will add the new data value at the + end of the duplicate set. If the database supports sorted + duplicates, the new data value is inserted at the correct + sorted location. </p> <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> <h3 class="title">Note</h3> <p> - If you are using the Heap access method and you are creating a - new record in the database, then the key that you provide to - the <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> method should be empty. The <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> method will - return the record's ID (RID) in the key. The RID is - automatically created for you when Heap database records are - created. + If you are using the Heap access method and you are + creating a new record in the database, then the key that + you provide to the <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> method should be empty. The + <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> method will return the record's ID (RID) in the + key. The RID is automatically created for you when Heap + database records are created. </p> </div> </div> diff --git a/docs/programmer_reference/am_second.html b/docs/programmer_reference/am_second.html index 2777904f..1056ff0c 100644 --- a/docs/programmer_reference/am_second.html +++ b/docs/programmer_reference/am_second.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_close.html">Prev</a> </td> - <th width="60%" align="center">Chapter 3. - Access Method Operations - </th> + <th width="60%" align="center">Chapter 3. Access Method Operations </th> <td width="20%" align="right"> <a accesskey="n" href="am_foreign.html">Next</a></td> </tr> </table> @@ -42,33 +40,46 @@ <dl> <dt> <span class="sect2"> - <a href="am_second.html#idp863384">Error Handling With Secondary Indexes</a> + <a href="am_second.html#idp382496">Error Handling With Secondary Indexes</a> </span> </dt> </dl> </div> - <p>A secondary index, put simply, is a way to efficiently access records -in a database (the primary) by means of some piece of information other -than the usual (primary) key. In Berkeley DB, this index is simply another -database whose keys are these pieces of information (the secondary -keys), and whose data are the primary keys. Secondary indexes can be -created manually by the application; there is no disadvantage, other -than complexity, to doing so. However, when the secondary key can be -mechanically derived from the primary key and datum that it points to, -as is frequently the case, Berkeley DB can automatically and transparently -manage secondary indexes.</p> - <p>As an example of how secondary indexes might be used, consider a -database containing a list of students at a college, each of whom has -a unique student ID number. A typical database would use the student -ID number as the key; however, one might also reasonably want to be -able to look up students by last name. To do this, one would construct -a secondary index in which the secondary key was this last name.</p> - <p>In SQL, this would be done by executing something like the following:</p> + <p> + A secondary index, put simply, is a way to efficiently + access records in a database (the primary) by means of some + piece of information other than the usual (primary) key. In + Berkeley DB, this index is simply another database whose keys + are these pieces of information (the secondary keys), and + whose data are the primary keys. Secondary indexes can be + created manually by the application; there is no disadvantage, + other than complexity, to doing so. However, when the + secondary key can be mechanically derived from the primary key + and datum that it points to, as is frequently the case, + Berkeley DB can automatically and transparently manage + secondary indexes. + </p> + <p> + As an example of how secondary indexes might be used, + consider a database containing a list of students at a + college, each of whom has a unique student ID number. A + typical database would use the student ID number as the key; + however, one might also reasonably want to be able to look up + students by last name. To do this, one would construct a + secondary index in which the secondary key was this last + name. + </p> + <p> + In SQL, this would be done by executing something like the + following: + </p> <pre class="programlisting">CREATE TABLE students(student_id CHAR(4) NOT NULL, lastname CHAR(15), firstname CHAR(15), PRIMARY KEY(student_id)); CREATE INDEX lname ON students(lastname);</pre> - <p>In Berkeley DB, this would work as follows (a -<a class="ulink" href="second.javas" target="_top">Java API example is also available</a>):</p> + <p> + In Berkeley DB, this would work as follows (a <a class="ulink" href="second.javas" target="_top">Java API example is also + available</a>): + </p> <a id="prog_am13"></a> <pre class="programlisting">struct student_record { char student_id[4]; @@ -130,13 +141,17 @@ getname(DB *secondary, const DBT *pkey, const DBT *pdata, DBT *skey) skey->size = sizeof(((struct student_record *)pdata->data)->last_name); return (0); }</pre> - <p>From the application's perspective, putting things into the database -works exactly as it does without a secondary index; one can simply -insert records into the primary database. In SQL one would do the -following:</p> + <p> + From the application's perspective, putting things into the + database works exactly as it does without a secondary index; + one can simply insert records into the primary database. In + SQL one would do the following: + </p> <pre class="programlisting">INSERT INTO student VALUES ("WC42", "Churchill ", "Winston ");</pre> - <p>and in Berkeley DB, one does:</p> + <p> + and in Berkeley DB, one does: + </p> <a id="prog_am14"></a> <pre class="programlisting">struct student_record s; DBT data, key; @@ -154,10 +169,10 @@ data.size = sizeof(s); if ((ret = dbp->put(dbp, txn, &key, &data, 0)) != 0) handle_error(ret); </pre> - <p>Internally, a record with secondary key "Churchill" is inserted into -the secondary database (in addition to the insertion of "WC42" into the -primary, of course).</p> - <p>Deletes are similar. The SQL clause:</p> + <p>Internally, a record with secondary key "Churchill" is + inserted into the secondary database (in addition to the + insertion of "WC42" into the primary, of course).</p> + <p>Deletes are similar. The SQL clause:</p> <pre class="programlisting">DELETE FROM student WHERE (student_id = "WC42");</pre> <p>looks like:</p> <a id="prog_am15"></a> @@ -168,14 +183,18 @@ key.data = "WC42"; key.size = 4; if ((ret = dbp->del(dbp, txn, &key, 0)) != 0) handle_error(ret);</pre> - <p>Deletes can also be performed on the secondary index directly; a delete -done this way will delete the "real" record in the primary as well. If -the secondary supports duplicates and there are duplicate occurrences of -the secondary key, then all records with that secondary key are removed -from both the secondary index and the primary database. In -SQL:</p> + <p> + Deletes can also be performed on the secondary index + directly; a delete done this way will delete the "real" record + in the primary as well. If the secondary supports duplicates + and there are duplicate occurrences of the secondary key, then + all records with that secondary key are removed from both the + secondary index and the primary database. In SQL: + </p> <pre class="programlisting">DELETE FROM lname WHERE (lastname = "Churchill ");</pre> - <p>In Berkeley DB:</p> + <p> + In Berkeley DB: + </p> <a id="prog_am16"></a> <pre class="programlisting">DBT skey; @@ -184,11 +203,16 @@ skey.data = "Churchill "; skey.size = 15; if ((ret = sdbp->del(sdbp, txn, &skey, 0)) != 0) handle_error(ret);</pre> - <p>Gets on a secondary automatically return the primary datum. If -<a href="../api_reference/C/dbget.html" class="olink">DB->pget()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->pget()</a> is used in lieu of <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a>, the primary key is returned as well. Thus, the -equivalent of:</p> + <p> + Gets on a secondary automatically return the primary datum. + If <a href="../api_reference/C/dbget.html" class="olink">DB->pget()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->pget()</a> is used in lieu of <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or + <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a>, the primary key is returned as well. Thus, the + equivalent of: + </p> <pre class="programlisting">SELECT * from lname WHERE (lastname = "Churchill ");</pre> - <p>would be:</p> + <p> + would be: + </p> <a id="prog_am17"></a> <pre class="programlisting">DBT data, pkey, skey; @@ -202,86 +226,125 @@ if ((ret = sdbp->pget(sdbp, txn, &skey, &pkey, &data, 0)) != 0) /* * Now pkey contains "WC42" and data contains Winston's record. */</pre> - <p>To create a secondary index to a Berkeley DB database, open the database that -is to become a secondary index normally, then pass it as the "secondary" -argument to the <a href="../api_reference/C/dbassociate.html" class="olink">DB->associate()</a> method for some primary database.</p> - <p>After a <a href="../api_reference/C/dbassociate.html" class="olink">DB->associate()</a> call is made, the secondary indexes become -alternate interfaces to the primary database. All updates to the -primary will be automatically reflected in each secondary index that has -been associated with it. All get operations using the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> -or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> methods on the secondary index return the primary datum -associated with the specified (or otherwise current, in the case of -cursor operations) secondary key. The <a href="../api_reference/C/dbget.html" class="olink">DB->pget()</a> and -<a href="../api_reference/C/dbcget.html" class="olink">DBC->pget()</a> methods also become usable; these behave just like -<a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> and <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a>, but return the primary key in -addition to the primary datum, for those applications that need it as -well.</p> - <p>Cursor get operations on a secondary index perform as expected; although -the data returned will by default be those of the primary database, a -position in the secondary index is maintained normally, and records will -appear in the order determined by the secondary key and the comparison -function or other structure of the secondary database.</p> - <p>Delete operations on a secondary index delete the item from the primary -database and all relevant secondaries, including the current one.</p> - <p>Put operations of any kind are forbidden on secondary indexes, as there -is no way to specify a primary key for a newly put item. Instead, the -application should use the <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> or <a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a> methods -on the primary database.</p> - <p>Any number of secondary indexes may be associated with a given primary -database, up to limitations on available memory and the number of open -file descriptors.</p> - <p>Note that although Berkeley DB guarantees that updates made using any -<a href="../api_reference/C/db.html" class="olink">DB</a> handle with an associated secondary will be reflected in the -that secondary, associating each primary handle with all the appropriate -secondaries is the responsibility of the application and is not enforced -by Berkeley DB. It is generally unsafe, but not forbidden by Berkeley DB, to modify -a database that has secondary indexes without having those indexes open -and associated. Similarly, it is generally unsafe, but not forbidden, -to modify a secondary index directly. Applications that violate these -rules face the possibility of outdated or incorrect results if the -secondary indexes are later used.</p> - <p>If a secondary index becomes outdated for any reason, it should be -discarded using the <a href="../api_reference/C/dbremove.html" class="olink">DB->remove()</a> method and a new one created -using the <a href="../api_reference/C/dbassociate.html" class="olink">DB->associate()</a> method. If a secondary index is no -longer needed, all of its handles should be closed using the -<a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> method, and then the database should be removed using -a new database handle and the <a href="../api_reference/C/dbremove.html" class="olink">DB->remove()</a> method.</p> - <p>Closing a primary database handle automatically dis-associates all -secondary database handles associated with it.</p> + <p> + To create a secondary index to a Berkeley DB database, open + the database that is to become a secondary index normally, + then pass it as the "secondary" argument to the <a href="../api_reference/C/dbassociate.html" class="olink">DB->associate()</a> + method for some primary database. + </p> + <p> + After a <a href="../api_reference/C/dbassociate.html" class="olink">DB->associate()</a> call is made, the secondary indexes + become alternate interfaces to the primary database. All + updates to the primary will be automatically reflected in each + secondary index that has been associated with it. All get + operations using the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> methods on the + secondary index return the primary datum associated with the + specified (or otherwise current, in the case of cursor + operations) secondary key. The <a href="../api_reference/C/dbget.html" class="olink">DB->pget()</a> and <a href="../api_reference/C/dbcget.html" class="olink">DBC->pget()</a> methods + also become usable; these behave just like <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> and + <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a>, but return the primary key in addition to the + primary datum, for those applications that need it as + well. + </p> + <p> + Cursor get operations on a secondary index perform as + expected; although the data returned will by default be those + of the primary database, a position in the secondary index is + maintained normally, and records will appear in the order + determined by the secondary key and the comparison function or + other structure of the secondary database. + </p> + <p> + Delete operations on a secondary index delete the item from + the primary database and all relevant secondaries, including + the current one. + </p> + <p> + Put operations of any kind are forbidden on secondary + indexes, as there is no way to specify a primary key for a + newly put item. Instead, the application should use the + <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> or <a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a> methods on the primary database. + </p> + <p> + Any number of secondary indexes may be associated with a + given primary database, up to limitations on available memory + and the number of open file descriptors. + </p> + <p> + Note that although Berkeley DB guarantees that updates made + using any <a href="../api_reference/C/db.html" class="olink">DB</a> handle with an associated secondary will be + reflected in the that secondary, associating each primary + handle with all the appropriate secondaries is the + responsibility of the application and is not enforced by + Berkeley DB. It is generally unsafe, but not forbidden by + Berkeley DB, to modify a database that has secondary indexes + without having those indexes open and associated. Similarly, + it is generally unsafe, but not forbidden, to modify a + secondary index directly. Applications that violate these + rules face the possibility of outdated or incorrect results if + the secondary indexes are later used. + </p> + <p> + If a secondary index becomes outdated for any reason, it + should be discarded using the <a href="../api_reference/C/dbremove.html" class="olink">DB->remove()</a> method and a new one + created using the <a href="../api_reference/C/dbassociate.html" class="olink">DB->associate()</a> method. If a secondary index + is no longer needed, all of its handles should be closed using + the <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> method, and then the database should be removed + using a new database handle and the <a href="../api_reference/C/dbremove.html" class="olink">DB->remove()</a> method. + </p> + <p> + Closing a primary database handle automatically + dis-associates all secondary database handles associated with + it. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp863384"></a>Error Handling With Secondary Indexes</h3> + <h3 class="title"><a id="idp382496"></a>Error Handling With Secondary Indexes</h3> </div> </div> </div> - <p>An error return during a secondary update in CDS or DS (which requires an abort in TDS) may leave a secondary index inconsistent in CDS or DS. There are a few non-error returns: -</p> + <p> + An error return during a secondary update in CDS or DS + (which requires an abort in TDS) may leave a secondary + index inconsistent in CDS or DS. There are a few non-error + returns: + </p> <div class="itemizedlist"> <ul type="disc"> - <li>0 -</li> - <li>DB_BUFFER_SMALL -</li> - <li>DB_NOTFOUND -</li> - <li>DB_KEYEMPTY -</li> - <li> -DB_KEYEXIST -</li> + <li>0 </li> + <li>DB_BUFFER_SMALL </li> + <li>DB_NOTFOUND </li> + <li>DB_KEYEMPTY </li> + <li> DB_KEYEXIST </li> </ul> </div> - <p> -In the case of any other error return during a secondary update in CDS or DS, delete the secondary indices, recreate them and set the <code class="literal">DB_CREATE flag</code> to the <code class="literal">DB->associate</code> method. Some examples of error returns that need to be handled this way are:</p> + <p> + In the case of any other error return during a + secondary update in CDS or DS, delete the secondary + indices, recreate them and set the <code class="literal">DB_CREATE + flag</code> to the <code class="literal">DB->associate</code> + method. Some examples of error returns that need to be + handled this way are: + </p> <div class="itemizedlist"> <ul type="disc"> - <li>ENOMEM - indicating there is insufficient memory to return the requested item</li> - <li>EINVAL - indicating that an invalid flag value or parameter is specified</li> + <li>ENOMEM - indicating there is insufficient memory + to return the requested item</li> + <li>EINVAL - indicating that an invalid flag value + or parameter is specified</li> </ul> </div> - <p>Note that <code class="literal">DB_RUNRECOVERY</code> and <code class="literal">DB_PAGE_NOTFOUND</code> are fatal errors which should never occur during normal use of CDS or DS. If those errors are returned by Berkeley DB when running without transactions, check the database integrity with the <code class="literal">DB->verify</code> method before rebuilding the secondary indices. </p> + <p> + Note that <code class="literal">DB_RUNRECOVERY</code> and + <code class="literal">DB_PAGE_NOTFOUND</code> are fatal errors + which should never occur during normal use of CDS or DS. + If those errors are returned by Berkeley DB when running + without transactions, check the database integrity with + the <code class="literal">DB->verify</code> method before rebuilding + the secondary indices. + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/am_stat.html b/docs/programmer_reference/am_stat.html index 1663a9cc..1c3a66b7 100644 --- a/docs/programmer_reference/am_stat.html +++ b/docs/programmer_reference/am_stat.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_delete.html">Prev</a> </td> - <th width="60%" align="center">Chapter 3. - Access Method Operations - </th> + <th width="60%" align="center">Chapter 3. Access Method Operations </th> <td width="20%" align="right"> <a accesskey="n" href="am_truncate.html">Next</a></td> </tr> </table> @@ -38,10 +36,16 @@ </div> </div> </div> - <p>The <a href="../api_reference/C/dbstat.html" class="olink">DB->stat()</a> method returns a set of statistics about the underlying -database, for example, the number of key/data pairs in the database, how -the database was originally configured, and so on.</p> - <p>There is a flag you can set to avoid time-consuming operations:</p> + <p> + The <a href="../api_reference/C/dbstat.html" class="olink">DB->stat()</a> method returns a set of statistics about the + underlying database, for example, the number of key/data pairs + in the database, how the database was originally configured, + and so on. + </p> + <p> + There is a flag you can set to avoid time-consuming + operations: + </p> <div class="variablelist"> <dl> <dt> @@ -49,8 +53,10 @@ the database was originally configured, and so on.</p> <a href="../api_reference/C/dbstat.html#stat_DB_FAST_STAT" class="olink">DB_FAST_STAT</a> </span> </dt> - <dd>Return only information that can be acquired without traversing the -entire database.</dd> + <dd> + Return only information that can be acquired + without traversing the entire database. + </dd> </dl> </div> </div> diff --git a/docs/programmer_reference/am_sync.html b/docs/programmer_reference/am_sync.html index 12ac2310..366cdbfb 100644 --- a/docs/programmer_reference/am_sync.html +++ b/docs/programmer_reference/am_sync.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_verify.html">Prev</a> </td> - <th width="60%" align="center">Chapter 3. - Access Method Operations - </th> + <th width="60%" align="center">Chapter 3. Access Method Operations </th> <td width="20%" align="right"> <a accesskey="n" href="am_close.html">Next</a></td> </tr> </table> @@ -38,26 +36,39 @@ </div> </div> </div> - <p>The <a href="../api_reference/C/dbsync.html" class="olink">DB->sync()</a> method flushes all modified records from the database -cache to disk.</p> <p> - <span class="bold"> - <strong>It is important to understand that flushing cached information -to disk only minimizes the window of opportunity for corrupted data, it -does not eliminate the possibility.</strong> - </span> - </p> - <p>While unlikely, it is possible for database corruption to happen if a -system or application crash occurs while writing data to the database. To -ensure that database corruption never occurs, applications must either:</p> + The <a href="../api_reference/C/dbsync.html" class="olink">DB->sync()</a> method flushes all modified records from the + database cache to disk. + </p> + <p> + <span class="bold"><strong>It is important to understand that + flushing cached information to disk only minimizes the + window of opportunity for corrupted data, it does not + eliminate the possibility.</strong></span> + </p> + <p> + While unlikely, it is possible for database corruption to + happen if a system or application crash occurs while writing + data to the database. To ensure that database corruption never + occurs, applications must either: + </p> <div class="itemizedlist"> <ul type="disc"> - <li>Use transactions and logging with automatic recovery.</li> - <li>Use logging and application-specific recovery.</li> - <li>Edit a copy of the database, and, once all applications -using the database have successfully called <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a>, use -system operations (for example, the POSIX rename system call) to -atomically replace the original database with the updated copy.</li> + <li> + Use transactions and logging with automatic + recovery. + </li> + <li> + Use logging and application-specific + recovery. + </li> + <li> + Edit a copy of the database, and, once all + applications using the database have successfully called + <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a>, use system operations (for example, the POSIX + rename system call) to atomically replace the original + database with the updated copy. + </li> </ul> </div> </div> @@ -72,7 +83,8 @@ atomically replace the original database with the updated copy.</li> <td width="40%" align="right"> <a accesskey="n" href="am_close.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Database verification and salvage </td> + <td width="40%" align="left" valign="top">Database verification and + salvage </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/am_truncate.html b/docs/programmer_reference/am_truncate.html index 2bb37475..96cc2c3d 100644 --- a/docs/programmer_reference/am_truncate.html +++ b/docs/programmer_reference/am_truncate.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_stat.html">Prev</a> </td> - <th width="60%" align="center">Chapter 3. - Access Method Operations - </th> + <th width="60%" align="center">Chapter 3. Access Method Operations </th> <td width="20%" align="right"> <a accesskey="n" href="am_upgrade.html">Next</a></td> </tr> </table> @@ -38,7 +36,10 @@ </div> </div> </div> - <p>The <a href="../api_reference/C/dbtruncate.html" class="olink">DB->truncate()</a> method empties a database of all records.</p> + <p> + The <a href="../api_reference/C/dbtruncate.html" class="olink">DB->truncate()</a> method empties a database of all + records. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/am_upgrade.html b/docs/programmer_reference/am_upgrade.html index 7366891f..9e08741f 100644 --- a/docs/programmer_reference/am_upgrade.html +++ b/docs/programmer_reference/am_upgrade.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_truncate.html">Prev</a> </td> - <th width="60%" align="center">Chapter 3. - Access Method Operations - </th> + <th width="60%" align="center">Chapter 3. Access Method Operations </th> <td width="20%" align="right"> <a accesskey="n" href="am_verify.html">Next</a></td> </tr> </table> @@ -38,41 +36,47 @@ </div> </div> </div> + <p> + When upgrading to a new release of Berkeley DB, it may be + necessary to upgrade the on-disk format of already-created + database files. <span class="bold"><strong>Berkeley DB database + upgrades are done in place, and so are potentially + destructive.</strong></span> This means that if the system + crashes during the upgrade procedure, or if the upgrade + procedure runs out of disk space, the databases may be left in + an inconsistent and unrecoverable state. To guard against + failure, the procedures outlined in the <a href="../upgrading/upgrade_process.html" class="olink">Upgrading Berkeley DB installations</a> chapter of + the Berkeley DB Installation and Build Guide should be carefully followed. If you are + not performing catastrophic archival as part of your + application upgrade process, you should at least copy your + database to archival media, verify that your archival media is + error-free and readable, and that copies of your backups are + stored offsite! + </p> <p> - - When upgrading to a new release of Berkeley DB, it may be necessary to - upgrade the on-disk format of already-created database files. - <span class="bold"><strong>Berkeley DB database upgrades are done in place, - and so are potentially destructive.</strong></span> This means that if - the system crashes during the upgrade - procedure, or if the upgrade procedure runs out of disk space, the - databases may be left in an inconsistent and unrecoverable state. To guard - against failure, the procedures outlined in the <a href="../upgrading/upgrade_process.html" class="olink">Upgrading Berkeley DB installations</a> chapter of the - Berkeley DB Installation and Build Guide should be carefully followed. If you are not performing - catastrophic archival as part of your application upgrade process, you - should at least copy your database to archival media, verify that your - archival media is error-free and readable, and that copies of your backups - are stored offsite! -</p> + The actual database upgrade is done using the <a href="../api_reference/C/dbupgrade.html" class="olink">DB->upgrade()</a> + method, or by dumping the database using the old version of + the Berkeley DB software and reloading it using the current + version. + </p> <p> - The actual database upgrade is done using the <a href="../api_reference/C/dbupgrade.html" class="olink">DB->upgrade()</a> method, or by - dumping the database using the old version of the Berkeley DB software - and reloading it using the current version.</p> - <p>After an - upgrade, Berkeley DB applications must be recompiled to use the new - Berkeley DB library before they can access an upgraded database. - <span class="bold"><strong>There is no guarantee that applications compiled - against previous releases of Berkeley DB will work correctly with - an upgraded database format. Nor is there any guarantee that - applications compiled against - newer releases of Berkeley DB will work correctly with the previous - database format.</strong></span> We do guarantee that any archived database may - be upgraded using a current Berkeley DB software release and the - <a href="../api_reference/C/dbupgrade.html" class="olink">DB->upgrade()</a> method, and there is no need to step-wise upgrade the database - using intermediate releases of Berkeley DB. Sites should consider - archiving appropriate copies of their application or application sources if - they may need to access archived databases without first upgrading them. -</p> + After an upgrade, Berkeley DB applications must be + recompiled to use the new Berkeley DB library before they can + access an upgraded database. <span class="bold"><strong>There is no + guarantee that applications compiled against previous + releases of Berkeley DB will work correctly with an + upgraded database format. Nor is there any guarantee that + applications compiled against newer releases of Berkeley + DB will work correctly with the previous database + format.</strong></span> We do guarantee that any archived + database may be upgraded using a current Berkeley DB software + release and the <a href="../api_reference/C/dbupgrade.html" class="olink">DB->upgrade()</a> method, and there is no need to + step-wise upgrade the database using intermediate releases of + Berkeley DB. Sites should consider archiving appropriate + copies of their application or application sources if they may + need to access archived databases without first upgrading + them. + </p> </div> <div class="navfooter"> <hr /> @@ -89,7 +93,8 @@ <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Database verification and salvage</td> + <td width="40%" align="right" valign="top"> Database verification and + salvage</td> </tr> </table> </div> diff --git a/docs/programmer_reference/am_verify.html b/docs/programmer_reference/am_verify.html index 4be61686..f3b1ced0 100644 --- a/docs/programmer_reference/am_verify.html +++ b/docs/programmer_reference/am_verify.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Database verification and salvage</th> + <th colspan="3" align="center">Database verification and + salvage</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_upgrade.html">Prev</a> </td> - <th width="60%" align="center">Chapter 3. - Access Method Operations - </th> + <th width="60%" align="center">Chapter 3. Access Method Operations </th> <td width="20%" align="right"> <a accesskey="n" href="am_sync.html">Next</a></td> </tr> </table> @@ -34,38 +33,51 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="am_verify"></a>Database verification and salvage</h2> + <h2 class="title" style="clear: both"><a id="am_verify"></a>Database verification and + salvage</h2> </div> </div> </div> - <p>The <a href="../api_reference/C/dbverify.html" class="olink">DB->verify()</a> method verifies that a file, and any databases it may -contain, are uncorrupted. In addition, the method may optionally be -called with a file stream argument to which all key/data pairs found in -the database are output. There are two modes for finding key/data pairs -to be output:</p> + <p> + The <a href="../api_reference/C/dbverify.html" class="olink">DB->verify()</a> method verifies that a file, and any + databases it may contain, are uncorrupted. In addition, the + method may optionally be called with a file stream argument to + which all key/data pairs found in the database are output. + There are two modes for finding key/data pairs to be + output: + </p> <div class="orderedlist"> <ol type="1"> - <li>If the <a href="../api_reference/C/dbverify.html#verify_DB_SALVAGE" class="olink">DB_SALVAGE</a> flag is specified, the key/data pairs in the -database are output. When run in this mode, the database is assumed to -be largely uncorrupted. For example, the <a href="../api_reference/C/dbverify.html" class="olink">DB->verify()</a> method will -search for pages that are no longer linked into the database, and will -output key/data pairs from such pages. However, key/data items that -have been marked as deleted in the database will not be output, as the -page structures are generally trusted in this mode.</li> - <li>If both the <a href="../api_reference/C/dbverify.html#verify_DB_SALVAGE" class="olink">DB_SALVAGE</a> and <a href="../api_reference/C/dbverify.html#verify_DB_AGGRESSIVE" class="olink">DB_AGGRESSIVE</a> flags are -specified, all possible key/data pairs are output. When run in this mode, -the database is assumed to be seriously corrupted. For example, key/data -pairs that have been deleted will re-appear in the output. In addition, -because pages may have been subsequently reused and modified during -normal database operations after the key/data pairs were deleted, it is -not uncommon for apparently corrupted key/data pairs to be output in this -mode, even when there is no corruption in the underlying database. The -output will almost always have to be edited by hand or other means before -the data is ready for reload into another database. We recommend that -<a href="../api_reference/C/dbverify.html#verify_DB_SALVAGE" class="olink">DB_SALVAGE</a> be tried first, and <a href="../api_reference/C/dbverify.html#verify_DB_AGGRESSIVE" class="olink">DB_AGGRESSIVE</a> only tried -if the output from that first attempt is obviously missing data items or -the data is sufficiently valuable that human review of the output is -preferable to any kind of data loss.</li> + <li> + If the <a href="../api_reference/C/dbverify.html#verify_DB_SALVAGE" class="olink">DB_SALVAGE</a> flag is specified, the key/data + pairs in the database are output. When run in this mode, + the database is assumed to be largely uncorrupted. For + example, the <a href="../api_reference/C/dbverify.html" class="olink">DB->verify()</a> method will search for pages that + are no longer linked into the database, and will output + key/data pairs from such pages. However, key/data items + that have been marked as deleted in the database will not + be output, as the page structures are generally trusted in + this mode. + </li> + <li> + If both the <a href="../api_reference/C/dbverify.html#verify_DB_SALVAGE" class="olink">DB_SALVAGE</a> and <a href="../api_reference/C/dbverify.html#verify_DB_AGGRESSIVE" class="olink">DB_AGGRESSIVE</a> flags + are specified, all possible key/data pairs are output. + When run in this mode, the database is assumed to be + seriously corrupted. For example, key/data pairs that have + been deleted will re-appear in the output. In addition, + because pages may have been subsequently reused and + modified during normal database operations after the + key/data pairs were deleted, it is not uncommon for + apparently corrupted key/data pairs to be output in this + mode, even when there is no corruption in the underlying + database. The output will almost always have to be edited + by hand or other means before the data is ready for reload + into another database. We recommend that <a href="../api_reference/C/dbverify.html#verify_DB_SALVAGE" class="olink">DB_SALVAGE</a> be + tried first, and <a href="../api_reference/C/dbverify.html#verify_DB_AGGRESSIVE" class="olink">DB_AGGRESSIVE</a> only tried if the output + from that first attempt is obviously missing data items or + the data is sufficiently valuable that human review of the + output is preferable to any kind of data loss. + </li> </ol> </div> </div> diff --git a/docs/programmer_reference/apprec.html b/docs/programmer_reference/apprec.html index a74c3f77..4792b862 100644 --- a/docs/programmer_reference/apprec.html +++ b/docs/programmer_reference/apprec.html @@ -14,13 +14,12 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 14. - Application Specific Logging and Recovery - </th> + <th colspan="3" align="center">Chapter 14. Application Specific Logging and + Recovery </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="xa_faq.html">Prev</a> </td> @@ -34,9 +33,8 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="apprec"></a>Chapter 14. - Application Specific Logging and Recovery - </h2> + <h2 class="title"><a id="apprec"></a>Chapter 14. Application Specific Logging and + Recovery </h2> </div> </div> </div> @@ -47,17 +45,20 @@ <dl> <dt> <span class="sect1"> - <a href="apprec.html#apprec_intro">Introduction to application specific logging and recovery</a> + <a href="apprec.html#apprec_intro">Introduction to application + specific logging and recovery</a> </span> </dt> <dt> <span class="sect1"> - <a href="apprec_def.html">Defining application-specific log records</a> + <a href="apprec_def.html">Defining application-specific log + records</a> </span> </dt> <dt> <span class="sect1"> - <a href="apprec_auto.html">Automatically generated functions</a> + <a href="apprec_auto.html">Automatically generated + functions</a> </span> </dt> <dt> @@ -71,69 +72,98 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="apprec_intro"></a>Introduction to application specific logging and recovery</h2> + <h2 class="title" style="clear: both"><a id="apprec_intro"></a>Introduction to application + specific logging and recovery</h2> </div> </div> </div> - <p>It is possible to use the Locking, Logging and Transaction subsystems -of Berkeley DB to provide transaction semantics on objects other than those -described by the Berkeley DB access methods. In these cases, the application -will need application-specific logging and recovery functions.</p> - <p>For example, consider an application that provides transaction semantics -on data stored in plain text files accessed using the POSIX read and -write system calls. The read and write operations for which transaction -protection is desired will be bracketed by calls to the standard Berkeley DB -transactional interfaces, <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a> and <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a>, and -the transaction's locker ID will be used to acquire relevant read and -write locks.</p> - <p>Before data is accessed, the application must make a call to the lock -manager, <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a>, for a lock of the appropriate type (for -example, read) on the object being locked. The object might be a page -in the file, a byte, a range of bytes, or some key. It is up to the -application to ensure that appropriate locks are acquired. Before a -write is performed, the application should acquire a write lock on the -object by making an appropriate call to the lock manager, -<a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a>. Then, the application should make a call to the log -manager, via the automatically-generated log-writing function described -as follows. This record should contain enough information to redo the -operation in case of failure after commit and to undo the operation in -case of abort.</p> - <p>When designing applications that will use the log subsystem, it is -important to remember that the application is responsible for providing -any necessary structure to the log record. For example, the application -must understand what part of the log record is an operation code, what -part identifies the file being modified, what part is redo information, -and what part is undo information.</p> - <p>After the log message is written, the application may issue the write -system call. After all requests are issued, the application may call -<a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a>. When <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a> returns, the caller is -guaranteed that all necessary log writes have been written to disk.</p> - <p>At any time before issuing a <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a>, the application may -call <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a>, which will result in restoration of the database -to a consistent pretransaction state. (The application may specify its -own recovery function for this purpose using the -<a href="../api_reference/C/envset_app_dispatch.html" class="olink">DB_ENV->set_app_dispatch()</a> method. The recovery function must be able to -either reapply or undo the update depending on the context, for each -different type of log record. The recovery functions must not use Berkeley DB -methods to access data in the environment as there is no way to -coordinate these accesses with either the aborting transaction or the -updates done by recovery or replication.)</p> - <p>If the application crashes, the recovery process uses the log to restore -the database to a consistent state.</p> - <p>Berkeley DB includes tools to assist in the development of application-specific -logging and recovery. Specifically, given a description of information -to be logged in a family of log records, these tools will automatically -create log-writing functions (functions that marshall their arguments -into a single log record), log-reading functions (functions that read -a log record and unmarshall it into a structure containing fields that -map into the arguments written to the log), log-printing functions -(functions that print the contents of a log record for debugging), and -templates for recovery functions (functions that review log records -during transaction abort or recovery). The tools and generated code -are C-language and POSIX-system based, but the generated code should be -usable on any system, not just POSIX systems.</p> - <p>A sample application that does application-specific recovery is included -in the Berkeley DB distribution, in the directory <code class="filename">examples_c/ex_apprec</code>.</p> + <p> + It is possible to use the Locking, Logging and Transaction + subsystems of Berkeley DB to provide transaction semantics on + objects other than those described by the Berkeley DB access + methods. In these cases, the application will need + application-specific logging and recovery functions. + </p> + <p> + For example, consider an application that provides + transaction semantics on data stored in plain text files + accessed using the POSIX read and write system calls. The read + and write operations for which transaction protection is + desired will be bracketed by calls to the standard Berkeley DB + transactional interfaces, <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a> and <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a>, and the + transaction's locker ID will be used to acquire relevant read + and write locks. + </p> + <p> + Before data is accessed, the application must make a call to + the lock manager, <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a>, for a lock of the appropriate + type (for example, read) on the object being locked. The + object might be a page in the file, a byte, a range of bytes, + or some key. It is up to the application to ensure that + appropriate locks are acquired. Before a write is performed, + the application should acquire a write lock on the object by + making an appropriate call to the lock manager, <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a>. + Then, the application should make a call to the log manager, + via the automatically-generated log-writing function described + as follows. This record should contain enough information to + redo the operation in case of failure after commit and to undo + the operation in case of abort. + </p> + <p> + When designing applications that will use the log subsystem, + it is important to remember that the application is + responsible for providing any necessary structure to the log + record. For example, the application must understand what part + of the log record is an operation code, what part identifies + the file being modified, what part is redo information, and + what part is undo information. + </p> + <p> + After the log message is written, the application may issue + the write system call. After all requests are issued, the + application may call <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a>. When <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a> returns, + the caller is guaranteed that all necessary log writes have + been written to disk. + </p> + <p> + At any time before issuing a <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a>, the application + may call <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a>, which will result in restoration of the + database to a consistent pretransaction state. (The + application may specify its own recovery function for this + purpose using the <a href="../api_reference/C/envset_app_dispatch.html" class="olink">DB_ENV->set_app_dispatch()</a> method. The recovery + function must be able to either reapply or undo the update + depending on the context, for each different type of log + record. The recovery functions must not use Berkeley DB + methods to access data in the environment as there is no way + to coordinate these accesses with either the aborting + transaction or the updates done by recovery or + replication.) + </p> + <p> + If the application crashes, the recovery process uses the + log to restore the database to a consistent state. + </p> + <p> + Berkeley DB includes tools to assist in the development of + application-specific logging and recovery. Specifically, given + a description of information to be logged in a family of log + records, these tools will automatically create log-writing + functions (functions that marshall their arguments into a + single log record), log-reading functions (functions that read + a log record and unmarshall it into a structure containing + fields that map into the arguments written to the log), + log-printing functions (functions that print the contents of a + log record for debugging), and templates for recovery + functions (functions that review log records during + transaction abort or recovery). The tools and generated code + are C-language and POSIX-system based, but the generated code + should be usable on any system, not just POSIX systems. + </p> + <p> + A sample application that does application-specific recovery + is included in the Berkeley DB distribution, in the directory + <code class="filename">examples/c/ex_apprec</code>. + </p> </div> </div> <div class="navfooter"> @@ -149,7 +179,8 @@ in the Berkeley DB distribution, in the directory <code class="filename">example <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Defining application-specific log records</td> + <td width="40%" align="right" valign="top"> Defining application-specific log + records</td> </tr> </table> </div> diff --git a/docs/programmer_reference/apprec_auto.html b/docs/programmer_reference/apprec_auto.html index 3f7d009e..12aa26eb 100644 --- a/docs/programmer_reference/apprec_auto.html +++ b/docs/programmer_reference/apprec_auto.html @@ -14,17 +14,17 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Automatically generated functions</th> + <th colspan="3" align="center">Automatically generated + functions</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="apprec_def.html">Prev</a> </td> - <th width="60%" align="center">Chapter 14. - Application Specific Logging and Recovery - </th> + <th width="60%" align="center">Chapter 14. Application Specific Logging and + Recovery </th> <td width="20%" align="right"> <a accesskey="n" href="apprec_config.html">Next</a></td> </tr> </table> @@ -34,41 +34,54 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="apprec_auto"></a>Automatically generated functions</h2> + <h2 class="title" style="clear: both"><a id="apprec_auto"></a>Automatically generated + functions</h2> </div> </div> </div> - <p>The XXX.src file is processed using the gen_rec.awk script included in -the dist directory of the Berkeley DB distribution. This is an awk script -that is executed from with the following command line:</p> + <p> + The XXX.src file is processed using the gen_rec.awk script + included in the dist directory of the Berkeley DB + distribution. This is an awk script that is executed from with + the following command line: + </p> <pre class="programlisting">awk -f gen_rec.awk \ - -v source_file=<span class="emphasis"><em>C_FILE</em></span> \ - -v header_file=<span class="emphasis"><em>H_FILE</em></span> \ - -v print_file=<span class="emphasis"><em>P_FILE</em></span> \ - -v template_file=<span class="emphasis"><em>TMP_FILE</em></span> < XXX.src</pre> - <p>where <span class="emphasis"><em>C_FILE</em></span> is the name of the file into which to place the -automatically generated C code, <span class="emphasis"><em>H_FILE</em></span> is the name of the -file into which to place the automatically generated data structures -and declarations, <span class="emphasis"><em>P_FILE</em></span> is the name of the file into which to -place the automatically generated C code that prints log records, -and <span class="emphasis"><em>TMP_FILE</em></span> is the name of the file into -which to place a template for the recovery routines.</p> - <p>Because the gen_rec.awk script uses sources files located relative to -the Berkeley DB dist directory, it must be run from the dist directory. For -example, in building the Berkeley DB logging and recovery routines for -ex_apprec, the following script is used to rebuild the automatically -generated files:</p> - <pre class="programlisting">E=../examples_c/ex_apprec -<p></p> + -v source_file=<span class="emphasis"><em>C_FILE</em></span> \ + -v header_file=<span class="emphasis"><em>H_FILE</em></span> \ + -v print_file=<span class="emphasis"><em>P_FILE</em></span> \ + -v template_file=<span class="emphasis"><em>TMP_FILE</em></span> < XXX.src</pre> + <p> + where <span class="emphasis"><em>C_FILE</em></span> is the name of the file + into which to place the automatically generated C code, + <span class="emphasis"><em>H_FILE</em></span> is the name of the file into + which to place the automatically generated data structures and + declarations, <span class="emphasis"><em>P_FILE</em></span> is the name of the + file into which to place the automatically generated C code + that prints log records, and <span class="emphasis"><em>TMP_FILE</em></span> is + the name of the file into which to place a template for the + recovery routines. + </p> + <p> + Because the gen_rec.awk script uses sources files located + relative to the Berkeley DB dist directory, it must be run + from the dist directory. For example, in building the Berkeley + DB logging and recovery routines for ex_apprec, the following + script is used to rebuild the automatically generated + files: + </p> + <pre class="programlisting">E=../examples/c/ex_apprec + cd ../../dist awk -f gen_rec.awk \ -v source_file=$E/ex_apprec_auto.c \ -v header_file=$E/ex_apprec_auto.h \ -v print_file=$E/ex_apprec_autop.c \ -v template_file=$E/ex_apprec_template < $E/ex_apprec.src</pre> - <p>For each log record description found in the XXX.src file, the following -structure declarations and #defines will be created in the file -<span class="emphasis"><em>header_file</em></span>:</p> + <p> + For each log record description found in the XXX.src file, + the following structure declarations and #defines will be + created in the file <span class="emphasis"><em>header_file</em></span>: + </p> <pre class="programlisting">#define DB_PREFIX_RECORD_TYPE /* Integer ID number */ typedef struct _PREFIX_RECORD_TYPE_args { @@ -94,26 +107,36 @@ typedef struct _PREFIX_RECORD_TYPE_args { * the entries in the record statement. */ };</pre> - <p>Thus, the auto-generated ex_apprec_mkdir_args structure looks as follows:</p> + <p> + Thus, the auto-generated ex_apprec_mkdir_args structure + looks as follows: + </p> <pre class="programlisting">typedef struct _ex_apprec_mkdir_args { - u_int32_t type; - DB_TXN *txnid; - DB_LSN prev_lsn; - DBT dirname; + u_int32_t type; + DB_TXN *txnid; + DB_LSN prev_lsn; + DBT dirname; } ex_apprec_mkdir_args;</pre> - <p>The template_file will contain a template for a recovery function. The -recovery function is called on each record read from the log during -system recovery, transaction abort, or the application of log records -on a replication client, and is expected to redo or undo the operations -described by that record. The details of the recovery function will be -specific to the record being logged and need to be written manually, but -the template provides a good starting point. (See ex_apprec_template -and ex_apprec_rec.c for an example of both the template produced and the -resulting recovery function.)</p> - <p>The template file should be copied to a source file in the application -(but not the automatically generated source_file, as that will get -overwritten each time gen_rec.awk is run) and fully developed there. -The recovery function takes the following parameters:</p> + <p> + The template_file will contain a template for a recovery + function. The recovery function is called on each record read + from the log during system recovery, transaction abort, or the + application of log records on a replication client, and is + expected to redo or undo the operations described by that + record. The details of the recovery function will be specific + to the record being logged and need to be written manually, + but the template provides a good starting point. (See + ex_apprec_template and ex_apprec_rec.c for an example of both + the template produced and the resulting recovery + function.) + </p> + <p> + The template file should be copied to a source file in the + application (but not the automatically generated source_file, + as that will get overwritten each time gen_rec.awk is run) and + fully developed there. The recovery function takes the + following parameters: + </p> <div class="blockquote"> <blockquote class="blockquote"> <div class="variablelist"> @@ -121,38 +144,56 @@ The recovery function takes the following parameters:</p> <dt> <span class="term">dbenv</span> </dt> - <dd>The environment in which recovery is running.</dd> + <dd> + The environment in which recovery is + running. + </dd> <dt> <span class="term">rec</span> </dt> - <dd>The record being recovered.</dd> + <dd> + The record being recovered. + </dd> <dt> <span class="term">lsn</span> </dt> - <dd>The log sequence number of the record being recovered. The -prev_lsn field, automatically included in every auto-generated log -record, should be returned through this argument. The prev_lsn field -is used to chain log records together to allow transaction aborts; -because the recovery function is the only place that a log record gets -parsed, the responsibility for returning this value lies with the -recovery function writer.</dd> + <dd> + The log sequence number of the record being + recovered. The prev_lsn field, automatically + included in every auto-generated log record, + should be returned through this argument. The + prev_lsn field is used to chain log records + together to allow transaction aborts; because the + recovery function is the only place that a log + record gets parsed, the responsibility for + returning this value lies with the recovery + function writer. + </dd> <dt> <span class="term">op</span> </dt> - <dd>A parameter of type db_recops, which indicates what operation is being -run (<a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_ABORT" class="olink">DB_TXN_ABORT</a>, <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_APPLY" class="olink">DB_TXN_APPLY</a>, <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_BACKWARD_ROLL" class="olink">DB_TXN_BACKWARD_ROLL</a>, -<a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_FORWARD_ROLL" class="olink">DB_TXN_FORWARD_ROLL</a> or <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_PRINT" class="olink">DB_TXN_PRINT</a>). -</dd> + <dd> + A parameter of type db_recops, which + indicates what operation is being run + (<a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_ABORT" class="olink">DB_TXN_ABORT</a>, <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_APPLY" class="olink">DB_TXN_APPLY</a>, + <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_BACKWARD_ROLL" class="olink">DB_TXN_BACKWARD_ROLL</a>, <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_FORWARD_ROLL" class="olink">DB_TXN_FORWARD_ROLL</a> or + <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_PRINT" class="olink">DB_TXN_PRINT</a>). + </dd> </dl> </div> </blockquote> </div> - <p>In addition to the header_file and template_file, a source_file is -created, containing a log, read, recovery, and print function for each -record type.</p> - <p>The log function marshalls the parameters into a buffer, and calls -<a href="../api_reference/C/logput.html" class="olink">DB_ENV->log_put()</a> on that buffer returning 0 on success and non-zero on -failure. The log function takes the following parameters:</p> + <p> + In addition to the header_file and template_file, a + source_file is created, containing a log, read, recovery, and + print function for each record type. + </p> + <p> + The log function marshalls the parameters into a buffer, and + calls <a href="../api_reference/C/logput.html" class="olink">DB_ENV->log_put()</a> on that buffer returning 0 on success and + non-zero on failure. The log function takes the following + parameters: + </p> <div class="blockquote"> <blockquote class="blockquote"> <div class="variablelist"> @@ -160,36 +201,52 @@ failure. The log function takes the following parameters:</p> <dt> <span class="term">dbenv</span> </dt> - <dd>The environment in which recovery is running.</dd> + <dd>The environment in which recovery is + running.</dd> <dt> <span class="term">txnid</span> </dt> - <dd>The transaction identifier for the transaction handle returned by -<a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a>.</dd> + <dd> + The transaction identifier for the + transaction handle returned by + <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a>. + </dd> <dt> <span class="term">lsnp</span> </dt> - <dd>A pointer to storage for a log sequence number into which the log -sequence number of the new log record will be returned.</dd> + <dd> + A pointer to storage for a log sequence + number into which the log sequence number of the + new log record will be returned. + </dd> <dt> <span class="term">syncflag</span> </dt> - <dd>A flag indicating whether the record must be written synchronously. -Valid values are 0 and <a href="../api_reference/C/logput.html#put_DB_FLUSH" class="olink">DB_FLUSH</a>.</dd> + <dd> + A flag indicating whether the record must be + written synchronously. Valid values are 0 and + <a href="../api_reference/C/logput.html#put_DB_FLUSH" class="olink">DB_FLUSH</a>. + </dd> <dt> <span class="term">args</span> </dt> - <dd>The remaining parameters to the log message are the fields described -in the XXX.src file, in order.</dd> + <dd> + The remaining parameters to the log message + are the fields described in the XXX.src file, in + order. + </dd> </dl> </div> </blockquote> </div> - <p>The read function takes a buffer and unmarshalls its contents into a -structure of the appropriate type. It returns 0 on success and non-zero -on error. After the fields of the structure have been used, the pointer -returned from the read function should be freed. The read function -takes the following parameters:</p> + <p> + The read function takes a buffer and unmarshalls its + contents into a structure of the appropriate type. It returns + 0 on success and non-zero on error. After the fields of the + structure have been used, the pointer returned from the read + function should be freed. The read function takes the + following parameters: + </p> <div class="blockquote"> <blockquote class="blockquote"> <div class="variablelist"> @@ -197,25 +254,36 @@ takes the following parameters:</p> <dt> <span class="term">dbenv</span> </dt> - <dd>The environment in which recovery is running.</dd> + <dd> + The environment in which recovery is + running. + </dd> <dt> <span class="term">recbuf</span> </dt> - <dd>A buffer.</dd> + <dd> + A buffer. + </dd> <dt> <span class="term">argp</span> </dt> - <dd>A pointer to a structure of the appropriate type.</dd> + <dd> + A pointer to a structure of the appropriate + type. + </dd> </dl> </div> </blockquote> </div> - <p>The print function displays the contents of the record. The print -function takes the same parameters as the recovery function described -previously. Although some of the parameters are unused by the print -function, taking the same parameters allows a single dispatch loop to -dispatch to a variety of functions. The print function takes the -following parameters:</p> + <p> + The print function displays the contents of the record. The + print function takes the same parameters as the recovery + function described previously. Although some of the parameters + are unused by the print function, taking the same parameters + allows a single dispatch loop to dispatch to a variety of + functions. The print function takes the following + parameters: + </p> <div class="blockquote"> <blockquote class="blockquote"> <div class="variablelist"> @@ -223,27 +291,40 @@ following parameters:</p> <dt> <span class="term">dbenv</span> </dt> - <dd>The environment in which recovery is running.</dd> + <dd> + The environment in which recovery is + running. + </dd> <dt> <span class="term">rec</span> </dt> - <dd>The record being recovered.</dd> + <dd> + The record being recovered. + </dd> <dt> <span class="term">lsn</span> </dt> - <dd>The log sequence number of the record being recovered.</dd> + <dd> + The log sequence number of the record being + recovered. + </dd> <dt> <span class="term">op</span> </dt> - <dd>Unused.</dd> + <dd> + Unused. + </dd> </dl> </div> </blockquote> </div> - <p>Finally, the source file will contain a function (named XXX_init_print, -where XXX is replaced by the prefix) which should be added to the -initialization part of the standalone <a href="../api_reference/C/db_printlog.html" class="olink">db_printlog</a> utility code -so that utility can be used to display application-specific log records.</p> + <p> + Finally, the source file will contain a function (named + XXX_init_print, where XXX is replaced by the prefix) which + should be added to the initialization part of the standalone + <a href="../api_reference/C/db_printlog.html" class="olink">db_printlog</a> utility code so that utility can be used to display + application-specific log records. + </p> </div> <div class="navfooter"> <hr /> @@ -256,7 +337,8 @@ so that utility can be used to display application-specific log records.</p> <td width="40%" align="right"> <a accesskey="n" href="apprec_config.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Defining application-specific log records </td> + <td width="40%" align="left" valign="top">Defining application-specific log + records </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/apprec_config.html b/docs/programmer_reference/apprec_config.html index 444c484a..cfac265c 100644 --- a/docs/programmer_reference/apprec_config.html +++ b/docs/programmer_reference/apprec_config.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,8 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="apprec_auto.html">Prev</a> </td> - <th width="60%" align="center">Chapter 14. - Application Specific Logging and Recovery - </th> + <th width="60%" align="center">Chapter 14. Application Specific Logging and + Recovery </th> <td width="20%" align="right"> <a accesskey="n" href="program.html">Next</a></td> </tr> </table> @@ -38,115 +37,167 @@ </div> </div> </div> - <p>The application should include a dispatch function that dispatches to -appropriate printing and/or recovery functions based on the log record -type and the operation code. The dispatch function should take the same -arguments as the recovery function, and should call the appropriate -recovery and/or printing functions based on the log record type and the -operation code. For example, the ex_apprec dispatch function is as -follows:</p> + <p> + The application should include a dispatch function that + dispatches to appropriate printing and/or recovery functions + based on the log record type and the operation code. The + dispatch function should take the same arguments as the + recovery function, and should call the appropriate recovery + and/or printing functions based on the log record type and the + operation code. For example, the ex_apprec dispatch function + is as follows: + </p> <pre class="programlisting">int apprec_dispatch(dbenv, dbt, lsn, op) - DB_ENV *dbenv; - DBT *dbt; - DB_LSN *lsn; - db_recops op; + DB_ENV *dbenv; + DBT *dbt; + DB_LSN *lsn; + db_recops op; { - u_int32_t rectype; - /* Pull the record type out of the log record. */ - memcpy(&rectype, dbt->data, sizeof(rectype)); - switch (rectype) { - case DB_ex_apprec_mkdir: - return (ex_apprec_mkdir_recover(dbenv, dbt, lsn, op)); - default: - /* - * We've hit an unexpected, allegedly user-defined record - * type. - */ - dbenv->errx(dbenv, "Unexpected log record type encountered"); - return (EINVAL); - } + u_int32_t rectype; + /* Pull the record type out of the log record. */ + memcpy(&rectype, dbt->data, sizeof(rectype)); + switch (rectype) { + case DB_ex_apprec_mkdir: + return (ex_apprec_mkdir_recover(dbenv, dbt, lsn, op)); + default: + /* + * We've hit an unexpected, allegedly user-defined record + * type. + */ + dbenv->errx(dbenv, "Unexpected log record type encountered"); + return (EINVAL); + } }</pre> - <p>Applications use this dispatch function and the automatically generated -functions as follows:</p> + <p> + Applications use this dispatch function and the + automatically generated functions as follows: + </p> <div class="orderedlist"> <ol type="1"> - <li>When the application starts, call the <a href="../api_reference/C/envset_app_dispatch.html" class="olink">DB_ENV->set_app_dispatch()</a> -with your dispatch function.</li> - <li>Issue a <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a> call before any operations you want to be -transaction-protected.</li> - <li>Before accessing any data, issue the appropriate lock call to lock the -data (either for reading or writing).</li> - <li>Before modifying any data that is transaction-protected, issue a call -to the appropriate log function.</li> - <li>Call <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a> -to cancel all of the modifications.</li> + <li> + When the application starts, call the + <a href="../api_reference/C/envset_app_dispatch.html" class="olink">DB_ENV->set_app_dispatch()</a> with your dispatch + function. + </li> + <li> + Issue a <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a> call before any operations you + want to be transaction-protected. + </li> + <li> + Before accessing any data, issue the appropriate + lock call to lock the data (either for reading or + writing). + </li> + <li> + Before modifying any data that is + transaction-protected, issue a call to the appropriate log + function. + </li> + <li> + Call <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a> to cancel all of the + modifications. + </li> </ol> </div> - <p>The recovery functions are called in the three following cases:</p> + <p> + The recovery functions are called in the three following + cases: + </p> <div class="orderedlist"> <ol type="1"> - <li>During recovery after application or system failure, with op set to -<a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_FORWARD_ROLL" class="olink">DB_TXN_FORWARD_ROLL</a> or <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_BACKWARD_ROLL" class="olink">DB_TXN_BACKWARD_ROLL</a>.</li> - <li>During transaction abort, with op set to <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_ABORT" class="olink">DB_TXN_ABORT</a>.</li> - <li>On a replicated client to apply updates from the master, with op set to -<a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_APPLY" class="olink">DB_TXN_APPLY</a>.</li> + <li> + During recovery after application or system failure, + with op set to <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_FORWARD_ROLL" class="olink">DB_TXN_FORWARD_ROLL</a> or + <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_BACKWARD_ROLL" class="olink">DB_TXN_BACKWARD_ROLL</a>. + </li> + <li> + During transaction abort, with op set to + <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_ABORT" class="olink">DB_TXN_ABORT</a>. + </li> + <li> + On a replicated client to apply updates from the + master, with op set to <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_APPLY" class="olink">DB_TXN_APPLY</a>. + </li> </ol> </div> - <p>For each log record type you declare, you must write the appropriate -function to undo and redo the modifications. The shell of these -functions will be generated for you automatically, but you must fill in -the details.</p> - <p>Your code must be able to detect whether the described modifications -have been applied to the data. The function will be called with the -"op" parameter set to <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_ABORT" class="olink">DB_TXN_ABORT</a> when a transaction that wrote -the log record aborts, with <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_FORWARD_ROLL" class="olink">DB_TXN_FORWARD_ROLL</a> and -<a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_BACKWARD_ROLL" class="olink">DB_TXN_BACKWARD_ROLL</a> during recovery, and with <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_APPLY" class="olink">DB_TXN_APPLY</a> -on a replicated client.</p> - <p>The actions for <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_ABORT" class="olink">DB_TXN_ABORT</a> and <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_BACKWARD_ROLL" class="olink">DB_TXN_BACKWARD_ROLL</a> -should generally be the same, and the actions for -<a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_FORWARD_ROLL" class="olink">DB_TXN_FORWARD_ROLL</a> and <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_APPLY" class="olink">DB_TXN_APPLY</a> should generally -be the same. However, if the application is using Berkeley DB replication -and another thread of control may be performing read operations while -log records are applied on a replication client, the recovery function -should perform appropriate locking during <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_APPLY" class="olink">DB_TXN_APPLY</a> -operations. In this case, the recovery function may encounter deadlocks -when issuing locking calls. The application should run with the -deadlock detector, and the recovery function should simply return -<a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a> if a deadlock is detected and a locking -operation fails with that error.</p> - <p>The <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_PRINT" class="olink">DB_TXN_PRINT</a> operation should print the log record, -typically using the auto-generated print function; it is not used in -the Berkeley DB library, but may be useful for debugging, as in the -<a href="../api_reference/C/db_printlog.html" class="olink">db_printlog</a> utility. Applications may safely ignore this -operation code, they may handle printing from the recovery function, or -they may dispatch directly to the auto-generated print function.</p> - <p>One common way to determine whether operations need to be undone or -redone is the use of log sequence numbers (LSNs). For example, each -access method database page contains the LSN of the most recent log -record that describes a modification to the page. When the access -method changes a page, it writes a log record describing the change and -including the LSN that was on the page before the change. This LSN is -referred to as the previous LSN. The recovery functions read the page -described by a log record, and compare the LSN on the page to the LSN -they were passed.</p> - <p>If the page LSN is less than the passed LSN and the operation is an -undo, no action is necessary (because the modifications have not been -written to the page). If the page LSN is the same as the previous LSN -and the operation is a redo, the actions described are reapplied to the -page. If the page LSN is equal to the passed LSN and the operation is -an undo, the actions are removed from the page; if the page LSN is -greater than the passed LSN and the operation is a redo, no further -action is necessary. If the action is a redo and the LSN on the page -is less than the previous LSN in the log record, it is an error because -it could happen only if some previous log record was not processed.</p> - <p>Examples of other recovery functions can be found in the Berkeley DB library -recovery functions (found in files named XXX_rec.c) and in the -application-specific recovery example (specifically, ex_apprec_rec.c).</p> - <p>Finally, applications need to ensure that any data modifications they -have made, that were part of a committed transaction, must be written -to stable storage before calling the <a href="../api_reference/C/txncheckpoint.html" class="olink">DB_ENV->txn_checkpoint()</a> method. This is -to allow the periodic removal of database environment log files.</p> + <p> + For each log record type you declare, you must write the + appropriate function to undo and redo the modifications. The + shell of these functions will be generated for you + automatically, but you must fill in the details. + </p> + <p> + Your code must be able to detect whether the described + modifications have been applied to the data. The function will + be called with the "op" parameter set to <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_ABORT" class="olink">DB_TXN_ABORT</a> when a + transaction that wrote the log record aborts, with + <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_FORWARD_ROLL" class="olink">DB_TXN_FORWARD_ROLL</a> and <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_BACKWARD_ROLL" class="olink">DB_TXN_BACKWARD_ROLL</a> during + recovery, and with <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_APPLY" class="olink">DB_TXN_APPLY</a> on a replicated + client. + </p> + <p> + The actions for <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_ABORT" class="olink">DB_TXN_ABORT</a> and <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_BACKWARD_ROLL" class="olink">DB_TXN_BACKWARD_ROLL</a> + should generally be the same, and the actions for + <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_FORWARD_ROLL" class="olink">DB_TXN_FORWARD_ROLL</a> and <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_APPLY" class="olink">DB_TXN_APPLY</a> should generally be + the same. However, if the application is using Berkeley DB + replication and another thread of control may be performing + read operations while log records are applied on a replication + client, the recovery function should perform appropriate + locking during <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_APPLY" class="olink">DB_TXN_APPLY</a> operations. In this case, the + recovery function may encounter deadlocks when issuing locking + calls. The application should run with the deadlock detector, + and the recovery function should simply return <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a> + if a deadlock is detected and a locking operation fails with that error. + </p> + <p> + The <a href="../api_reference/C/envset_app_dispatch.html#set_app_dispatch_DB_TXN_PRINT" class="olink">DB_TXN_PRINT</a> operation should print the log record, + typically using the auto-generated print function; it is not + used in the Berkeley DB library, but may be useful for + debugging, as in the <a href="../api_reference/C/db_printlog.html" class="olink">db_printlog</a> utility. Applications may safely + ignore this operation code, they may handle printing from the + recovery function, or they may dispatch directly to the + auto-generated print function. + </p> + <p> + One common way to determine whether operations need to be + undone or redone is the use of log sequence numbers (LSNs). + For example, each access method database page contains the LSN + of the most recent log record that describes a modification to + the page. When the access method changes a page, it writes a + log record describing the change and including the LSN that + was on the page before the change. This LSN is referred to as + the previous LSN. The recovery functions read the page + described by a log record, and compare the LSN on the page to + the LSN they were passed. + </p> + <p> + If the page LSN is less than the passed LSN and the + operation is an undo, no action is necessary (because the + modifications have not been written to the page). If the page + LSN is the same as the previous LSN and the operation is a + redo, the actions described are reapplied to the page. If the + page LSN is equal to the passed LSN and the operation is an + undo, the actions are removed from the page; if the page LSN + is greater than the passed LSN and the operation is a redo, no + further action is necessary. If the action is a redo and the + LSN on the page is less than the previous LSN in the log + record, it is an error because it could happen only if some + previous log record was not processed. + </p> + <p> + Examples of other recovery functions can be found in the + Berkeley DB library recovery functions (found in files named + XXX_rec.c) and in the application-specific recovery example + (specifically, ex_apprec_rec.c). + </p> + <p> + Finally, applications need to ensure that any data + modifications they have made, that were part of a committed + transaction, must be written to stable storage before calling + the <a href="../api_reference/C/txncheckpoint.html" class="olink">DB_ENV->txn_checkpoint()</a> method. This is to allow the periodic + removal of database environment log files. + </p> </div> <div class="navfooter"> <hr /> @@ -159,13 +210,12 @@ to allow the periodic removal of database environment log files.</p> <td width="40%" align="right"> <a accesskey="n" href="program.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Automatically generated functions </td> + <td width="40%" align="left" valign="top">Automatically generated + functions </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 15. - Programmer Notes - </td> + <td width="40%" align="right" valign="top"> Chapter 15. Programmer Notes </td> </tr> </table> </div> diff --git a/docs/programmer_reference/apprec_def.html b/docs/programmer_reference/apprec_def.html index 9de018a8..db90da18 100644 --- a/docs/programmer_reference/apprec_def.html +++ b/docs/programmer_reference/apprec_def.html @@ -14,17 +14,17 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Defining application-specific log records</th> + <th colspan="3" align="center">Defining application-specific log + records</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="apprec.html">Prev</a> </td> - <th width="60%" align="center">Chapter 14. - Application Specific Logging and Recovery - </th> + <th width="60%" align="center">Chapter 14. Application Specific Logging and + Recovery </th> <td width="20%" align="right"> <a accesskey="n" href="apprec_auto.html">Next</a></td> </tr> </table> @@ -34,95 +34,156 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="apprec_def"></a>Defining application-specific log records</h2> + <h2 class="title" style="clear: both"><a id="apprec_def"></a>Defining application-specific log + records</h2> </div> </div> </div> - <p>By convention, log records are described in files named <code class="filename">XXX.src</code>, -where "XXX" is typically a descriptive name for a subsystem or other -logical group of logging functions. These files contain interface -definition language descriptions for each type of log record that is -used by the subsystem.</p> - <p>All blank lines and lines beginning with a hash ("#") character in -the XXX.src files are ignored.</p> - <p>The first non-comment line in the file should begin with the keyword -PREFIX, followed by a string that will be prepended to every generated -function name. Frequently, the PREFIX is either identical or similar -to the name of the <code class="filename">XXX.src</code> file. For example, the Berkeley DB -application-specific recovery example uses the file -<code class="filename">ex_apprec.src</code>, which begins with the following PREFIX line:</p> + <p> + By convention, log records are described in files named + <code class="filename">XXX.src</code>, where "XXX" is typically a + descriptive name for a subsystem or other logical group of + logging functions. These files contain interface definition + language descriptions for each type of log record that is used + by the subsystem. + </p> + <p> + All blank lines and lines beginning with a hash ("#") + character in the XXX.src files are ignored. + </p> + <p> + The first non-comment line in the file should begin with the + keyword PREFIX, followed by a string that will be prepended to + every generated function name. Frequently, the PREFIX is + either identical or similar to the name of the + <code class="filename">XXX.src</code> file. For example, the + Berkeley DB application-specific recovery example uses the + file <code class="filename">ex_apprec.src</code>, which begins with the + following PREFIX line: + </p> <pre class="programlisting">PREFIX ex_apprec</pre> - <p>Following the PREFIX line are the include files required by the -automatically generated functions. The include files should be listed -in order, prefixed by the keyword INCLUDE. For example, the Berkeley DB -application-specific recovery example lists the following include -files:</p> + <p> + Following the PREFIX line are the include files required by + the automatically generated functions. The include files + should be listed in order, prefixed by the keyword INCLUDE. + For example, the Berkeley DB application-specific recovery + example lists the following include files: + </p> <pre class="programlisting">INCLUDE #include "ex_apprec.h"</pre> - <p>The rest of the XXX.src file consists of log record descriptions. Each -log record description begins with one of the following lines:</p> + <p> + The rest of the XXX.src file consists of log record + descriptions. Each log record description begins with one of + the following lines: + </p> <pre class="programlisting">BEGIN <span class="emphasis"><em>RECORD_NAME</em></span> <span class="emphasis"><em>DB_VERSION_NUMBER</em></span> <span class="emphasis"><em>RECORD_NUMBER</em></span></pre> <pre class="programlisting">BEGIN_COMPAT <span class="emphasis"><em>RECORD_NAME</em></span> <span class="emphasis"><em>DB_VERSION_NUMBER</em></span> <span class="emphasis"><em>RECORD_NUMBER</em></span></pre> - <p>and ends with the line:</p> + <p> + and ends with the line: + </p> <pre class="programlisting">END</pre> - <p>The <span class="emphasis"><em>BEGIN</em></span> line should be used for most record types.</p> - <p>The <span class="emphasis"><em>BEGIN_COMPAT</em></span> is used for log record compatibility to facilitate -online upgrades of replication groups. Records created with this keyword will -produce reading and printing routines, but no logging routines. The recovery -routines are retrieved from older releases, so no recovery templates will be -generated for these records.</p> - <p>The <span class="emphasis"><em>DB_VERSION_NUMBER</em></span> variable should be replaced with the -current major and minor version of Berkeley DB, with all punctuation removed. -For example, Berkeley DB version 4.2 should be 42, version 4.5 should be 45.</p> - <p>The <span class="emphasis"><em>RECORD_NAME</em></span> variable should be replaced with a record -name for this log record. The <span class="emphasis"><em>RECORD_NUMBER</em></span> variable should -be replaced with a record number.</p> - <p>The combination of PREFIX name and <span class="emphasis"><em>RECORD_NAME</em></span>, and the -<span class="emphasis"><em>RECORD_NUMBER</em></span> must be unique for the application, that is, -values for application-specific and Berkeley DB log records may not overlap. -Further, because record numbers are stored in log files, which are -usually portable across application and Berkeley DB releases, any change to -the record numbers or log record format or should be handled as -described in the section on log format changes in -the <a href="../upgrading/upgrade_process.html" class="olink">Upgrading Berkeley DB installations</a> chapter of the Berkeley DB Installation and Build Guide. The record number space -below 10,000 is reserved for Berkeley DB itself; applications should choose -record number values equal to or greater than 10,000.</p> - <p>Between the BEGIN and END keywords there should be one optional -<span class="emphasis"><em>DUPLICATE</em></span> line and one line for each -data item logged as part of this log record.</p> - <p>The <span class="emphasis"><em>DUPLICATE</em></span> line is of the form:</p> + <p> + The <span class="emphasis"><em>BEGIN</em></span> line should be used for most + record types. + </p> + <p> + The <span class="emphasis"><em>BEGIN_COMPAT</em></span> is used for log record + compatibility to facilitate online upgrades of replication + groups. Records created with this keyword will produce reading + and printing routines, but no logging routines. The recovery + routines are retrieved from older releases, so no recovery + templates will be generated for these records. + </p> + <p> + The <span class="emphasis"><em>DB_VERSION_NUMBER</em></span> variable should + be replaced with the current major and minor version of + Berkeley DB, with all punctuation removed. For example, + Berkeley DB version 4.2 should be 42, version 4.5 should be + 45. + </p> + <p> + The <span class="emphasis"><em>RECORD_NAME</em></span> variable should be + replaced with a record name for this log record. The + <span class="emphasis"><em>RECORD_NUMBER</em></span> variable should be + replaced with a record number. + </p> + <p> + The combination of PREFIX name and + <span class="emphasis"><em>RECORD_NAME</em></span>, and the + <span class="emphasis"><em>RECORD_NUMBER</em></span> must be unique for the + application, that is, values for application-specific and + Berkeley DB log records may not overlap. Further, because + record numbers are stored in log files, which are usually + portable across application and Berkeley DB releases, any + change to the record numbers or log record format or should be + handled as described in the section on log format changes in + the <a href="../upgrading/upgrade_process.html" class="olink">Upgrading Berkeley DB installations</a> chapter of the Berkeley DB Installation and Build Guide. The record + number space below 10,000 is reserved for Berkeley DB itself; + applications should choose record number values equal to or + greater than 10,000. + </p> + <p> + Between the BEGIN and END keywords there should be one + optional <span class="emphasis"><em>DUPLICATE</em></span> line and one line for + each data item logged as part of this log record.</p> + <p> + The <span class="emphasis"><em>DUPLICATE</em></span> line is of the + form: + </p> <pre class="programlisting">DUPLICATE <span class="emphasis"><em>RECORD_NAME</em></span> <span class="emphasis"><em>DB_VERSION_NUMBER</em></span> <span class="emphasis"><em>RECORD_NUMBER</em></span></pre> - <p>The <span class="emphasis"><em>DUPLICATE</em></span> specifier should be used when creating a record -that requires its own record number but can use the argument structure, -reading and printing routines from another record. In this case, we will -create a new log record type, but use the enclosing log record type for -the argument structure and the log reading and printing routines.</p> - <p>The format of lines for each data item logged is as follows:</p> - <pre class="programlisting">ARG | DBT | POINTER <span class="emphasis"><em>variable_name</em></span> <span class="emphasis"><em>variable_type</em></span> <span class="emphasis"><em>printf_format</em></span></pre> - <p>The keyword ARG indicates that the argument is a simple parameter of -the type specified. For example, a file ID might be logged as:</p> - <pre class="programlisting">ARG fileID int d</pre> - <p>The keyword DBT indicates that the argument is a Berkeley DB DBT structure, -containing a length and pointer to a byte string. The keyword POINTER -indicates that the argument is a pointer to the data type specified (of -course the data type, not the pointer, is what is logged).</p> - <p>The <span class="emphasis"><em>variable_name</em></span> is the field name within the structure that -will be used to refer to this item. The <span class="emphasis"><em>variable_type</em></span> is -the C-language type of the variable, and the printf format is the -C-language format string, without the leading percent ("%") character, -that should be used to display the contents of the field (for example, -"s" for string, "d" for signed integral type, "u" for unsigned integral -type, "ld" for signed long integral type, "lu" for long unsigned -integral type, and so on).</p> - <p>For example, ex_apprec.src defines a single log record type, used to -log a directory name that has been stored in a DBT:</p> + <p> + The <span class="emphasis"><em>DUPLICATE</em></span> specifier should be used + when creating a record that requires its own record number but + can use the argument structure, reading and printing routines + from another record. In this case, we will create a new log + record type, but use the enclosing log record type for the + argument structure and the log reading and printing + routines. + </p> + <p> + The format of lines for each data item logged is as + follows: + </p> + <pre class="programlisting">ARG | DBT | POINTER <span class="emphasis"><em>variable_name</em></span> <span class="emphasis"><em>variable_type</em></span> <span class="emphasis"><em>printf_format</em></span></pre> + <p> + The keyword ARG indicates that the argument is a simple + parameter of the type specified. For example, a file ID might + be logged as: + </p> + <pre class="programlisting">ARG fileID int d</pre> + <p> + The keyword DBT indicates that the argument is a Berkeley DB + DBT structure, containing a length and pointer to a byte + string. The keyword POINTER indicates that the argument is a + pointer to the data type specified (of course the data type, + not the pointer, is what is logged). + </p> + <p> + The <span class="emphasis"><em>variable_name</em></span> is the field name + within the structure that will be used to refer to this item. + The <span class="emphasis"><em>variable_type</em></span> is the C-language type + of the variable, and the printf format is the C-language + format string, without the leading percent ("%") character, + that should be used to display the contents of the field (for + example, "s" for string, "d" for signed integral type, "u" for + unsigned integral type, "ld" for signed long integral type, + "lu" for long unsigned integral type, and so on). + </p> + <p> + For example, ex_apprec.src defines a single log record type, + used to log a directory name that has been stored in a + DBT: + </p> <pre class="programlisting">BEGIN mkdir 10000 DBT dirname DBT s END</pre> - <p>As the name suggests, this example of an application-defined log record -will be used to log the creation of a directory. There are many more -examples of XXX.src files in the Berkeley DB distribution. For example, the -file btree/btree.src contains the definitions for the log records -supported by the Berkeley DB Btree access method.</p> + <p> + As the name suggests, this example of an application-defined + log record will be used to log the creation of a directory. + There are many more examples of XXX.src files in the Berkeley + DB distribution. For example, the file btree/btree.src + contains the definitions for the log records supported by the + Berkeley DB Btree access method. + </p> </div> <div class="navfooter"> <hr /> @@ -135,13 +196,13 @@ supported by the Berkeley DB Btree access method.</p> <td width="40%" align="right"> <a accesskey="n" href="apprec_auto.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 14. - Application Specific Logging and Recovery - </td> + <td width="40%" align="left" valign="top">Chapter 14. Application Specific Logging and + Recovery </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Automatically generated functions</td> + <td width="40%" align="right" valign="top"> Automatically generated + functions</td> </tr> </table> </div> diff --git a/docs/programmer_reference/arch.html b/docs/programmer_reference/arch.html index 9c56c602..4f69a48c 100644 --- a/docs/programmer_reference/arch.html +++ b/docs/programmer_reference/arch.html @@ -14,13 +14,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 8. - Berkeley DB Architecture - </th> + <th colspan="3" align="center">Chapter 8. Berkeley DB Architecture </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="stl_known_issues.html">Prev</a> </td> @@ -34,9 +32,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="arch"></a>Chapter 8. - Berkeley DB Architecture - </h2> + <h2 class="title"><a id="arch"></a>Chapter 8. Berkeley DB Architecture </h2> </div> </div> </div> @@ -64,27 +60,27 @@ <dl> <dt> <span class="sect2"> - <a href="arch_apis.html#idp1156848">C</a> + <a href="arch_apis.html#idp1016624">C</a> </span> </dt> <dt> <span class="sect2"> - <a href="arch_apis.html#idp1467704">C++</a> + <a href="arch_apis.html#idp1020280">C++</a> </span> </dt> <dt> <span class="sect2"> - <a href="arch_apis.html#idp1468224">STL</a> + <a href="arch_apis.html#idp1041496">STL</a> </span> </dt> <dt> <span class="sect2"> - <a href="arch_apis.html#idp1467768">Java</a> + <a href="arch_apis.html#idp1042312">Java</a> </span> </dt> <dt> <span class="sect2"> - <a href="arch_apis.html#idp1485576">Dbm/Ndbm, Hsearch</a> + <a href="arch_apis.html#idp1061592">Dbm/Ndbm, Hsearch</a> </span> </dt> </dl> @@ -98,17 +94,17 @@ <dl> <dt> <span class="sect2"> - <a href="arch_script.html#idp1461432">Perl</a> + <a href="arch_script.html#idp1034968">Perl</a> </span> </dt> <dt> <span class="sect2"> - <a href="arch_script.html#idp1460536">PHP</a> + <a href="arch_script.html#idp1033112">PHP</a> </span> </dt> <dt> <span class="sect2"> - <a href="arch_script.html#idp1477280">Tcl</a> + <a href="arch_script.html#idp1052216">Tcl</a> </span> </dt> </dl> @@ -128,130 +124,182 @@ </div> </div> </div> - <p>The previous chapters in this Reference Guide have described -applications that use the Berkeley DB access methods for fast data storage -and retrieval. The applications described in the following chapters -are similar in nature to the access method applications, but they are -also threaded and/or recoverable in the face of application or system -failure.</p> - <p>Application code that uses only the Berkeley DB access methods might appear -as follows:</p> + <p> + The previous chapters in this Reference Guide have described + applications that use the Berkeley DB access methods for fast + data storage and retrieval. The applications described in the + following chapters are similar in nature to the access method + applications, but they are also threaded and/or recoverable in + the face of application or system failure. + </p> + <p> + Application code that uses only the Berkeley DB access + methods might appear as follows: + </p> <pre class="programlisting">switch (ret = dbp->/put(dbp, NULL, &key, &data, 0)) { case 0: - printf("db: %s: key stored.\n", (char *)key.data); - break; + printf("db: %s: key stored.\n", (char *)key.data); + break; default: - dbp->/err(dbp, ret, "dbp->/put"); - exit (1); + dbp->/err(dbp, ret, "dbp->/put"); + exit (1); }</pre> - <p>The underlying Berkeley DB architecture that supports this is</p> + <p> + The underlying Berkeley DB architecture that supports this + is + </p> <div class="mediaobject"> <img src="arch_smallpic.gif" /> </div> - <p>As you can see from this diagram, the application makes calls into the -access methods, and the access methods use the underlying shared memory -buffer cache to hold recently used file pages in main memory.</p> - <p>When applications require recoverability, their calls to the Access -Methods must be wrapped in calls to the transaction subsystem. The -application must inform Berkeley DB where to begin and end transactions, and -must be prepared for the possibility that an operation may fail at any -particular time, causing the transaction to abort.</p> - <p>An example of transaction-protected code might appear as follows:</p> + <p> + As you can see from this diagram, the application makes + calls into the access methods, and the access methods use the + underlying shared memory buffer cache to hold recently used + file pages in main memory. + </p> + <p> + When applications require recoverability, their calls to the + Access Methods must be wrapped in calls to the transaction + subsystem. The application must inform Berkeley DB where to + begin and end transactions, and must be prepared for the + possibility that an operation may fail at any particular time, + causing the transaction to abort. + </p> + <p> + An example of transaction-protected code might appear as + follows: + </p> <pre class="programlisting">for (fail = 0;;) { - /* Begin the transaction. */ - if ((ret = dbenv->/txn_begin(dbenv, NULL, &tid, 0)) != 0) { - dbenv->/err(dbenv, ret, "dbenv->/txn_begin"); - exit (1); - } + /* Begin the transaction. */ + if ((ret = dbenv->/txn_begin(dbenv, NULL, &tid, 0)) != 0) { + dbenv->/err(dbenv, ret, "dbenv->/txn_begin"); + exit (1); + } - /* Store the key. */ - switch (ret = dbp->/put(dbp, tid, &key, &data, 0)) { - case 0: - /* Success: commit the change. */ - printf("db: %s: key stored.\n", (char *)key.data); - if ((ret = tid->/commit(tid, 0)) != 0) { - dbenv->/err(dbenv, ret, "DB_TXN->/commit"); - exit (1); - } - return (0); - case DB_LOCK_DEADLOCK: - default: - /* Failure: retry the operation. */ - if ((t_ret = tid->/abort(tid)) != 0) { - dbenv->/err(dbenv, t_ret, "DB_TXN->/abort"); - exit (1); - } - if (fail++ == MAXIMUM_RETRY) - return (ret); - continue; - } + /* Store the key. */ + switch (ret = dbp->/put(dbp, tid, &key, &data, 0)) { + case 0: + /* Success: commit the change. */ + printf("db: %s: key stored.\n", (char *)key.data); + if ((ret = tid->/commit(tid, 0)) != 0) { + dbenv->/err(dbenv, ret, "DB_TXN->/commit"); + exit (1); + } + return (0); + case DB_LOCK_DEADLOCK: + default: + /* Failure: retry the operation. */ + if ((t_ret = tid->/abort(tid)) != 0) { + dbenv->/err(dbenv, t_ret, "DB_TXN->/abort"); + exit (1); + } + if (fail++ == MAXIMUM_RETRY) + return (ret); + continue; + } }</pre> - <p>In this example, the same operation is being done as before; however, -it is wrapped in transaction calls. The transaction is started with -<a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a> and finished with <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a>. If the -operation fails due to a deadlock, the transaction is aborted using -<a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a>, after which the operation may be retried.</p> - <p>There are actually five major subsystems in Berkeley DB, as follows:</p> + <p> + In this example, the same operation is being done as before; + however, it is wrapped in transaction calls. The transaction + is started with <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a> and finished with <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a>. If + the operation fails due to a deadlock, the transaction is + aborted using <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a>, after which the operation may be + retried. + </p> + <p> + There are actually five major subsystems in Berkeley DB, as + follows: + </p> <div class="variablelist"> <dl> <dt> <span class="term">Access Methods</span> </dt> - <dd>The access methods subsystem provides general-purpose support for -creating and accessing database files formatted as Btrees, Hashed files, -and Fixed- and Variable-length records. These modules are useful in -the absence of transactions for applications that need fast formatted -file support. See <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> and <a href="../api_reference/C/dbcursor.html" class="olink">DB->cursor()</a> for more -information. These functions were already discussed in detail in the -previous chapters.</dd> + <dd> + The access methods subsystem provides + general-purpose support for creating and accessing + database files formatted as Btrees, Hashed files, and + Fixed- and Variable-length records. These modules are + useful in the absence of transactions for applications + that need fast formatted file support. See <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> + and <a href="../api_reference/C/dbcursor.html" class="olink">DB->cursor()</a> for more information. These functions + were already discussed in detail in the previous + chapters. + </dd> <dt> <span class="term">Memory Pool</span> </dt> - <dd>The Memory Pool subsystem is the general-purpose shared memory buffer pool -used by Berkeley DB. This is the shared memory cache that allows multiple -processes and threads within processes to share access to databases. This -module is useful outside of the Berkeley DB package for processes that require -portable, page-oriented, cached, shared file access.</dd> + <dd> + The Memory Pool subsystem is the general-purpose + shared memory buffer pool used by Berkeley DB. This is + the shared memory cache that allows multiple processes + and threads within processes to share access to + databases. This module is useful outside of the + Berkeley DB package for processes that require + portable, page-oriented, cached, shared file + access. + </dd> <dt> <span class="term">Transaction</span> </dt> - <dd>The Transaction subsystem allows a group of database changes to be -treated as an atomic unit so that either all of the changes are done, -or none of the changes are done. The transaction subsystem implements -the Berkeley DB transaction model. This module is useful outside of the Berkeley DB -package for processes that want to transaction-protect their own data -modifications.</dd> + <dd> + The Transaction subsystem allows a group of + database changes to be treated as an atomic unit so + that either all of the changes are done, or none of + the changes are done. The transaction subsystem + implements the Berkeley DB transaction model. This + module is useful outside of the Berkeley DB package + for processes that want to transaction-protect their + own data modifications. + </dd> <dt> <span class="term">Locking</span> </dt> - <dd>The Locking subsystem is the general-purpose lock manager used by Berkeley DB. -This module is useful outside of the Berkeley DB package for processes that -require a portable, fast, configurable lock manager.</dd> + <dd> + The Locking subsystem is the general-purpose + lock manager used by Berkeley DB. This module is + useful outside of the Berkeley DB package for + processes that require a portable, fast, configurable + lock manager. + </dd> <dt> <span class="term">Logging</span> </dt> - <dd>The Logging subsystem is the write-ahead logging used to support the -Berkeley DB transaction model. It is largely specific to the Berkeley DB package, -and unlikely to be useful elsewhere except as a supporting module for -the Berkeley DB transaction subsystem.</dd> + <dd> + The Logging subsystem is the write-ahead logging + used to support the Berkeley DB transaction model. It + is largely specific to the Berkeley DB package, and + unlikely to be useful elsewhere except as a supporting + module for the Berkeley DB transaction + subsystem. + </dd> </dl> </div> - <p>Here is a more complete picture of the Berkeley DB library:</p> + <p> + Here is a more complete picture of the Berkeley DB + library: + </p> <div class="mediaobject"> <img src="arch_bigpic.gif" /> </div> - <p>In this model, the application makes calls to the access methods and to -the Transaction subsystem. The access methods and Transaction subsystems -in turn make calls into the Memory Pool, Locking and Logging subsystems -on behalf of the application.</p> - <p>The underlying subsystems can be used independently by applications. -For example, the Memory Pool subsystem can be used apart from the rest -of Berkeley DB by applications simply wanting a shared memory buffer pool, or -the Locking subsystem may be called directly by applications that are -doing their own locking outside of Berkeley DB. However, this usage is not -common, and most applications will either use only the access methods -subsystem, or the access methods subsystem wrapped in calls to the Berkeley DB -transaction interfaces.</p> + <p> + In this model, the application makes calls to the access + methods and to the Transaction subsystem. The access methods + and Transaction subsystems in turn make calls into the Memory + Pool, Locking and Logging subsystems on behalf of the + application. + </p> + <p> + The underlying subsystems can be used independently by + applications. For example, the Memory Pool subsystem can be + used apart from the rest of Berkeley DB by applications simply + wanting a shared memory buffer pool, or the Locking subsystem + may be called directly by applications that are doing their + own locking outside of Berkeley DB. However, this usage is not + common, and most applications will either use only the access + methods subsystem, or the access methods subsystem wrapped in + calls to the Berkeley DB transaction interfaces. + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/arch_apis.html b/docs/programmer_reference/arch_apis.html index 34e486ce..e592ea0a 100644 --- a/docs/programmer_reference/arch_apis.html +++ b/docs/programmer_reference/arch_apis.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="arch_progmodel.html">Prev</a> </td> - <th width="60%" align="center">Chapter 8. - Berkeley DB Architecture - </th> + <th width="60%" align="center">Chapter 8. Berkeley DB Architecture </th> <td width="20%" align="right"> <a accesskey="n" href="arch_script.html">Next</a></td> </tr> </table> @@ -42,155 +40,216 @@ <dl> <dt> <span class="sect2"> - <a href="arch_apis.html#idp1156848">C</a> + <a href="arch_apis.html#idp1016624">C</a> </span> </dt> <dt> <span class="sect2"> - <a href="arch_apis.html#idp1467704">C++</a> + <a href="arch_apis.html#idp1020280">C++</a> </span> </dt> <dt> <span class="sect2"> - <a href="arch_apis.html#idp1468224">STL</a> + <a href="arch_apis.html#idp1041496">STL</a> </span> </dt> <dt> <span class="sect2"> - <a href="arch_apis.html#idp1467768">Java</a> + <a href="arch_apis.html#idp1042312">Java</a> </span> </dt> <dt> <span class="sect2"> - <a href="arch_apis.html#idp1485576">Dbm/Ndbm, Hsearch</a> + <a href="arch_apis.html#idp1061592">Dbm/Ndbm, Hsearch</a> </span> </dt> </dl> </div> - <p>The Berkeley DB subsystems can be accessed through interfaces from multiple -languages. Applications can use Berkeley DB via C, C++ or Java, as well as a -variety of scripting languages such as Perl, Python, Ruby or Tcl. -Environments can be shared among applications written by using any of -these interfaces. For example, you might have a local server written -in C or C++, a script for an administrator written in Perl or Tcl, and -a Web-based user interface written in Java -- all sharing a single -database environment.</p> + <p> + The Berkeley DB subsystems can be accessed through + interfaces from multiple languages. Applications can use + Berkeley DB via C, C++ or Java, as well as a variety of + scripting languages such as Perl, Python, Ruby or Tcl. + Environments can be shared among applications written by using + any of these interfaces. For example, you might have a local + server written in C or C++, a script for an administrator + written in Perl or Tcl, and a Web-based user interface written + in Java -- all sharing a single database environment. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1156848"></a>C</h3> + <h3 class="title"><a id="idp1016624"></a>C</h3> </div> </div> </div> - <p>The Berkeley DB library is written entirely in ANSI C. C applications use a -single include file:</p> + <p> + The Berkeley DB library is written entirely in ANSI C. C + applications use a single include file: + </p> <pre class="programlisting">#include <db.h></pre> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1467704"></a>C++</h3> + <h3 class="title"><a id="idp1020280"></a>C++</h3> </div> </div> </div> - <p>The C++ classes provide a thin wrapper around the C API, with the major -advantages being improved encapsulation and an optional exception -mechanism for errors. C++ applications use a single include file:</p> + <p> + The C++ classes provide a thin wrapper around the C API, + with the major advantages being improved encapsulation and + an optional exception mechanism for errors. C++ + applications use a single include file: + </p> <pre class="programlisting">#include <db_cxx.h></pre> - <p>The classes and methods are named in a fashion that directly corresponds -to structures and functions in the C interface. Likewise, arguments to -methods appear in the same order as the C interface, except to remove the -explicit <span class="bold"><strong>this</strong></span> pointer. The #defines used for flags are identical -between the C and C++ interfaces.</p> - <p>As a rule, each C++ object has exactly one structure from the underlying -C API associated with it. The C structure is allocated with each -constructor call and deallocated with each destructor call. Thus, the -rules the user needs to follow in allocating and deallocating structures -are the same between the C and C++ interfaces.</p> - <p>To ensure portability to many platforms, both new and old, Berkeley DB makes -as few assumptions as possible about the C++ compiler and library. For -example, it does not expect STL, templates, or namespaces to be -available. The newest C++ feature used is exceptions, which are used -liberally to transmit error information. Even the use of exceptions -can be disabled at runtime.</p> + <p> + The classes and methods are named in a fashion that + directly corresponds to structures and functions in the C + interface. Likewise, arguments to methods appear in the + same order as the C interface, except to remove the + explicit <span class="bold"><strong>this</strong></span> pointer. + The #defines used for flags are identical between the C + and C++ interfaces. + </p> + <p> + As a rule, each C++ object has exactly one structure + from the underlying C API associated with it. The C + structure is allocated with each constructor call and + deallocated with each destructor call. Thus, the rules the + user needs to follow in allocating and deallocating + structures are the same between the C and C++ + interfaces. + </p> + <p> + To ensure portability to many platforms, both new and + old, Berkeley DB makes as few assumptions as possible + about the C++ compiler and library. For example, it does + not expect STL, templates, or namespaces to be available. + The newest C++ feature used is exceptions, which are used + liberally to transmit error information. Even the use of + exceptions can be disabled at runtime. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1468224"></a>STL</h3> + <h3 class="title"><a id="idp1041496"></a>STL</h3> </div> </div> </div> <p> -dbstl is an C++ STL style API for Berkeley DB, based on the C++ API above. With it, you can store data/objects of any type into -or retrieve them from Berkeley DB databases as if you are using C++ STL containers. The full functionality -of Berkeley DB can still be utilized via dbstl with little performance overhead, e.g. you can use all transaction and/or replication functionality of Berkeley DB. - -</p> + dbstl is an C++ STL style API for Berkeley DB, based on + the C++ API above. With it, you can store data/objects of + any type into or retrieve them from Berkeley DB databases + as if you are using C++ STL containers. The full + functionality of Berkeley DB can still be utilized via + dbstl with little performance overhead, e.g. you can use + all transaction and/or replication functionality of + Berkeley DB. + </p> <p> -dbstl container/iterator class templates reside in header files dbstl_vector.h, dbstl_map.h and dbstl_set.h. - -Among them, dbstl_vector.h contains dbstl::db_vector and its iterators; -dbstl_map.h contains dbstl::db_map, dbstl::db_multimap and their iterators; dbstl_set.h contains dbstl::db_set and dbstl::db_multiset and their iterators. You should -include needed header file(s) to use the container/iterator. Note that we don't use the file name with no extention --- To use dbstl::db_vector, you should do this: -</p> + dbstl container/iterator class templates reside in + header files dbstl_vector.h, dbstl_map.h and dbstl_set.h. + Among them, dbstl_vector.h contains dbstl::db_vector and + its iterators; dbstl_map.h contains dbstl::db_map, + dbstl::db_multimap and their iterators; dbstl_set.h + contains dbstl::db_set and dbstl::db_multiset and their + iterators. You should include needed header file(s) to use + the container/iterator. Note that we don't use the file + name with no extention --- To use dbstl::db_vector, you + should do this: + </p> <pre class="programlisting">#include "dbstl_vector.h"</pre> - <p> -rather than this: -</p> + <p> + rather than this: + </p> <pre class="programlisting">#include "dbstl_vector"</pre> <p> -And these header files reside in "stl" directory inside Berkeley DB source root directory. If you have installed -Berkeley DB, they are also available in the "include" directory in the directory where Berkeley DB is installed. -</p> + And these header files reside in "stl" directory inside + Berkeley DB source root directory. If you have installed + Berkeley DB, they are also available in the "include" + directory in the directory where Berkeley DB is installed. + </p> + <p> + Apart from the above three header files, you may also + need to include db_exception.h and db_utility.h files. The + db_exception.h file contains all exception classes of + dbstl, which integrate seamlessly with Berkeley DB C++ API + exceptions and C++ standard exception classes in std + namespace. And the db_utility.h file contains the + DbstlElemTraits which helps you to store complex objects. + These five header files are all that you need to include + in order to make use of dbstl. + </p> <p> -Apart from the above three header files, you may also need to include db_exception.h and db_utility.h files. The db_exception.h file contains all exception classes of dbstl, which -integrate seamlessly with Berkeley DB C++ API exceptions and C++ standard exception classes in std namespace. And the db_utility.h file contains the DbstlElemTraits which helps you -to store complex objects. These five header files are all that you need to include in order to make use of dbstl. -</p> - <p> -All symbols of dbstl, including classes, class templates, global functions, etc, reside in the namespace "dbstl", so in order to use them, you may also want to do this: -</p> + All symbols of dbstl, including classes, class + templates, global functions, etc, reside in the namespace + "dbstl", so in order to use them, you may also want to do + this: + </p> <pre class="programlisting">using namespace dbstl;</pre> - <p> -The dbstl library is always at the same place where Berkeley DB library is located, you will need to build it and link with it to use dbstl. -</p> - <p> -While making use of dbstl, you will probably want to create environment or databases directly, or set/get configurations to Berkeley DB environment or databases, etc. You are allowed to do so via Berkeley DB C/C++ API. -</p> + <p> + The dbstl library is always at the same place where + Berkeley DB library is located, you will need to build it + and link with it to use dbstl. + </p> + <p> + While making use of dbstl, you will probably want to + create environment or databases directly, or set/get + configurations to Berkeley DB environment or databases, + etc. You are allowed to do so via Berkeley DB C/C++ API. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1467768"></a>Java</h3> + <h3 class="title"><a id="idp1042312"></a>Java</h3> </div> </div> </div> - <p>The Java classes provide a layer around the C API that is almost identical -to the C++ layer. The classes and methods are, for the most part -identical to the C++ layer. Berkeley DB constants and #defines are represented as -"static final int" values. Error conditions are communicated as Java -exceptions.</p> - <p>As in C++, each Java object has exactly one structure from the underlying -C API associated with it. The Java structure is allocated with each -constructor or open call, but is deallocated only by the Java garbage -collector. Because the timing of garbage collection is not predictable, -applications should take care to do a close when finished with any object -that has a close method.</p> + <p> + The Java classes provide a layer around the C API that + is almost identical to the C++ layer. The classes and + methods are, for the most part identical to the C++ layer. + Berkeley DB constants and #defines are represented as + "static final int" values. Error conditions are + communicated as Java exceptions. + </p> + <p> + As in C++, each Java object has exactly one structure + from the underlying C API associated with it. The Java + structure is allocated with each constructor or open call, + but is deallocated only by the Java garbage collector. + Because the timing of garbage collection is not + predictable, applications should take care to do a close + when finished with any object that has a close + method. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1485576"></a>Dbm/Ndbm, Hsearch</h3> + <h3 class="title"><a id="idp1061592"></a>Dbm/Ndbm, Hsearch</h3> </div> </div> </div> - <p>Berkeley DB supports the standard UNIX <a href="../api_reference/C/dbm.html" class="olink">dbm</a> and <a href="../api_reference/C/hsearch.html" class="olink">hsearch</a> interfaces. After including a new header file and recompiling, programs will run orders of magnitude faster, and underlying databases can grow as large as necessary. Also, historic <a href="../api_reference/C/dbm.html" class="olink">dbm</a> applications can fail once some number of entries are inserted into the database, in which the number depends on the effectiveness of the internal hashing function on the particular data set. This is not a problem with Berkeley DB.</p> + <p> + Berkeley DB supports the standard UNIX <a href="../api_reference/C/dbm.html" class="olink">dbm</a> and + <a href="../api_reference/C/hsearch.html" class="olink">hsearch</a> interfaces. After including a new header file + and recompiling, programs will run orders of magnitude + faster, and underlying databases can grow as large as + necessary. Also, historic <a href="../api_reference/C/dbm.html" class="olink">dbm</a> applications can fail once + some number of entries are inserted into the database, in + which the number depends on the effectiveness of the + internal hashing function on the particular data set. This + is not a problem with Berkeley DB. + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/arch_progmodel.html b/docs/programmer_reference/arch_progmodel.html index a1136d2b..d459c9ac 100644 --- a/docs/programmer_reference/arch_progmodel.html +++ b/docs/programmer_reference/arch_progmodel.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="arch.html">Prev</a> </td> - <th width="60%" align="center">Chapter 8. - Berkeley DB Architecture - </th> + <th width="60%" align="center">Chapter 8. Berkeley DB Architecture </th> <td width="20%" align="right"> <a accesskey="n" href="arch_apis.html">Next</a></td> </tr> </table> @@ -38,15 +36,19 @@ </div> </div> </div> - <p> - Berkeley DB is a database library, in which the library is linked into the address space of the - application using it. One or more applications link the Berkeley DB library - directly into their address spaces. There may be many threads of control in this model because - Berkeley DB supports locking for both multiple processes and for multiple threads within a - process. This model provides significantly faster access to the database functionality, but - implies trust among all threads of control sharing the database environment because they will - have the ability to read, write and potentially corrupt each other's data. -</p> + <p> + Berkeley DB is a database library, in which the library is + linked into the address space of the application using it. One + or more applications link the Berkeley DB library directly + into their address spaces. There may be many threads of + control in this model because Berkeley DB supports locking for + both multiple processes and for multiple threads within a + process. This model provides significantly faster access to + the database functionality, but implies trust among all + threads of control sharing the database environment because + they will have the ability to read, write and potentially + corrupt each other's data. + </p> </div> <div class="navfooter"> <hr /> @@ -59,9 +61,7 @@ <td width="40%" align="right"> <a accesskey="n" href="arch_apis.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 8. - Berkeley DB Architecture - </td> + <td width="40%" align="left" valign="top">Chapter 8. Berkeley DB Architecture </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/arch_script.html b/docs/programmer_reference/arch_script.html index b17edcc0..c9062bbf 100644 --- a/docs/programmer_reference/arch_script.html +++ b/docs/programmer_reference/arch_script.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="arch_apis.html">Prev</a> </td> - <th width="60%" align="center">Chapter 8. - Berkeley DB Architecture - </th> + <th width="60%" align="center">Chapter 8. Berkeley DB Architecture </th> <td width="20%" align="right"> <a accesskey="n" href="arch_utilities.html">Next</a></td> </tr> </table> @@ -42,17 +40,17 @@ <dl> <dt> <span class="sect2"> - <a href="arch_script.html#idp1461432">Perl</a> + <a href="arch_script.html#idp1034968">Perl</a> </span> </dt> <dt> <span class="sect2"> - <a href="arch_script.html#idp1460536">PHP</a> + <a href="arch_script.html#idp1033112">PHP</a> </span> </dt> <dt> <span class="sect2"> - <a href="arch_script.html#idp1477280">Tcl</a> + <a href="arch_script.html#idp1052216">Tcl</a> </span> </dt> </dl> @@ -61,39 +59,44 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1461432"></a>Perl</h3> + <h3 class="title"><a id="idp1034968"></a>Perl</h3> </div> </div> </div> - <p>Two Perl wrappers are distributed with the Berkeley DB release. The Perl -interface to Berkeley DB version 1.85 is called DB_File. The Perl interface -to Berkeley DB version 2 and later is called BerkeleyDB. See -<a class="xref" href="ext_perl.html" title="Using Berkeley DB with Perl">Using Berkeley DB with Perl</a> for more -information.</p> + <p> + Two Perl wrappers are distributed with the Berkeley DB + release. The Perl interface to Berkeley DB version 1.85 is + called DB_File. The Perl interface to Berkeley DB version + 2 and later is called BerkeleyDB. See <a class="xref" href="ext_perl.html" title="Using Berkeley DB with Perl">Using Berkeley DB with Perl</a> for more + information. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1460536"></a>PHP</h3> + <h3 class="title"><a id="idp1033112"></a>PHP</h3> </div> </div> </div> - <p>A PHP wrapper is distributed with the Berkeley DB release. See -<a class="xref" href="ext_php.html" title="Using Berkeley DB with PHP">Using Berkeley DB with PHP</a> for more -information.</p> + <p> + A PHP wrapper is distributed with the Berkeley DB + release. See <a class="xref" href="ext_php.html" title="Using Berkeley DB with PHP">Using Berkeley DB with PHP</a> for more information. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1477280"></a>Tcl</h3> + <h3 class="title"><a id="idp1052216"></a>Tcl</h3> </div> </div> </div> - <p>A Tcl wrapper is distributed with the Berkeley DB release. See -<a class="xref" href="tcl.html#tcl_intro" title="Loading Berkeley DB with Tcl">Loading Berkeley DB with Tcl</a> for more -information.</p> + <p> + A Tcl wrapper is distributed with the Berkeley DB + release. See <a class="xref" href="tcl.html#tcl_intro" title="Loading Berkeley DB with Tcl">Loading Berkeley DB with Tcl</a> for more + information. + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/arch_utilities.html b/docs/programmer_reference/arch_utilities.html index 3651fe58..63a2414e 100644 --- a/docs/programmer_reference/arch_utilities.html +++ b/docs/programmer_reference/arch_utilities.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="arch_script.html">Prev</a> </td> - <th width="60%" align="center">Chapter 8. - Berkeley DB Architecture - </th> + <th width="60%" align="center">Chapter 8. Berkeley DB Architecture </th> <td width="20%" align="right"> <a accesskey="n" href="env.html">Next</a></td> </tr> </table> @@ -38,87 +36,139 @@ </div> </div> </div> - <p>The following are the standalone utilities that provide supporting -functionality for the Berkeley DB environment:</p> + <p> + The following are the standalone utilities that provide + supporting functionality for the Berkeley DB + environment: + </p> <div class="variablelist"> <dl> <dt> <span class="term"><a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility</span> </dt> - <dd>The <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility supports database backup and archival, -and log file administration. It facilitates log reclamation and the -creation of database snapshots. Generally, some form of log archival -must be done if a database environment has been configured for logging -or transactions.</dd> + <dd> + The <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility supports database backup and + archival, and log file administration. It facilitates + log reclamation and the creation of database + snapshots. Generally, some form of log archival must + be done if a database environment has been configured + for logging or transactions. + </dd> <dt> <span class="term"><a href="../api_reference/C/db_checkpoint.html" class="olink">db_checkpoint</a> utility</span> </dt> - <dd>The <a href="../api_reference/C/db_checkpoint.html" class="olink">db_checkpoint</a> utility runs as a daemon process, monitoring -the database log and periodically issuing checkpoints. It facilitates -log reclamation and the creation of database snapshots. Generally, some -form of database checkpointing must be done if a database environment has -been configured for transactions.</dd> + <dd> + The <a href="../api_reference/C/db_checkpoint.html" class="olink">db_checkpoint</a> utility runs as a daemon process, + monitoring the database log and periodically issuing + checkpoints. It facilitates log reclamation and the + creation of database snapshots. Generally, some form + of database checkpointing must be done if a database + environment has been configured for + transactions. + </dd> <dt> <span class="term"><a href="../api_reference/C/db_deadlock.html" class="olink">db_deadlock</a> utility</span> </dt> - <dd>The <a href="../api_reference/C/db_deadlock.html" class="olink">db_deadlock</a> utility runs as a daemon process, periodically -traversing the database lock structures and aborting transactions when it -detects a deadlock. Generally, some form of deadlock detection must be -done if a database environment has been configured for locking.</dd> + <dd> + The <a href="../api_reference/C/db_deadlock.html" class="olink">db_deadlock</a> utility runs as a daemon process, + periodically traversing the database lock structures + and aborting transactions when it detects a deadlock. + Generally, some form of deadlock detection must be + done if a database environment has been configured for + locking. + </dd> <dt> <span class="term"><a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility</span> </dt> - <dd>The <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility writes a copy of the database to a flat-text file in a portable format.</dd> + <dd> + The <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility writes a copy of the database to a + flat-text file in a portable format. + </dd> <dt> <span class="term"><a href="../api_reference/C/db_hotbackup.html" class="olink">db_hotbackup</a> utility</span> </dt> - <dd>The <a href="../api_reference/C/db_hotbackup.html" class="olink">db_hotbackup</a> utility creates "hot backup" or "hot failover" -snapshots of Berkeley DB database environments.</dd> + <dd> + The <a href="../api_reference/C/db_hotbackup.html" class="olink">db_hotbackup</a> utility creates "hot backup" or "hot + failover" snapshots of Berkeley DB database + environments. + </dd> <dt> <span class="term"><a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility</span> </dt> - <dd>The <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility reads the flat-text file produced by the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility and loads it into a database file.</dd> + <dd> + The <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility reads the flat-text file produced + by the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility and loads it into a database + file. + </dd> <dt> <span class="term"><a href="../api_reference/C/db_printlog.html" class="olink">db_printlog</a> utility</span> </dt> - <dd>The <a href="../api_reference/C/db_printlog.html" class="olink">db_printlog</a> utility displays the contents of Berkeley DB log files in a human-readable and parsable format.</dd> + <dd> + The <a href="../api_reference/C/db_printlog.html" class="olink">db_printlog</a> utility displays the contents of + Berkeley DB log files in a human-readable and parsable + format. + </dd> <dt> <span class="term"><a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility</span> </dt> - <dd>The <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility runs after an unexpected Berkeley DB or system -failure to restore the database to a consistent state. Generally, some -form of database recovery must be done if databases are being modified.</dd> + <dd> + The <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility runs after an unexpected + Berkeley DB or system failure to restore the database + to a consistent state. Generally, some form of + database recovery must be done if databases are being + modified. + </dd> <dt> <span class="term">db_sql_codegen</span> </dt> - <dd>The db_sql_codegen utility translates a - schema description written in a SQL Data Definition Language dialect - into C code that implements the schema using Berkeley DB.</dd> + <dd> + The db_sql_codegen utility translates a schema + description written in a SQL Data Definition Language + dialect into C code that implements the schema using + Berkeley DB. + </dd> <dt> <span class="term"><a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility</span> </dt> - <dd>The <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility displays statistics for databases and database -environments.</dd> + <dd> + The <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility displays statistics for databases + and database environments. + </dd> <dt> <span class="term"><a href="../api_reference/C/db_tuner.html" class="olink">db_tuner</a> utility</span> </dt> - <dd>The <a href="../api_reference/C/db_tuner.html" class="olink">db_tuner</a> utility suggests a page size for btree databases that optimizes cache efficiency and storage space requirements.</dd> + <dd> + The <a href="../api_reference/C/db_tuner.html" class="olink">db_tuner</a> utility suggests a page size for btree + databases that optimizes cache efficiency and storage + space requirements. + </dd> <dt> <span class="term"><a href="../api_reference/C/db_upgrade.html" class="olink">db_upgrade</a> utility</span> </dt> - <dd>The <a href="../api_reference/C/db_upgrade.html" class="olink">db_upgrade</a> utility provides a command-line interface for upgrading underlying database formats.</dd> + <dd> + The <a href="../api_reference/C/db_upgrade.html" class="olink">db_upgrade</a> utility provides a command-line + interface for upgrading underlying database + formats. + </dd> <dt> <span class="term"><a href="../api_reference/C/db_verify.html" class="olink">db_verify</a> utility</span> </dt> - <dd>The <a href="../api_reference/C/db_verify.html" class="olink">db_verify</a> utility provides a command-line interface for verifying the database format.</dd> + <dd> + The <a href="../api_reference/C/db_verify.html" class="olink">db_verify</a> utility provides a command-line + interface for verifying the database + format. + </dd> </dl> </div> - <p>All of the functionality implemented for these utilities is also available -as part of the standard Berkeley DB API. This means that threaded applications -can easily create a thread that calls the same Berkeley DB functions as do the -utilities. This often simplifies an application environment by removing -the necessity for multiple processes to negotiate database and database -environment creation and shut down.</p> + <p> + All of the functionality implemented for these utilities is + also available as part of the standard Berkeley DB API. This + means that threaded applications can easily create a thread + that calls the same Berkeley DB functions as do the utilities. + This often simplifies an application environment by removing + the necessity for multiple processes to negotiate database and + database environment creation and shut down. + </p> </div> <div class="navfooter"> <hr /> @@ -135,9 +185,7 @@ environment creation and shut down.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 9. - The Berkeley DB Environment - </td> + <td width="40%" align="right" valign="top"> Chapter 9. The Berkeley DB Environment </td> </tr> </table> </div> diff --git a/docs/programmer_reference/blobs.html b/docs/programmer_reference/blobs.html new file mode 100644 index 00000000..3aedb7ee --- /dev/null +++ b/docs/programmer_reference/blobs.html @@ -0,0 +1,366 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> +<html xmlns="http://www.w3.org/1999/xhtml"> + <head> + <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> + <title>BLOB support</title> + <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> + <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> + <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> + <link rel="up" href="am_misc.html" title="Chapter 4. Access Method Wrapup" /> + <link rel="prev" href="am_misc_diskspace.html" title="Disk space requirements" /> + <link rel="next" href="am_misc_db_sql.html" title="Specifying a Berkeley DB schema using SQL DDL" /> + </head> + <body> + <div xmlns="" class="navheader"> + <div class="libver"> + <p>Library Version 12.1.6.1</p> + </div> + <table width="100%" summary="Navigation header"> + <tr> + <th colspan="3" align="center">BLOB support</th> + </tr> + <tr> + <td width="20%" align="left"><a accesskey="p" href="am_misc_diskspace.html">Prev</a> </td> + <th width="60%" align="center">Chapter 4. Access Method Wrapup </th> + <td width="20%" align="right"> <a accesskey="n" href="am_misc_db_sql.html">Next</a></td> + </tr> + </table> + <hr /> + </div> + <div class="sect1" lang="en" xml:lang="en"> + <div class="titlepage"> + <div> + <div> + <h2 class="title" style="clear: both"><a id="blobs"></a>BLOB support</h2> + </div> + </div> + </div> + <div class="toc"> + <dl> + <dt> + <span class="sect2"> + <a href="blobs.html#blob_threshold">The BLOB threshold</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="blobs.html#blob_create">Creating BLOBs</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="blobs.html#blob_access">BLOB access</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="blobs.html#blob_storage">BLOB storage</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="blobs.html#blob_replication">BLOBs and Replication</a> + </span> + </dt> + </dl> + </div> + <p> + Binary Large Objects (BLOB) support is designed for + efficient storage of large objects. An object is considered to + be large if it is more than a third of the size of a page. + Without BLOB support, large objects must be broken up into + smaller pieces, and then reassembled and/or disassembled every + time the record is read or updated. Berkeley DB BLOB support + avoids this assembly/disassembly process by storing the large + object in a special directory set aside for the purpose. The + data itself is not kept in the database, nor is it placed into + the in-memory cache. + </p> + <p> + BLOBs can only be stored using the data portion of a + key/data pair. They are supported only for Btree, Hash, and + Heap databases, and only so long as the database is not + configured for checksums, encryption, duplicate records, or + duplicate sorted records. In addition, the DBT that you use to + access the BLOB data cannot be configured as a partial DBT if + you want to access the data using the BLOB's streaming + interface (introduced below). + </p> + <p> + Note that if the environment is transactionally-protected, + then all access to the BLOB is also transactionally protected. + </p> + <div class="sect2" lang="en" xml:lang="en"> + <div class="titlepage"> + <div> + <div> + <h3 class="title"><a id="blob_threshold"></a>The BLOB threshold</h3> + </div> + </div> + </div> + <p> + The BLOB threshold is a positive integer, in bytes, + which indicates how large an object must be before it is + considered a BLOB. By default, the BLOB threshold for any + given database is <code class="literal">0</code>, which means that + no object will ever be considered a BLOB. This means that + the BLOB feature is not used by default for Berkeley DB + databases. + </p> + <p> + In order to use the BLOB feature, you must set the BLOB + threshold to a non-zero, positive integer value. You do + this for a given database using the <a href="../api_reference/C/set_blob_threshold.html" class="olink">DB->set_blob_threshold()</a> + method. Note that this value must be set before you create + the database. At any point after database creation time, + this method is ignored. + </p> + <p> + In addition, if you are using an environment, you can + change the default threshold for databases created in that + environment to something other than <code class="literal">0</code> + by using the <a href="../api_reference/C/envset_blob_threshold.html" class="olink">DB_ENV->set_blob_threshold()</a> method. + </p> + <p> You can retrieve the BLOB threshold set for a database + using the <a href="../api_reference/C/get_blob_threshold.html" class="olink">DB->get_blob_threshold()</a>. You can retrieve the + default BLOB threshold set for your environment using the + <a href="../api_reference/C/envget_blob_threshold.html" class="olink">DB_ENV->get_blob_threshold()</a>. + </p> + </div> + <div class="sect2" lang="en" xml:lang="en"> + <div class="titlepage"> + <div> + <div> + <h3 class="title"><a id="blob_create"></a>Creating BLOBs</h3> + </div> + </div> + </div> + <p> + There are two ways to create a BLOB. Before you can use + either mechanism, you must set the BLOB threshold to a + non-zero positive integer value (see the previous section + for details). Once the BLOB threshold has been set, you + create a BLOB using one of the two following mechanisms: + </p> + <div class="itemizedlist"> + <ul type="disc"> + <li> + <p> + Configure the DBT used to access the BLOB data + (that is, the DBT used for the data portion of the + record) with the <a href="../api_reference/C/dbt.html#dbt_DB_DBT_BLOB" class="olink">DB_DBT_BLOB</a> flag. This causes + the data to be stored as a BLOB regardless of its + size, so long as the database otherwise supports + BLOBs. + </p> + </li> + <li> + <p> + Alternatively, creating a data item with a size + greater than the BLOB threshold will cause that + data item to be automatically stored as a BLOB. + </p> + </li> + </ul> + </div> + </div> + <div class="sect2" lang="en" xml:lang="en"> + <div class="titlepage"> + <div> + <div> + <h3 class="title"><a id="blob_access"></a>BLOB access</h3> + </div> + </div> + </div> + <p> + BLOBs may be accessed in the same way as other DBT + data, so long as the data itself will fit into memory. + More likely, you will find it necessary to use the BLOB + streaming API to read and write BLOB data. You open a BLOB + stream using the <a href="../api_reference/C/dbstream.html" class="olink">DBC->db_stream()</a> method, close it with the + <a href="../api_reference/C/dbstream_close.html" class="olink">DB_STREAM->close()</a> method, write to it using the the + <a href="../api_reference/C/dbstream_write.html" class="olink">DB_STREAM->write()</a> method, and read it using the + <a href="../api_reference/C/dbstream_read.html" class="olink">DB_STREAM->read()</a> method. + </p> + <p> + The following example code fragment can be found in + your DB distribution at + <code class="filename">.../db/examples/c/ex_blob.c</code>. + </p> + <pre class="programlisting">... + /* Some necessary variable declarations */ + DBC *dbc; /* Cursor handle */ + DB_ENV *dbenv; /* Environment handle */ + DB *dbp; /* Database handle */ + DB_STREAM *dbs; /* Stream handle */ + DB_TXN *txn; /* Transaction handle */ + DBT data, key; /* DBT handles */ + int ret; + db_off_t size; + + ... + + /* Environment creation skipped for brevity's sake */ + + ... + + /* Enable blob files and set the size threshold. */ + if ((ret = dbenv->set_blob_threshold(dbenv, 1000, 0)) != 0) { + dbenv->err(dbenv, ret, "set_blob_threshold"); + goto err; + } + + ... + + /* Database and DBT creation skipped for brevity's sake */ + + ... + + /* + Access the BLOB using the DB_STREAM API. + */ + if ((ret = dbenv->txn_begin(dbenv, NULL, &txn, 0)) != 0){ + dbenv->err(dbenv, ret, "txn"); + goto err; + } + + if ((ret = dbp->cursor(dbp, txn, &dbc, 0)) != 0) { + dbenv->err(dbenv, ret, "cursor"); + goto err; + } + + /* + * Set the cursor to a blob. Use DB_DBT_PARTIAL with + * dlen == 0 to avoid getting any blob data. + */ + data.flags = DB_DBT_USERMEM | DB_DBT_PARTIAL; + data.dlen = 0; + if ((ret = dbc->get(dbc, &key, &data, DB_FIRST)) != 0) { + dbenv->err(dbenv, ret, "Not a blob"); + goto err; + } + data.flags = DB_DBT_USERMEM; + + /* Create a stream on the blob the cursor points to. */ + if ((ret = dbc->db_stream(dbc, &dbs, DB_STREAM_WRITE)) != 0) { + dbenv->err(dbenv, 0, "Creating stream."); + goto err; + } + + /* Get the size of the blob. */ + if ((ret = dbs->size(dbs, &size, 0)) != 0) { + dbenv->err(dbenv, 0, "Stream size."); + goto err; + } + /* Read from the blob. */ + if ((ret = dbs->read(dbs, &data, 0, (u_int32_t)size, 0)) != 0) { + dbenv->err(dbenv, 0, "Stream read."); + goto err; + } + /* Write data to the blob, increasing its size. */ + if ((ret = dbs->write(dbs, &data, size/2, 0)) != 0) { + dbenv->err(dbenv, 0, "Stream write."); + goto err; + } + /* Close the stream. */ + if ((ret = dbs->close(dbs, 0)) != 0) { + dbenv->err(dbenv, 0, "Stream close."); + goto err; + } + dbs = NULL; + dbc->close(dbc); + dbc = NULL; + txn->commit(txn, 0); + txn = NULL; + free(data.data); + data.data = NULL; + + ... + + /* Handle clean up skipped. */ </pre> + </div> + <div class="sect2" lang="en" xml:lang="en"> + <div class="titlepage"> + <div> + <div> + <h3 class="title"><a id="blob_storage"></a>BLOB storage</h3> + </div> + </div> + </div> + <p> + BLOBs are not stored in the normal database files on + disk in the same way as is other data managed by DB. + Instead, they are stored as binary files in a special + directory set aside for the purpose. + </p> + <p> + If you are not using environments, this special BLOB + directory is created relative to the current working + directory from which your application is running. You can + modify this default location using the <a href="../api_reference/C/set_blob_dir.html" class="olink">DB->set_blob_dir()</a> + method, and retrieve the current BLOB directory using + <a href="../api_reference/C/get_blob_dir.html" class="olink">DB->get_blob_dir()</a>. + </p> + <p> + If you are using an environment, then by default the + BLOB directory is created within the environment's home + directory. You can change this default location using + <a href="../api_reference/C/envset_blob_dir.html" class="olink">DB_ENV->set_blob_dir()</a> and retrieve the current default + location using <a href="../api_reference/C/envget_blob_dir.html" class="olink">DB_ENV->get_blob_dir()</a>. (Note that + <a href="../api_reference/C/envget_blob_dir.html" class="olink">DB_ENV->get_blob_dir()</a> can successfully retrieve the BLOB + directory only if <a href="../api_reference/C/envset_blob_dir.html" class="olink">DB_ENV->set_blob_dir()</a> was previously + called.) + </p> + <p> + Note that because BLOBs are stored outside of the + Berkeley DB database files, they are not confined by the + four gigabyte limit used for Berkeley DB key and data + items. The BLOB size limit is system dependent. It can be + the maximum value in bytes of a signed 32 bit integer (if + the Berkeley DB-defined type <code class="literal">db_off_t</code> + is four bytes in size), or a signed 64 bit integer (if + <code class="literal">db_off_t</code> is eight bytes in size). + </p> + </div> + <div class="sect2" lang="en" xml:lang="en"> + <div class="titlepage"> + <div> + <div> + <h3 class="title"><a id="blob_replication"></a>BLOBs and Replication</h3> + </div> + </div> + </div> + <p> + Replication supports BLOBs without any special requirements. + However, enabling BLOBs in a replicated environment can + result in long synchronization times between the client + and master sites. To avoid this, execute a transaction + checkpoint after updating or deleting one or more BLOB + records. + </p> + </div> + </div> + <div class="navfooter"> + <hr /> + <table width="100%" summary="Navigation footer"> + <tr> + <td width="40%" align="left"><a accesskey="p" href="am_misc_diskspace.html">Prev</a> </td> + <td width="20%" align="center"> + <a accesskey="u" href="am_misc.html">Up</a> + </td> + <td width="40%" align="right"> <a accesskey="n" href="am_misc_db_sql.html">Next</a></td> + </tr> + <tr> + <td width="40%" align="left" valign="top">Disk space + requirements </td> + <td width="20%" align="center"> + <a accesskey="h" href="index.html">Home</a> + </td> + <td width="40%" align="right" valign="top"> Specifying a Berkeley DB schema + using SQL DDL</td> + </tr> + </table> + </div> + </body> +</html> diff --git a/docs/programmer_reference/bt_conf.html b/docs/programmer_reference/bt_conf.html index b270901d..98774713 100644 --- a/docs/programmer_reference/bt_conf.html +++ b/docs/programmer_reference/bt_conf.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="general_am_conf.html">Prev</a> </td> - <th width="60%" align="center">Chapter 2. - Access Method Configuration - </th> + <th width="60%" align="center">Chapter 2. Access Method Configuration </th> <td width="20%" align="right"> <a accesskey="n" href="hash_conf.html">Next</a></td> </tr> </table> @@ -47,7 +45,8 @@ </dt> <dt> <span class="sect2"> - <a href="bt_conf.html#am_conf_bt_prefix">Btree prefix comparison</a> + <a href="bt_conf.html#am_conf_bt_prefix">Btree prefix + comparison</a> </span> </dt> <dt> @@ -68,9 +67,10 @@ </dl> </div> <p> - There are a series of configuration tasks which you can perform when - using the Btree access method. They are described in the following sections. -</p> + There are a series of configuration tasks which you can + perform when using the Btree access method. They are described + in the following sections. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> @@ -79,25 +79,34 @@ </div> </div> </div> - <p>The Btree data structure is a sorted, balanced tree structure storing -associated key/data pairs. By default, the sort order is lexicographical, -with shorter keys collating before longer keys. The user can specify the -sort order for the Btree by using the <a href="../api_reference/C/dbset_bt_compare.html" class="olink">DB->set_bt_compare()</a> method.</p> - <p>Sort routines are passed pointers to keys as arguments. The keys are -represented as <a href="../api_reference/C/dbt.html" class="olink">DBT</a> structures. The routine must return an integer -less than, equal to, or greater than zero if the first argument is -considered to be respectively less than, equal to, or greater than the -second argument. The only fields that the routines may examine in the -<a href="../api_reference/C/dbt.html" class="olink">DBT</a> structures are <span class="bold"><strong>data</strong></span> and <span class="bold"><strong>size</strong></span> fields.</p> - <p>An example routine that might be used to sort integer keys in the database -is as follows:</p> + <p> + The Btree data structure is a sorted, balanced tree + structure storing associated key/data pairs. By default, the + sort order is lexicographical, with shorter keys collating + before longer keys. The user can specify the sort order for + the Btree by using the <a href="../api_reference/C/dbset_bt_compare.html" class="olink">DB->set_bt_compare()</a> method. + </p> + <p> + Sort routines are passed pointers to keys as arguments. The + keys are represented as <a href="../api_reference/C/dbt.html" class="olink">DBT</a> structures. The routine must + return an integer less than, equal to, or greater than zero if + the first argument is considered to be respectively less than, + equal to, or greater than the second argument. The only fields + that the routines may examine in the <a href="../api_reference/C/dbt.html" class="olink">DBT</a> structures are + <span class="bold"><strong>data</strong></span> and <span class="bold"><strong>size</strong></span> fields. + </p> + <p> + An example routine that might be used to sort integer keys + in the database is as follows: + </p> <a id="prog_am2"></a> - <pre class="programlisting"> -int -compare_int(DB *dbp, const DBT *a, const DBT *b) + <pre class="programlisting">int +compare_int(DB *dbp, const DBT *a, const DBT *b, size_t *locp) { int ai, bi; + + locp = NULL; /* * Returns: * < 0 if a < b @@ -109,23 +118,29 @@ compare_int(DB *dbp, const DBT *a, const DBT *b) return (ai - bi); } </pre> - <p>Note that the data must first be copied into memory that is appropriately -aligned, as Berkeley DB does not guarantee any kind of alignment of the -underlying data, including for comparison routines. When writing -comparison routines, remember that databases created on machines of -different architectures may have different integer byte orders, for which -your code may need to compensate.</p> - <p>An example routine that might be used to sort keys based on the first -five bytes of the key (ignoring any subsequent bytes) is as follows:</p> + <p> + Note that the data must first be copied into memory that is + appropriately aligned, as Berkeley DB does not guarantee any + kind of alignment of the underlying data, including for + comparison routines. When writing comparison routines, + remember that databases created on machines of different + architectures may have different integer byte orders, for + which your code may need to compensate. + </p> + <p> + An example routine that might be used to sort keys based on + the first five bytes of the key (ignoring any subsequent + bytes) is as follows: + </p> <a id="prog_am3"></a> - <pre class="programlisting"> -int -compare_dbt(DB *dbp, const DBT *a, const DBT *b) + <pre class="programlisting">int +compare_dbt(DB *dbp, const DBT *a, const DBT *b, size_t *locp) { int len; u_char *p1, *p2; + locp = NULL; /* * Returns: * < 0 if a < b @@ -138,52 +153,71 @@ compare_dbt(DB *dbp, const DBT *a, const DBT *b) return (0); } </pre> - <p>All comparison functions must cause the keys in the database to be -well-ordered. The most important implication of being well-ordered is -that the key relations must be transitive, that is, if key A is less -than key B, and key B is less than key C, then the comparison routine -must also return that key A is less than key C.</p> - <p>It is reasonable for a comparison function to not examine an entire key -in some applications, which implies partial keys may be specified to the -Berkeley DB interfaces. When partial keys are specified to Berkeley DB, interfaces -which retrieve data items based on a user-specified key (for example, -<a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> and <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> with the <a href="../api_reference/C/dbcget.html#dbcget_DB_SET" class="olink">DB_SET</a> flag), will -modify the user-specified key by returning the actual key stored in the -database.</p> + <p> + All comparison functions must cause the keys in the database + to be well-ordered. The most important implication of being + well-ordered is that the key relations must be transitive, + that is, if key A is less than key B, and key B is less than + key C, then the comparison routine must also return that key A + is less than key C. + </p> + <p> + It is reasonable for a comparison function to not examine an + entire key in some applications, which implies partial keys + may be specified to the Berkeley DB interfaces. When partial + keys are specified to Berkeley DB, interfaces which retrieve + data items based on a user-specified key (for example, <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> + and <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> with the <a href="../api_reference/C/dbcget.html#dbcget_DB_SET" class="olink">DB_SET</a> flag), will modify the + user-specified key by returning the actual key stored in the + database. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="am_conf_bt_prefix"></a>Btree prefix comparison</h3> + <h3 class="title"><a id="am_conf_bt_prefix"></a>Btree prefix + comparison</h3> </div> </div> </div> - <p>The Berkeley DB Btree implementation maximizes the number of keys that can be -stored on an internal page by storing only as many bytes of each key as -are necessary to distinguish it from adjacent keys. The prefix -comparison routine is what determines this minimum number of bytes (that -is, the length of the unique prefix), that must be stored. A prefix -comparison function for the Btree can be specified by calling -<a href="../api_reference/C/dbset_bt_prefix.html" class="olink">DB->set_bt_prefix()</a>.</p> - <p>The prefix comparison routine must be compatible with the overall -comparison function of the Btree, since what distinguishes any two keys -depends entirely on the function used to compare them. This means that -if a prefix comparison routine is specified by the application, a -compatible overall comparison routine must also have been specified.</p> - <p>Prefix comparison routines are passed pointers to keys as arguments. -The keys are represented as <a href="../api_reference/C/dbt.html" class="olink">DBT</a> structures. The only fields -the routines may examine in the <a href="../api_reference/C/dbt.html" class="olink">DBT</a> structures are <span class="bold"><strong>data</strong></span> -and <span class="bold"><strong>size</strong></span> fields.</p> - <p>The prefix comparison function must return the number of bytes necessary -to distinguish the two keys. If the keys are identical (equal and equal -in length), the length should be returned. If the keys are equal up to -the smaller of the two lengths, then the length of the smaller key plus -1 should be returned.</p> - <p>An example prefix comparison routine follows:</p> + <p> + The Berkeley DB Btree implementation maximizes the number of + keys that can be stored on an internal page by storing only as + many bytes of each key as are necessary to distinguish it from + adjacent keys. The prefix comparison routine is what + determines this minimum number of bytes (that is, the length + of the unique prefix), that must be stored. A prefix + comparison function for the Btree can be specified by calling + <a href="../api_reference/C/dbset_bt_prefix.html" class="olink">DB->set_bt_prefix()</a>. + </p> + <p> + The prefix comparison routine must be compatible with the + overall comparison function of the Btree, since what + distinguishes any two keys depends entirely on the function + used to compare them. This means that if a prefix comparison + routine is specified by the application, a compatible overall + comparison routine must also have been specified. + </p> + <p> + Prefix comparison routines are passed pointers to keys as + arguments. The keys are represented as <a href="../api_reference/C/dbt.html" class="olink">DBT</a> structures. The + only fields the routines may examine in the <a href="../api_reference/C/dbt.html" class="olink">DBT</a> structures + are <span class="bold"><strong>data</strong></span> and <span class="bold"><strong>size</strong></span> fields. + </p> + <p> + The prefix comparison function must return the number of + bytes necessary to distinguish the two keys. If the keys are + identical (equal and equal in length), the length should be + returned. If the keys are equal up to the smaller of the two + lengths, then the length of the smaller key plus 1 should be + returned. + </p> + <p> + An example prefix comparison routine follows: + </p> <a id="prog_am4"></a> - <pre class="programlisting"> -size_t + <pre class="programlisting">size_t compare_prefix(DB *dbp, const DBT *a, const DBT *b) { @@ -207,8 +241,11 @@ compare_prefix(DB *dbp, const DBT *a, const DBT *b) return (b->size); } </pre> - <p>The usefulness of this functionality is data-dependent, but in some data -sets can produce significantly reduced tree sizes and faster search times.</p> + <p> + The usefulness of this functionality is data-dependent, but + in some data sets can produce significantly reduced tree sizes + and faster search times. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> @@ -218,38 +255,58 @@ sets can produce significantly reduced tree sizes and faster search times.</p> </div> </div> </div> - <p>The number of keys stored on each page affects the size of a Btree and -how it is maintained. Therefore, it also affects the retrieval and search -performance of the tree. For each Btree, Berkeley DB computes a maximum key -and data size. This size is a function of the page size and the fact that -at least two key/data pairs must fit on any Btree page. Whenever key or -data items exceed the calculated size, they are stored on overflow pages -instead of in the standard Btree leaf pages.</p> - <p>Applications may use the <a href="../api_reference/C/dbset_bt_minkey.html" class="olink">DB->set_bt_minkey()</a> method to change the minimum -number of keys that must fit on a Btree page from two to another value. -Altering this value in turn alters the on-page maximum size, and can be -used to force key and data items which would normally be stored in the -Btree leaf pages onto overflow pages.</p> - <p>Some data sets can benefit from this tuning. For example, consider an -application using large page sizes, with a data set almost entirely -consisting of small key and data items, but with a few large items. By -setting the minimum number of keys that must fit on a page, the -application can force the outsized items to be stored on overflow pages. -That in turn can potentially keep the tree more compact, that is, with -fewer internal levels to traverse during searches.</p> - <p>The following calculation is similar to the one performed by the Btree -implementation. (The <span class="bold"><strong>minimum_keys</strong></span> value is multiplied by 2 -because each key/data pair requires 2 slots on a Btree page.)</p> + <p> + The number of keys stored on each page affects the size of a + Btree and how it is maintained. Therefore, it also affects the + retrieval and search performance of the tree. For each Btree, + Berkeley DB computes a maximum key and data size. This size is + a function of the page size and the fact that at least two + key/data pairs must fit on any Btree page. Whenever key or + data items exceed the calculated size, they are stored on + overflow pages instead of in the standard Btree leaf + pages. + </p> + <p> + Applications may use the <a href="../api_reference/C/dbset_bt_minkey.html" class="olink">DB->set_bt_minkey()</a> method to change + the minimum number of keys that must fit on a Btree page from + two to another value. Altering this value in turn alters the + on-page maximum size, and can be used to force key and data + items which would normally be stored in the Btree leaf pages + onto overflow pages. + </p> + <p> + Some data sets can benefit from this tuning. For example, + consider an application using large page sizes, with a data + set almost entirely consisting of small key and data items, + but with a few large items. By setting the minimum number of + keys that must fit on a page, the application can force the + outsized items to be stored on overflow pages. That in turn + can potentially keep the tree more compact, that is, with + fewer internal levels to traverse during searches. + </p> + <p> + The following calculation is similar to the one performed by + the Btree implementation. (The <span class="bold"><strong>minimum_keys + </strong></span> value is multiplied by 2 because + each key/data pair requires 2 slots on a Btree page.) + </p> <pre class="programlisting">maximum_size = page_size / (minimum_keys * 2)</pre> - <p>Using this calculation, if the page size is 8KB and the default -<span class="bold"><strong>minimum_keys</strong></span> value of 2 is used, then any key or data items -larger than 2KB will be forced to an overflow page. If an application -were to specify a <span class="bold"><strong>minimum_key</strong></span> value of 100, then any key or data -items larger than roughly 40 bytes would be forced to overflow pages.</p> - <p>It is important to remember that accesses to overflow pages do not perform -as well as accesses to the standard Btree leaf pages, and so setting the -value incorrectly can result in overusing overflow pages and decreasing -the application's overall performance.</p> + <p> + Using this calculation, if the page size is 8KB and the + default <span class="bold"><strong>minimum_keys</strong></span> value of + 2 is used, then any key or data items larger than 2KB will be + forced to an overflow page. If an application were to specify + a <span class="bold"><strong>minimum_key</strong></span> value of 100, + then any key or data items larger than roughly 40 bytes would + be forced to overflow pages. + </p> + <p> + It is important to remember that accesses to overflow pages + do not perform as well as accesses to the standard Btree leaf + pages, and so setting the value incorrectly can result in + overusing overflow pages and decreasing the application's + overall performance. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> @@ -259,21 +316,28 @@ the application's overall performance.</p> </div> </div> </div> - <p>The Btree access method optionally supports retrieval by logical record -numbers. To configure a Btree to support record numbers, call the -<a href="../api_reference/C/dbset_flags.html" class="olink">DB->set_flags()</a> method with the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RECNUM" class="olink">DB_RECNUM</a> flag.</p> - <p>Configuring a Btree for record numbers should not be done lightly. -While often useful, it may significantly slow down the speed at which -items can be stored into the database, and can severely impact -application throughput. Generally it should be avoided in trees with -a need for high write concurrency.</p> - <p>To retrieve by record number, use the <a href="../api_reference/C/dbget.html#dbget_DB_SET_RECNO" class="olink">DB_SET_RECNO</a> flag to the -<a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> and <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> methods. The following is an example of -a routine that displays the data item for a Btree database created with -the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RECNUM" class="olink">DB_RECNUM</a> option.</p> + <p> + The Btree access method optionally supports retrieval by + logical record numbers. To configure a Btree to support record + numbers, call the <a href="../api_reference/C/dbset_flags.html" class="olink">DB->set_flags()</a> method with the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RECNUM" class="olink">DB_RECNUM</a> + flag. + </p> + <p> + Configuring a Btree for record numbers should not be done + lightly. While often useful, it may significantly slow down + the speed at which items can be stored into the database, and + can severely impact application throughput. Generally it + should be avoided in trees with a need for high write + concurrency. + </p> + <p> + To retrieve by record number, use the <a href="../api_reference/C/dbget.html#dbget_DB_SET_RECNO" class="olink">DB_SET_RECNO</a> flag + to the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> and <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> methods. The following is an + example of a routine that displays the data item for a Btree + database created with the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RECNUM" class="olink">DB_RECNUM</a> option. + </p> <a id="prog_am5"></a> - <pre class="programlisting"> -int + <pre class="programlisting">int rec_display(DB *dbp, db_recno_t recno) { @@ -292,12 +356,14 @@ rec_display(DB *dbp, db_recno_t recno) return (0); } </pre> - <p>To determine a key's record number, use the <a href="../api_reference/C/dbcget.html#dbcget_DB_GET_RECNO" class="olink">DB_GET_RECNO</a> flag -to the <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method. The following is an example of a routine that -displays the record number associated with a specific key.</p> + <p> + To determine a key's record number, use the <a href="../api_reference/C/dbcget.html#dbcget_DB_GET_RECNO" class="olink">DB_GET_RECNO</a> + flag to the <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method. The following is an example of a + routine that displays the record number associated with a + specific key. + </p> <a id="prog_am6"></a> - <pre class="programlisting"> -int + <pre class="programlisting">int recno_display(DB *dbp, char *keyvalue) { @@ -355,55 +421,59 @@ err: /* Close the cursor. */ </div> </div> </div> - <p> - The Btree access method supports the automatic compression of key/data - pairs upon their insertion into the database. The key/data pairs are - decompressed before they are returned to the application, making an - application's interaction with a compressed database identical to that - for a non-compressed database. To configure Berkeley DB for - compression, call the <a href="../api_reference/C/dbset_bt_compress.html" class="olink">DB->set_bt_compress()</a> method and specify custom - compression and decompression functions. If <a href="../api_reference/C/dbset_bt_compress.html" class="olink">DB->set_bt_compress()</a> is - called with NULL compression and decompression functions, Berkeley DB - will use its default compression functions. -</p> + <p> + The Btree access method supports the automatic compression + of key/data pairs upon their insertion into the database. The + key/data pairs are decompressed before they are returned to + the application, making an application's interaction with a + compressed database identical to that for a non-compressed + database. To configure Berkeley DB for compression, call the + <a href="../api_reference/C/dbset_bt_compress.html" class="olink">DB->set_bt_compress()</a> method and specify custom compression and + decompression functions. If <a href="../api_reference/C/dbset_bt_compress.html" class="olink">DB->set_bt_compress()</a> is called with + NULL compression and decompression functions, Berkeley DB will + use its default compression functions. + </p> <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> <h3 class="title">Note</h3> - <p> - Compression only works with the Btree access method, and then only - so long as your database is not configured for unsorted duplicates. - </p> + <p> + Compression only works with the Btree access method, + and then only so long as your database is not configured + for unsorted duplicates. + </p> </div> <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> <h3 class="title">Note</h3> - <p> - The default compression function is not guaranteed to reduce the - size of the on-disk database in every case. It has been tested - and shown to work well with English-language text. Of course, in - order to determine if the default compression algorithm is beneficial - for your application, it is important to test both the final size - and the performance using a representative set of data and access - patterns. - </p> + <p> + The default compression function is not guaranteed to + reduce the size of the on-disk database in every case. It + has been tested and shown to work well with + English-language text. Of course, in order to determine if + the default compression algorithm is beneficial for your + application, it is important to test both the final size + and the performance using a representative set of data and + access patterns. + </p> </div> <p> - The default compression function performs prefix compression on each - key added to the database. This means that, for a key - <span class="emphasis"><em>n</em></span> bytes in length, the first - <span class="emphasis"><em>i</em></span> bytes that match the first - <span class="emphasis"><em>i</em></span> bytes of the previous key exactly are omitted - and only the final <span class="emphasis"><em>n-i</em></span> bytes are stored in the - database. If the bytes of key being stored match the bytes of the - previous key exactly, then the same prefix compression algorithm is - applied to the data value being stored. To use Berkeley DB's default - compression behavior, both the default compression and decompression - functions must be used. -</p> + The default compression function performs prefix + compression on each key added to the database. This means + that, for a key <span class="emphasis"><em>n</em></span> bytes in length, the + first <span class="emphasis"><em>i</em></span> bytes that match the first + <span class="emphasis"><em>i</em></span> bytes of the previous key exactly + are omitted and only the final <span class="emphasis"><em>n-i</em></span> bytes + are stored in the database. If the bytes of key being stored + match the bytes of the previous key exactly, then the same + prefix compression algorithm is applied to the data value + being stored. To use Berkeley DB's default compression + behavior, both the default compression and decompression + functions must be used. + </p> <p> - For example, to configure your database for default compression: -</p> + For example, to configure your database for default + compression: + </p> <a id="prog_am7"></a> - <pre class="programlisting"> - DB *dbp = NULL; + <pre class="programlisting">DB *dbp = NULL; DB_ENV *envp = NULL; u_int32_t db_flags; const char *file_name = "mydb.db"; @@ -441,60 +511,64 @@ err: /* Close the cursor. */ <div class="titlepage"> <div> <div> - <h4 class="title"><a id="am_conf_bt_custom_compress"></a>Custom compression</h4> + <h4 class="title"><a id="am_conf_bt_custom_compress"></a>Custom + compression</h4> </div> </div> </div> <p> - An application wishing to perform its own compression may supply a - compression and decompression function which will be called instead of - Berkeley DB's default functions. The compression function is - passed five <a href="../api_reference/C/dbt.html" class="olink">DBT</a> structures: -</p> + An application wishing to perform its own compression + may supply a compression and decompression function which + will be called instead of Berkeley DB's default functions. + The compression function is passed five <a href="../api_reference/C/dbt.html" class="olink">DBT</a> structures: + </p> <div class="itemizedlist"> <ul type="disc"> <li> - <p> - The key and data immediately preceeding the key/data pair - that is being stored. - </p> + <p> + The key and data immediately preceeding the + key/data pair that is being stored. + </p> </li> <li> <p> - The key and data being stored in the tree. - </p> + The key and data being stored in the tree. + </p> </li> <li> - <p> - The buffer where the compressed data should be written. - </p> + <p> + The buffer where the compressed data should be + written. + </p> </li> </ul> </div> + <p> + The total size of the buffer used to store the + compressed data is identified in the <a href="../api_reference/C/dbt.html" class="olink">DBT</a>'s + <code class="literal">ulen</code> field. If the compressed data + cannot fit in the buffer, the compression function should + store the amount of space needed in <a href="../api_reference/C/dbt.html" class="olink">DBT</a>'s + <code class="literal">size</code> field and then return + <code class="literal">DB_BUFFER_SMALL</code>. Berkeley DB will + subsequently re-call the compression function with the + required amount of space allocated in the compression data + buffer. + </p> + <p> + Multiple compressed key/data pairs will likely be + written to the same buffer and the compression function + should take steps to ensure it does not overwrite data. + </p> <p> - The total size of the buffer used to store the compressed data is - identified in the <a href="../api_reference/C/dbt.html" class="olink">DBT</a>'s <code class="literal">ulen</code> field. If the - compressed data cannot fit in the buffer, the compression function - should store the amount of space needed in <a href="../api_reference/C/dbt.html" class="olink">DBT</a>'s - <code class="literal">size</code> field and then return - <code class="literal">DB_BUFFER_SMALL</code>. Berkeley DB will subsequently - re-call the compression function with the required amount of space - allocated in the compression data buffer. - </p> - <p> - Multiple compressed key/data pairs will likely be written to the - same buffer and the compression function should take steps to - ensure it does not overwrite data. - </p> - <p> - For example, the following code fragments illustrate the use of a custom - compression routine. This code is actually a much simplified - example of the default compression provided by Berkeley DB. It does - simple prefix compression on the key part of the data. - </p> + For example, the following code fragments illustrate + the use of a custom compression routine. This code is + actually a much simplified example of the default + compression provided by Berkeley DB. It does simple prefix + compression on the key part of the data. + </p> <a id="prog_am8"></a> - <pre class="programlisting"> - int compress(DB *dbp, const DBT *prevKey, const DBT *prevData, + <pre class="programlisting">int compress(DB *dbp, const DBT *prevKey, const DBT *prevData, const DBT *key, const DBT *data, DBT *dest) { u_int8_t *dest_data_ptr; @@ -538,49 +612,53 @@ err: /* Close the cursor. */ return (0); } </pre> <p> - The corresponding decompression function is likewise passed five <a href="../api_reference/C/dbt.html" class="olink">DBT</a> structures: - </p> + The corresponding decompression function is likewise + passed five <a href="../api_reference/C/dbt.html" class="olink">DBT</a> structures: + </p> <div class="itemizedlist"> <ul type="disc"> <li> - <p> - The key and data <a href="../api_reference/C/dbt.html" class="olink">DBT</a>s immediately preceding the - decompressed key and data. - </p> + <p> + The key and data <a href="../api_reference/C/dbt.html" class="olink">DBT</a>s immediately preceding + the decompressed key and data. + </p> </li> <li> - <p> - The compressed data from the database. - </p> + <p> + The compressed data from the database. + </p> </li> <li> - <p> - One to store the decompressed key and another one for the - decompressed data. - </p> + <p> + One to store the decompressed key and another + one for the decompressed data. + </p> </li> </ul> </div> + <p> + Because the compression of <code class="literal">record X</code> + relies upon <code class="literal">record X-1</code>, the + decompression function can be called repeatedly to + linearally decompress a set of records stored in the + compressed buffer. + </p> <p> - Because the compression of <code class="literal">record X</code> relies upon - <code class="literal">record X-1</code>, the decompression function can be - called repeatedly to linearally decompress a set of records stored - in the compressed buffer. - </p> - <p> - The total size of the buffer available to store the decompressed data is - identified in the destination <a href="../api_reference/C/dbt.html" class="olink">DBT</a>'s <code class="literal">ulen</code> field. If the - decompressed data cannot fit in the buffer, the decompression function - should store the amount of space needed in the destination <a href="../api_reference/C/dbt.html" class="olink">DBT</a>'s - <code class="literal">size</code> field and then return - <code class="literal">DB_BUFFER_SMALL</code>. Berkeley DB will subsequently - re-call the decompression function with the required amount of space - allocated in the decompression data buffer. - </p> + The total size of the buffer available to store the + decompressed data is identified in the destination <a href="../api_reference/C/dbt.html" class="olink">DBT</a>'s + <code class="literal">ulen</code> field. If the decompressed + data cannot fit in the buffer, the decompression function + should store the amount of space needed in the destination + <a href="../api_reference/C/dbt.html" class="olink">DBT</a>'s <code class="literal">size</code> field and then return + <code class="literal">DB_BUFFER_SMALL</code>. Berkeley DB will + subsequently re-call the decompression function with the + required amount of space allocated in the decompression + data buffer. + </p> <p> - For example, the decompression routine that corresponds to the - example compression routine provided above is: -</p> + For example, the decompression routine that corresponds + to the example compression routine provided above is: + </p> <a id="prog_am9"></a> <pre class="programlisting">int decompress(DB *dbp, const DBT *prevKey, const DBT *prevData, DBT *compressed, DBT *destKey, DBT *destData) @@ -642,46 +720,49 @@ err: /* Close the cursor. */ <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idm1755952"></a>Programmer Notes</h4> + <h4 class="title"><a id="idm2102248"></a>Programmer Notes</h4> </div> </div> </div> - <p> - As you use compression with your databases, be aware of the - following: - </p> + <p> + As you use compression with your databases, be aware of + the following: + </p> <div class="itemizedlist"> <ul type="disc"> <li> - <p> - Compression works by placing key/data pairs from a single - database page into a single block of compressed data. This is true - whether you use DB's default compression, or you write - your own compression. Because all of key/data data is - placed in a single block of memory, you cannot decompress - data unless you have decompressed everything that came - before it in the block. That is, you cannot decompress item - <span class="emphasis"><em>n</em></span> in the data block, unless you also - decompress items <span class="emphasis"><em>0</em></span> through - <span class="emphasis"><em>n-1</em></span>. - </p> + <p> + Compression works by placing key/data pairs + from a single database page into a single block of + compressed data. This is true whether you use + DB's default compression, or you write your + own compression. Because all of key/data data is + placed in a single block of memory, you cannot + decompress data unless you have decompressed + everything that came before it in the block. That + is, you cannot decompress item + <span class="emphasis"><em>n</em></span> in the data block, + unless you also decompress items + <span class="emphasis"><em>0</em></span> through + <span class="emphasis"><em>n-1</em></span>. + </p> </li> <li> <p> - If you increase the minimum number of key/data pairs placed - on a Btree leaf page (using <a href="../api_reference/C/dbset_bt_minkey.html" class="olink">DB->set_bt_minkey()</a>), you will - decrease your seek times on a compressed database. However, - this will also decrease the effectiveness of the - compression. - </p> + If you increase the minimum number of key/data + pairs placed on a Btree leaf page (using + <a href="../api_reference/C/dbset_bt_minkey.html" class="olink">DB->set_bt_minkey()</a>), you will decrease your seek + times on a compressed database. However, this will + also decrease the effectiveness of the + compression. + </p> </li> <li> - <p> - Compressed databases are fastest if bulk load is used to - add data to them. See - <a class="xref" href="am_misc_bulk.html" title="Retrieving and updating records in bulk">Retrieving and updating records in bulk</a> - for information on using bulk load. - </p> + <p> + Compressed databases are fastest if bulk load + is used to add data to them. See <a class="xref" href="am_misc_bulk.html" title="Retrieving and updating records in bulk">Retrieving and updating records in bulk</a> for information + on using bulk load. + </p> </li> </ul> </div> @@ -703,7 +784,8 @@ err: /* Close the cursor. */ <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Hash access method specific configuration</td> + <td width="40%" align="right" valign="top"> Hash access method specific + configuration</td> </tr> </table> </div> diff --git a/docs/programmer_reference/cam.html b/docs/programmer_reference/cam.html index 5591dcf4..8110199d 100644 --- a/docs/programmer_reference/cam.html +++ b/docs/programmer_reference/cam.html @@ -14,13 +14,12 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 10. - Berkeley DB Concurrent Data Store Applications - </th> + <th colspan="3" align="center">Chapter 10. Berkeley DB Concurrent Data Store + Applications </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="env_faq.html">Prev</a> </td> @@ -34,9 +33,8 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="cam"></a>Chapter 10. - Berkeley DB Concurrent Data Store Applications - </h2> + <h2 class="title"><a id="cam"></a>Chapter 10. Berkeley DB Concurrent Data Store + Applications </h2> </div> </div> </div> @@ -47,7 +45,8 @@ <dl> <dt> <span class="sect1"> - <a href="cam.html#cam_intro">Concurrent Data Store introduction</a> + <a href="cam.html#cam_intro">Concurrent Data Store + introduction</a> </span> </dt> <dt> @@ -66,110 +65,161 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="cam_intro"></a>Concurrent Data Store introduction</h2> + <h2 class="title" style="clear: both"><a id="cam_intro"></a>Concurrent Data Store + introduction</h2> </div> </div> </div> - <p>It is often desirable to have concurrent read-write access to a database -when there is no need for full recoverability or transaction semantics. -For this class of applications, Berkeley DB provides interfaces supporting -deadlock-free, multiple-reader/single writer access to the database. -This means that at any instant in time, there may be either multiple -readers accessing data or a single writer modifying data. The -application is entirely unaware of which is happening, and Berkeley DB -implements the necessary locking and blocking to ensure this behavior.</p> - <p>To create Berkeley DB Concurrent Data Store applications, you must first initialize an environment -by calling <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a>. You must specify the <a href="../api_reference/C/envopen.html#envopen_DB_INIT_CDB" class="olink">DB_INIT_CDB</a> and <a href="../api_reference/C/envopen.html#envopen_DB_INIT_MPOOL" class="olink">DB_INIT_MPOOL</a> -flags to that method. It is an error to -specify any of the other <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> subsystem or recovery -configuration flags, for example, <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOCK" class="olink">DB_INIT_LOCK</a>, <a href="../api_reference/C/envopen.html#envopen_DB_INIT_TXN" class="olink">DB_INIT_TXN</a> or <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> -All databases must, of -course, be created in this environment by using the <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a> -function or <a href="../api_reference/CXX/db.html" class="olink">Db</a> constructor, and specifying the environment -as an argument.</p> - <p>Berkeley DB performs appropriate locking so that safe enforcement of the -deadlock-free, multiple-reader/single-writer semantic is transparent to -the application. However, a basic understanding of Berkeley DB Concurrent Data Store locking -behavior is helpful when writing Berkeley DB Concurrent Data Store applications.</p> - <p>Berkeley DB Concurrent Data Store -avoids deadlocks without the need for a deadlock detector by performing -all locking on an entire database at once (or on an entire environment -in the case of the <a href="../api_reference/C/envset_flags.html#set_flags_DB_CDB_ALLDB" class="olink">DB_CDB_ALLDB</a> flag), and by ensuring that at -any given time only one thread of control is allowed to simultaneously -hold a read (shared) lock and attempt to acquire a write (exclusive) -lock.</p> - <p>All open Berkeley DB cursors hold a read lock, which serves as a guarantee -that the database will not change beneath them; likewise, all -non-cursor <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> operations temporarily acquire and release -a read lock that is held during the actual traversal of the database. -Because read locks will not conflict with each other, any number of -cursors in any number of threads of control may be open simultaneously, -and any number of <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> operations may be concurrently in -progress.</p> - <p>To enforce the rule that only one thread of control at a time can -attempt to upgrade a read lock to a write lock, however, Berkeley DB must -forbid multiple cursors from attempting to write concurrently. This is -done using the <a href="../api_reference/C/dbcursor.html#cursor_DB_WRITECURSOR" class="olink">DB_WRITECURSOR</a> flag to the <a href="../api_reference/C/dbcursor.html" class="olink">DB->cursor()</a> method. -This is the only difference between access method calls in Berkeley DB Concurrent Data Store and -in the other Berkeley DB products. The <a href="../api_reference/C/dbcursor.html#cursor_DB_WRITECURSOR" class="olink">DB_WRITECURSOR</a> flag causes the -newly created cursor to be a "write" cursor; that is, a cursor capable -of performing writes as well as reads. Only cursors thus created are -permitted to perform write operations (either deletes or puts), and only -one such cursor can exist at any given time.</p> - <p>Any attempt to create a second write cursor or to perform a non-cursor -write operation while a write cursor is open will block until that write -cursor is closed. Read cursors may open and perform reads without blocking -while a write cursor is extant. However, any attempts to actually perform -a write, either using the write cursor or directly using the -<a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> or <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a> methods, will block until all read cursors -are closed. This is how the multiple-reader/single-writer semantic is -enforced, and prevents reads from seeing an inconsistent database state -that may be an intermediate stage of a write operation.</p> - <p>By default, Berkeley DB Concurrent Data Store does locking on a per-database basis. For this -reason, using cursors to access multiple databases in different orders -in different threads or processes, or leaving cursors open on one -database while accessing another database, can cause an application to -hang. If this behavior is a requirement for the application, Berkeley DB -should be configured to do locking on an environment-wide basis. See -the <a href="../api_reference/C/envset_flags.html#set_flags_DB_CDB_ALLDB" class="olink">DB_CDB_ALLDB</a> flag of the <a href="../api_reference/C/envset_flags.html" class="olink">DB_ENV->set_flags()</a> method -for more information.</p> - <p>With these behaviors, Berkeley DB can guarantee deadlock-free concurrent -database access, so that multiple threads of control are free to perform -reads and writes without needing to handle synchronization themselves -or having to run a deadlock detector. Berkeley DB has no direct knowledge of -which cursors belong to which threads, so some care must be taken to -ensure that applications do not inadvertently block themselves, causing -the application to hang and be unable to proceed.</p> - <p>As a consequence of the Berkeley DB Concurrent Data Store locking model, the following sequences -of operations will cause a thread to block itself indefinitely:</p> + <p> + It is often desirable to have concurrent read-write access + to a database when there is no need for full recoverability or + transaction semantics. For this class of applications, + Berkeley DB provides interfaces supporting deadlock-free, + multiple-reader/single writer access to the database. This + means that at any instant in time, there may be either + multiple readers accessing data or a single writer modifying + data. The application is entirely unaware of which is + happening, and Berkeley DB implements the necessary locking + and blocking to ensure this behavior. + </p> + <p> + To create Berkeley DB Concurrent Data Store applications, + you must first initialize an environment by calling <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a>. + You must specify the <a href="../api_reference/C/envopen.html#envopen_DB_INIT_CDB" class="olink">DB_INIT_CDB</a> and <a href="../api_reference/C/envopen.html#envopen_DB_INIT_MPOOL" class="olink">DB_INIT_MPOOL</a> flags + to that method. It is an error to specify any of the other + <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> subsystem or recovery configuration flags, for + example, <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOCK" class="olink">DB_INIT_LOCK</a>, <a href="../api_reference/C/envopen.html#envopen_DB_INIT_TXN" class="olink">DB_INIT_TXN</a> or <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> All + databases must, of course, be created in this environment by + using the <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a> function or <a href="../api_reference/CXX/db.html" class="olink">Db</a> constructor, + and specifying the environment as an argument. + </p> + <p> + Berkeley DB performs appropriate locking so that safe + enforcement of the deadlock-free, + multiple-reader/single-writer semantic is transparent to the + application. However, a basic understanding of Berkeley DB + Concurrent Data Store locking behavior is helpful when writing + Berkeley DB Concurrent Data Store applications. + </p> + <p> + Berkeley DB Concurrent Data Store avoids deadlocks without + the need for a deadlock detector by performing all locking on + an entire database at once (or on an entire environment in the + case of the <a href="../api_reference/C/envset_flags.html#set_flags_DB_CDB_ALLDB" class="olink">DB_CDB_ALLDB</a> flag), and by ensuring that at any + given time only one thread of control is allowed to + simultaneously hold a read (shared) lock and attempt to + acquire a write (exclusive) lock. + </p> + <p> + All open Berkeley DB cursors hold a read lock, which serves + as a guarantee that the database will not change beneath them; + likewise, all non-cursor <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> operations temporarily + acquire and release a read lock that is held during the actual + traversal of the database. Because read locks will not + conflict with each other, any number of cursors in any number + of threads of control may be open simultaneously, and any + number of <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> operations may be concurrently in + progress. + </p> + <p> + To enforce the rule that only one thread of control at a + time can attempt to upgrade a read lock to a write lock, + however, Berkeley DB must forbid multiple cursors from + attempting to write concurrently. This is done using the + <a href="../api_reference/C/dbcursor.html#cursor_DB_WRITECURSOR" class="olink">DB_WRITECURSOR</a> flag to the <a href="../api_reference/C/dbcursor.html" class="olink">DB->cursor()</a> method. This is the + only difference between access method calls in Berkeley DB + Concurrent Data Store and in the other Berkeley DB products. + The <a href="../api_reference/C/dbcursor.html#cursor_DB_WRITECURSOR" class="olink">DB_WRITECURSOR</a> flag causes the newly created cursor to + be a "write" cursor; that is, a cursor capable of performing + writes as well as reads. Only cursors thus created are + permitted to perform write operations (either deletes or + puts), and only one such cursor can exist at any given + time. + </p> + <p> + Any attempt to create a second write cursor or to perform a + non-cursor write operation while a write cursor is open will + block until that write cursor is closed. Read cursors may open + and perform reads without blocking while a write cursor is + extant. However, any attempts to actually perform a write, + either using the write cursor or directly using the <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> or + <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a> methods, will block until all read cursors are closed. + This is how the multiple-reader/single-writer semantic is + enforced, and prevents reads from seeing an inconsistent + database state that may be an intermediate stage of a write + operation. + </p> + <p> + By default, Berkeley DB Concurrent Data Store does locking + on a per-database basis. For this reason, using cursors to + access multiple databases in different orders in different + threads or processes, or leaving cursors open on one database + while accessing another database, can cause an application to + hang. If this behavior is a requirement for the application, + Berkeley DB should be configured to do locking on an + environment-wide basis. See the <a href="../api_reference/C/envset_flags.html#set_flags_DB_CDB_ALLDB" class="olink">DB_CDB_ALLDB</a> flag of the + <a href="../api_reference/C/envset_flags.html" class="olink">DB_ENV->set_flags()</a> method for more information. + </p> + <p> + With these behaviors, Berkeley DB can guarantee + deadlock-free concurrent database access, so that multiple + threads of control are free to perform reads and writes + without needing to handle synchronization themselves or having + to run a deadlock detector. Berkeley DB has no direct + knowledge of which cursors belong to which threads, so some + care must be taken to ensure that applications do not + inadvertently block themselves, causing the application to + hang and be unable to proceed. + </p> + <p> + As a consequence of the Berkeley DB Concurrent Data Store + locking model, the following sequences of operations will + cause a thread to block itself indefinitely: + </p> <div class="orderedlist"> <ol type="1"> - <li>Keeping a cursor open while issuing a <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> or <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a> -access method call.</li> - <li>Attempting to open a write cursor while another cursor is already being -held open by the same thread of control. Note that it is correct -operation for one thread of control to attempt to open a write cursor -or to perform a non-cursor write (<a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> or <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a>) -while a write cursor is already active in another thread. It is only a -problem if these things are done within a single thread of control -- -in which case that thread will block and never be able to release the -lock that is blocking it.</li> - <li>Not testing Berkeley DB error return codes (if any cursor operation returns -an unexpected error, that cursor must still be closed).</li> + <li> + Keeping a cursor open while issuing a <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> or + <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a> access method call. + </li> + <li> + Attempting to open a write cursor while another + cursor is already being held open by the same thread of + control. Note that it is correct operation for one thread + of control to attempt to open a write cursor or to perform + a non-cursor write (<a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> or <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a>) while a write + cursor is already active in another thread. It is only a + problem if these things are done within a single thread of + control -- in which case that thread will block and never + be able to release the lock that is blocking + it. + </li> + <li> + Not testing Berkeley DB error return codes (if any + cursor operation returns an unexpected error, that cursor + must still be closed). + </li> </ol> </div> - <p>If the application needs to open multiple cursors in a single thread to -perform an operation, it can indicate to Berkeley DB that the cursor locks -should not block each other by creating a Berkeley DB Concurrent Data Store <span class="bold"><strong>group</strong></span>, using -<a href="../api_reference/C/envcdsgroup_begin.html" class="olink">DB_ENV->cdsgroup_begin()</a>. This creates a locker ID that is shared by all -cursors opened in the group.</p> - <p>Berkeley DB Concurrent Data Store groups use a <a href="../api_reference/C/txn.html" class="olink">TXN</a> handle to indicate the shared locker -ID to Berkeley DB calls, and call <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a> to end the group. This -is a convenient way to pass the locked ID to the calls where it is -needed, but should not be confused with the real transactional semantics -provided by Berkeley DB Transactional Data Store. In particular, Berkeley DB Concurrent Data Store groups do not provide any -abort or recovery facilities, and have no impact on durability of -operations.</p> + <p> + If the application needs to open multiple cursors in a + single thread to perform an operation, it can indicate to + Berkeley DB that the cursor locks should not block each other + by creating a Berkeley DB Concurrent Data Store <span class="bold"><strong>group</strong></span>, using <a href="../api_reference/C/envcdsgroup_begin.html" class="olink">DB_ENV->cdsgroup_begin()</a>. + This creates a locker ID that is shared by all cursors opened + in the group. + </p> + <p> + Berkeley DB Concurrent Data Store groups use a <a href="../api_reference/C/txn.html" class="olink">TXN</a> handle + to indicate the shared locker ID to Berkeley DB calls, and + call <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a> to end the group. This is a convenient way to + pass the locked ID to the calls where it is needed, but should + not be confused with the real transactional semantics provided + by Berkeley DB Transactional Data Store. In particular, + Berkeley DB Concurrent Data Store groups do not provide any + abort or recovery facilities, and have no impact on durability + of operations. + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/cam_app.html b/docs/programmer_reference/cam_app.html index 07889503..b2287e72 100644 --- a/docs/programmer_reference/cam_app.html +++ b/docs/programmer_reference/cam_app.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,8 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="cam_fail.html">Prev</a> </td> - <th width="60%" align="center">Chapter 10. - Berkeley DB Concurrent Data Store Applications - </th> + <th width="60%" align="center">Chapter 10. Berkeley DB Concurrent Data Store + Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp.html">Next</a></td> </tr> </table> @@ -38,197 +37,299 @@ </div> </div> </div> - <p>When building Data Store and Concurrent Data Store applications, the -architecture decisions involve application startup (cleaning up any -existing databases, the removal of any existing database environment -and creation of a new environment), and handling system or application -failure. "Cleaning up" databases involves removal and re-creation -of the database, restoration from an archival copy and/or verification -and optional salvage, as described in <a class="xref" href="cam_fail.html" title="Handling failure in Data Store and Concurrent Data Store applications">Handling failure in Data Store and Concurrent Data Store applications</a>.</p> - <p>Data Store or Concurrent Data Store applications without database -environments are single process, by definition. These applications -should start up, re-create, restore, or verify and optionally salvage -their databases and run until eventual exit or application or system -failure. After system or application failure, that process can simply -repeat this procedure. This document will not discuss the case of these -applications further.</p> - <p>Otherwise, the first question of Data Store and Concurrent Data Store -architecture is the cleaning up existing databases and the removal of -existing database environments, and the subsequent creation of a new -environment. For obvious reasons, the application must serialize the -re-creation, restoration, or verification and optional salvage of its -databases. Further, environment removal and creation must be -single-threaded, that is, one thread of control (where a thread of -control is either a true thread or a process) must remove and re-create -the environment before any other thread of control can use the new -environment. It may simplify matters that Berkeley DB serializes creation of -the environment, so multiple threads of control attempting to create a -environment will serialize behind a single creating thread.</p> - <p>Removing a database environment will first mark the environment as -"failed", causing any threads of control still running in the -environment to fail and return to the application. This feature allows -applications to remove environments without concern for threads of -control that might still be running in the removed environment.</p> - <p>One consideration in removing a database environment which may be in use -by another thread, is the type of mutex being used by the Berkeley DB library. -In the case of database environment failure when using test-and-set -mutexes, threads of control waiting on a mutex when the environment is -marked "failed" will quickly notice the failure and will return an error -from the Berkeley DB API. In the case of environment failure when using -blocking mutexes, where the underlying system mutex implementation does -not unblock mutex waiters after the thread of control holding the mutex -dies, threads waiting on a mutex when an environment is recovered might -hang forever. Applications blocked on events (for example, an -application blocked on a network socket or a GUI event) may also fail -to notice environment recovery within a reasonable amount of time. -Systems with such mutex implementations are rare, but do exist; -applications on such systems should use an application architecture -where the thread recovering the database environment can explicitly -terminate any process using the failed environment, or configure Berkeley DB -for test-and-set mutexes, or incorporate some form of long-running timer -or watchdog process to wake or kill blocked processes should they block -for too long.</p> - <p>Regardless, it makes little sense for multiple threads of control to -simultaneously attempt to remove and re-create a environment, since the -last one to run will remove all environments created by the threads of -control that ran before it. However, for some few applications, it may -make sense for applications to have a single thread of control that -checks the existing databases and removes the environment, after which -the application launches a number of processes, any of which are able -to create the environment.</p> - <p>With respect to cleaning up existing databases, the database environment -must be removed before the databases are cleaned up. Removing the -environment causes any Berkeley DB library calls made by threads of control -running in the failed environment to return failure to the application. -Removing the database environment first ensures the threads of control -in the old environment do not race with the threads of control cleaning -up the databases, possibly overwriting them after the cleanup has -finished. Where the application architecture and system permit, many -applications kill all threads of control running in the failed database -environment before removing the failed database environment, on general -principles as well as to minimize overall system resource usage. It -does not matter if the new environment is created before or after the -databases are cleaned up.</p> - <p>After having dealt with database and database environment recovery after -failure, the next issue to manage is application failure. As described -in <a class="xref" href="cam_fail.html" title="Handling failure in Data Store and Concurrent Data Store applications">Handling failure in Data Store and Concurrent Data Store applications</a>, when a thread of control in a Data Store or -Concurrent Data Store application fails, it may exit holding data -structure mutexes or logical database locks. These mutexes and locks -must be released to avoid the remaining threads of control hanging -behind the failed thread of control's mutexes or locks.</p> - <p>There are three common ways to architect Berkeley DB Data Store and Concurrent -Data Store applications. The one chosen is usually based on whether or -not the application is comprised of a single process or group of -processes descended from a single process (for example, a server started -when the system first boots), or if the application is comprised of -unrelated processes (for example, processes started by web connections -or users logging into the system).</p> + <p> + When building Data Store and Concurrent Data Store + applications, the architecture decisions involve application + startup (cleaning up any existing databases, the removal of + any existing database environment and creation of a new + environment), and handling system or application failure. + "Cleaning up" databases involves removal and re-creation of + the database, restoration from an archival copy and/or + verification and optional salvage, as described in <a class="xref" href="cam_fail.html" title="Handling failure in Data Store and Concurrent Data Store applications">Handling failure in Data Store and Concurrent Data Store applications</a>. + </p> + <p> + Data Store or Concurrent Data Store applications without + database environments are single process, by definition. These + applications should start up, re-create, restore, or verify + and optionally salvage their databases and run until eventual + exit or application or system failure. After system or + application failure, that process can simply repeat this + procedure. This document will not discuss the case of these + applications further. + </p> + <p> + Otherwise, the first question of Data Store and Concurrent + Data Store architecture is the cleaning up existing databases + and the removal of existing database environments, and the + subsequent creation of a new environment. For obvious reasons, + the application must serialize the re-creation, restoration, + or verification and optional salvage of its databases. + Further, environment removal and creation must be + single-threaded, that is, one thread of control (where a + thread of control is either a true thread or a process) must + remove and re-create the environment before any other thread + of control can use the new environment. It may simplify + matters that Berkeley DB serializes creation of the + environment, so multiple threads of control attempting to + create a environment will serialize behind a single creating + thread. + </p> + <p> + Removing a database environment will first mark the + environment as "failed", causing any threads of control still + running in the environment to fail and return to the + application. This feature allows applications to remove + environments without concern for threads of control that might + still be running in the removed environment. + </p> + <p> + One consideration in removing a database environment which + may be in use by another thread, is the type of mutex being + used by the Berkeley DB library. In the case of database + environment failure when using test-and-set mutexes, threads + of control waiting on a mutex when the environment is marked + "failed" will quickly notice the failure and will return an + error from the Berkeley DB API. In the case of environment + failure when using blocking mutexes, where the underlying + system mutex implementation does not unblock mutex waiters + after the thread of control holding the mutex dies, threads + waiting on a mutex when an environment is recovered might hang + forever. Applications blocked on events (for example, an + application blocked on a network socket or a GUI event) may + also fail to notice environment recovery within a reasonable + amount of time. Systems with such mutex implementations are + rare, but do exist; applications on such systems should use an + application architecture where the thread recovering the + database environment can explicitly terminate any process + using the failed environment, or configure Berkeley DB for + test-and-set mutexes, or incorporate some form of long-running + timer or watchdog process to wake or kill blocked processes + should they block for too long. + </p> + <p> + Regardless, it makes little sense for multiple threads of + control to simultaneously attempt to remove and re-create a + environment, since the last one to run will remove all + environments created by the threads of control that ran before + it. However, for some few applications, it may make sense for + applications to have a single thread of control that checks + the existing databases and removes the environment, after + which the application launches a number of processes, any of + which are able to create the environment. + </p> + <p> + With respect to cleaning up existing databases, the database + environment must be removed before the databases are cleaned + up. Removing the environment causes any Berkeley DB library + calls made by threads of control running in the failed + environment to return failure to the application. Removing the + database environment first ensures the threads of control in + the old environment do not race with the threads of control + cleaning up the databases, possibly overwriting them after the + cleanup has finished. Where the application architecture and + system permit, many applications kill all threads of control + running in the failed database environment before removing the + failed database environment, on general principles as well as + to minimize overall system resource usage. It does not matter + if the new environment is created before or after the + databases are cleaned up. + </p> + <p> + After having dealt with database and database environment + recovery after failure, the next issue to manage is + application failure. As described in <a class="xref" href="cam_fail.html" title="Handling failure in Data Store and Concurrent Data Store applications">Handling failure in Data Store and Concurrent Data Store applications</a>, when a thread of control in a + Data Store or Concurrent Data Store application fails, it may + exit holding data structure mutexes or logical database locks. + These mutexes and locks must be released to avoid the + remaining threads of control hanging behind the failed thread + of control's mutexes or locks. + </p> + <p> + There are three common ways to architect Berkeley DB Data + Store and Concurrent Data Store applications. The one chosen + is usually based on whether or not the application is + comprised of a single process or group of processes descended + from a single process (for example, a server started when the + system first boots), or if the application is comprised of + unrelated processes (for example, processes started by web + connections or users logging into the system). + </p> <div class="orderedlist"> <ol type="1"> - <li>The first way to architect Data Store and Concurrent Data Store -applications is as a single process (the process may or may not be -multithreaded.) -<p>When this process starts, it removes any existing database environment -and creates a new environment. It then cleans up the databases and -opens those databases in the environment. The application can -subsequently create new threads of control as it chooses. Those threads -of control can either share already open Berkeley DB <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> and -<a href="../api_reference/C/db.html" class="olink">DB</a> handles, or create their own. In this architecture, -databases are rarely opened or closed when more than a single thread of -control is running; that is, they are opened when only a single thread -is running, and closed after all threads but one have exited. The last -thread of control to exit closes the databases and the database -environment.</p><p>This architecture is simplest to implement because thread serialization -is easy and failure detection does not require monitoring multiple -processes.</p><p>If the application's thread model allows the process to continue after -thread failure, the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method can be used to determine if -the database environment is usable after the failure. If the -application does not call <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a>, or -<a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> returns <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, the application -must behave as if there has been a system failure, removing the -environment and creating a new environment, and cleaning up any -databases it wants to continue to use. Once these actions have been -taken, other threads of control can continue (as long as all existing -Berkeley DB handles are first discarded), or restarted.</p></li> - <li>The second way to architect Data Store and Concurrent Data Store -applications is as a group of related processes (the processes may or -may not be multithreaded). -<p>This architecture requires the order in which threads of control are -created be controlled to serialize database environment removal and -creation, and database cleanup.</p><p>In addition, this architecture requires that threads of control be -monitored. If any thread of control exits with open Berkeley DB handles, the -application may call the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method to determine if the -database environment is usable after the exit. If the application does -not call <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a>, or <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> returns -<a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, the application must behave as if there has been -a system failure, removing the environment and creating a new -environment, and cleaning up any databases it wants to continue to use. -Once these actions have been taken, other threads of control can -continue (as long as all existing Berkeley DB handles are first discarded), -or restarted.</p><p>The easiest way to structure groups of related processes is to first -create a single "watcher" process (often a script) that starts when the -system first boots, removes and creates the database environment, cleans -up the databases and then creates the processes or threads that will -actually perform work. The initial thread has no further -responsibilities other than to wait on the threads of control it has -started, to ensure none of them unexpectedly exit. If a thread of -control exits, the watcher process optionally calls the -<a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method. If the application does not call -<a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a>, or if <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> returns -<a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, the environment can no longer be used, the -watcher kills all of the threads of control using the failed -environment, cleans up, and starts new threads of control to perform -work.</p></li> - <li>The third way to architect Data Store and Concurrent Data Store -applications is as a group of unrelated processes (the processes may or -may not be multithreaded). This is the most difficult architecture to -implement because of the level of difficulty in some systems of finding -and monitoring unrelated processes. -<p>One solution is to log a thread of control ID when a new Berkeley DB handle -is opened. For example, an initial "watcher" process could open/create -the database environment, clean up the databases and then create a -sentinel file. Any "worker" process wanting to use the environment -would check for the sentinel file. If the sentinel file does not exist, -the worker would fail or wait for the sentinel file to be created. Once -the sentinel file exists, the worker would register its process ID with -the watcher (via shared memory, IPC or some other registry mechanism), -and then the worker would open its <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handles and proceed. -When the worker finishes using the environment, it would unregister its -process ID with the watcher. The watcher periodically checks to ensure -that no worker has failed while using the environment. If a worker -fails while using the environment, the watcher removes the sentinel -file, kills all of the workers currently using the environment, cleans -up the environment and databases, and finally creates a new sentinel -file.</p><p>The weakness of this approach is that, on some systems, it is difficult -to determine if an unrelated process is still running. For example, -POSIX systems generally disallow sending signals to unrelated processes. -The trick to monitoring unrelated processes is to find a system resource -held by the process that will be modified if the process dies. On POSIX -systems, flock- or fcntl-style locking will work, as will LockFile on -Windows systems. Other systems may have to use other process-related -information such as file reference counts or modification times. In the -worst case, threads of control can be required to periodically -re-register with the watcher process: if the watcher has not heard from -a thread of control in a specified period of time, the watcher will take -action, cleaning up the environment.</p><p>If it is not practical to monitor the processes sharing a database -environment, it may be possible to monitor the environment to detect if -a thread of control has failed holding open Berkeley DB handles. This would -be done by having a "watcher" process periodically call the -<a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method. If <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> returns -<a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, the watcher would then take action, cleaning up -the environment.</p><p>The weakness of this approach is that all threads of control using the -environment must specify an "ID" function and an "is-alive" function -using the <a href="../api_reference/C/envset_thread_id.html" class="olink">DB_ENV->set_thread_id()</a> method. (In other words, the Berkeley DB -library must be able to assign a unique ID to each thread of control, -and additionally determine if the thread of control is still running. -It can be difficult to portably provide that information in applications -using a variety of different programming languages and running on a -variety of different platforms.)</p></li> + <li> + The first way to architect Data Store and Concurrent + Data Store applications is as a single process (the + process may or may not be multithreaded.) + <p> + When this + process starts, it removes any existing database + environment and creates a new environment. It then + cleans up the databases and opens those databases in + the environment. The application can subsequently + create new threads of control as it chooses. Those + threads of control can either share already open + Berkeley DB <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> and <a href="../api_reference/C/db.html" class="olink">DB</a> handles, or create their + own. In this architecture, databases are rarely opened + or closed when more than a single thread of control is + running; that is, they are opened when only a single + thread is running, and closed after all threads but + one have exited. The last thread of control to exit + closes the databases and the database + environment. + </p><p> + This architecture is simplest to implement because + thread serialization is easy and failure detection + does not require monitoring multiple processes. + </p><p> + If the application's thread model allows the process + to continue after thread failure, the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> + method can be used to determine if the database + environment is usable after the failure. If the + application does not call <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a>, or + <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> returns <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, + the application must behave as if there has been a system + failure, removing the environment and creating a new + environment, and cleaning up any databases it wants to + continue to use. Once these actions have been taken, other + threads of control can continue (as long as all existing + Berkeley DB handles are first discarded), or restarted. + </p><p> + Note that by default <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> will only notify the + calling thread that the database environment is unusable. + However, you can optionally cause <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> to broadcast + this to other threads of control by using the + <code class="literal">--enable-failchk_broadcast</code> flag when you + compile your Berkeley DB library. If this option is turned + on, then all threads of control using the database + environment will return + <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a> + when they attempt to obtain a mutex lock. In this + situation, a <code class="literal">DB_EVENT_FAILCHK_PANIC</code> or + <code class="literal">DB_EVENT_MUTEX_DIED</code> event will also be + raised. (You use <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> to examine events). + </p></li> + <li> + The second way to architect Data Store and + Concurrent Data Store applications is as a group of + related processes (the processes may or may not be + multithreaded). + <p> + This architecture requires the order + in which threads of control are created be controlled + to serialize database environment removal and + creation, and database cleanup. + </p><p> + In addition, this architecture requires that threads + of control be monitored. If any thread of control + exits with open Berkeley DB handles, the application + may call the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method to determine if the + database environment is usable after the exit. If the + application does not call <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a>, or + <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> returns <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, + the application must + behave as if there has been a system failure, removing + the environment and creating a new environment, and + cleaning up any databases it wants to continue to use. + Once these actions have been taken, other threads of + control can continue (as long as all existing Berkeley + DB handles are first discarded), or restarted. + </p><p> + The easiest way to structure groups of related + processes is to first create a single "watcher" + process (often a script) that starts when the system + first boots, removes and creates the database + environment, cleans up the databases and then creates + the processes or threads that will actually perform + work. The initial thread has no further + responsibilities other than to wait on the threads of + control it has started, to ensure none of them + unexpectedly exit. If a thread of control exits, the + watcher process optionally calls the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> + method. If the application does not call <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a>, + or if <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> returns <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, the environment can no + longer be used, the watcher kills all of the threads + of control using the failed environment, cleans up, + and starts new threads of control to perform + work. + </p></li> + <li> + The third way to architect Data Store and Concurrent + Data Store applications is as a group of unrelated + processes (the processes may or may not be multithreaded). + This is the most difficult architecture to implement + because of the level of difficulty in some systems of + finding and monitoring unrelated processes. + <p> + One + solution is to log a thread of control ID when a new + Berkeley DB handle is opened. For example, an initial + "watcher" process could open/create the database + environment, clean up the databases and then create a + sentinel file. Any "worker" process wanting to use the + environment would check for the sentinel file. If the + sentinel file does not exist, the worker would fail or + wait for the sentinel file to be created. Once the + sentinel file exists, the worker would register its + process ID with the watcher (via shared memory, IPC or + some other registry mechanism), and then the worker + would open its <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handles and proceed. When the + worker finishes using the environment, it would + unregister its process ID with the watcher. The + watcher periodically checks to ensure that no worker + has failed while using the environment. If a worker + fails while using the environment, the watcher removes + the sentinel file, kills all of the workers currently + using the environment, cleans up the environment and + databases, and finally creates a new sentinel + file. + </p><p> + The weakness of this approach is that, on some + systems, it is difficult to determine if an unrelated + process is still running. For example, POSIX systems + generally disallow sending signals to unrelated + processes. The trick to monitoring unrelated processes + is to find a system resource held by the process that + will be modified if the process dies. On POSIX + systems, flock- or fcntl-style locking will work, as + will LockFile on Windows systems. Other systems may + have to use other process-related information such as + file reference counts or modification times. In the + worst case, threads of control can be required to + periodically re-register with the watcher process: if + the watcher has not heard from a thread of control in + a specified period of time, the watcher will take + action, cleaning up the environment. + </p><p> + If it is not practical to monitor the processes + sharing a database environment, it may be possible to + monitor the environment to detect if a thread of + control has failed holding open Berkeley DB handles. + This would be done by having a "watcher" process + periodically call the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method. If + <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> returns <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, + the watcher would then + take action, cleaning up the environment. + </p><p> + The weakness of this approach is that all threads of + control using the environment must specify an "ID" + function and an "is-alive" function using the + <a href="../api_reference/C/envset_thread_id.html" class="olink">DB_ENV->set_thread_id()</a> method. (In other words, the + Berkeley DB library must be able to assign a unique ID + to each thread of control, and additionally determine + if the thread of control is still running. It can be + difficult to portably provide that information in + applications using a variety of different programming + languages and running on a variety of different + platforms.) + </p></li> </ol> </div> - <p>Obviously, when implementing a process to monitor other threads of -control, it is important the watcher process' code be as simple and -well-tested as possible, because the application may hang if it fails.</p> + <p> + Obviously, when implementing a process to monitor other + threads of control, it is important the watcher process' code + be as simple and well-tested as possible, because the + application may hang if it fails. + </p> </div> <div class="navfooter"> <hr /> @@ -245,9 +346,7 @@ well-tested as possible, because the application may hang if it fails.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 11. - Berkeley DB Transactional Data Store Applications - </td> + <td width="40%" align="right" valign="top"> Chapter 11. Berkeley DB Transactional Data Store Applications </td> </tr> </table> </div> diff --git a/docs/programmer_reference/cam_fail.html b/docs/programmer_reference/cam_fail.html index f638350d..f5669dea 100644 --- a/docs/programmer_reference/cam_fail.html +++ b/docs/programmer_reference/cam_fail.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,8 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="cam.html">Prev</a> </td> - <th width="60%" align="center">Chapter 10. - Berkeley DB Concurrent Data Store Applications - </th> + <th width="60%" align="center">Chapter 10. Berkeley DB Concurrent Data Store + Applications </th> <td width="20%" align="right"> <a accesskey="n" href="cam_app.html">Next</a></td> </tr> </table> @@ -38,69 +37,117 @@ </div> </div> </div> - <p>When building Data Store and Concurrent Data Store applications, there -are design issues to consider whenever a thread of control with open -Berkeley DB handles fails for any reason (where a thread of control may be -either a true thread or a process).</p> - <p>The simplest case is handling system failure for any Data Store or -Concurrent Data Store application. In the case of system failure, it -doesn't matter if the application has opened a database environment or -is just using standalone databases: if the system fails, after the -application has modified a database and has not subsequently flushed the -database to stable storage (by calling either the <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a>, -<a href="../api_reference/C/dbsync.html" class="olink">DB->sync()</a> or <a href="../api_reference/C/mempsync.html" class="olink">DB_ENV->memp_sync()</a> methods), the database may be left in a -corrupted state. In this case, before accessing the database again, the -database should either be:</p> + <p> + When building Data Store and Concurrent Data Store + applications, there are design issues to consider whenever a + thread of control with open Berkeley DB handles fails for any + reason (where a thread of control may be either a true thread + or a process). + </p> + <p> + The simplest case is handling system failure for any Data + Store or Concurrent Data Store application. In the case of + system failure, it doesn't matter if the application has + opened a database environment or is just using standalone + databases: if the system fails, after the application has + modified a database and has not subsequently flushed the + database to stable storage (by calling either the <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a>, + <a href="../api_reference/C/dbsync.html" class="olink">DB->sync()</a> or <a href="../api_reference/C/mempsync.html" class="olink">DB_ENV->memp_sync()</a> methods), the database may be left in a + corrupted state. In this case, before accessing the database + again, the database should either be: + </p> <div class="itemizedlist"> <ul type="disc"> - <li>removed and re-created,</li> - <li>removed and restored from the last known good backup, or</li> - <li>verified using the <a href="../api_reference/C/dbverify.html" class="olink">DB->verify()</a> method or <a href="../api_reference/C/db_verify.html" class="olink">db_verify</a> utility. If -the database does not verify cleanly, the contents may be salvaged using -the <span class="bold"><strong>-R</strong></span> and <span class="bold"><strong>-r</strong></span> options of the <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility.</li> + <li> + removed and re-created, + </li> + <li> + removed and restored from the last known good + backup, or + </li> + <li> + verified using the <a href="../api_reference/C/dbverify.html" class="olink">DB->verify()</a> method or <a href="../api_reference/C/db_verify.html" class="olink">db_verify</a> utility. + If the database does not verify cleanly, the contents may + be salvaged using the <span class="bold"><strong>-R</strong></span> + and <span class="bold"><strong>-r</strong></span> options of the + <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility. + </li> </ul> </div> - <p>Applications where the potential for data loss is unacceptable should -consider the Berkeley DB Transactional Data Store product, which offers standard transactional -durability guarantees, including recoverability after failure.</p> - <p>Additionally, system failure requires that any persistent database -environment (that is, any database environment not created using the -<a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a> flag), be removed. Database environments may be -removed using the <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> method. If the persistent database -environment was backed by the filesystem (that is, the environment was -not created using the <a href="../api_reference/C/envopen.html#envopen_DB_SYSTEM_MEM" class="olink">DB_SYSTEM_MEM</a> flag), the database -environment may also be safely removed by deleting the environment's -files with standard system utilities.</p> - <p>The second case is application failure for a Data Store application, -with or without a database environment, or application failure for a -Concurrent Data Store application without a database environment: as in -the case of system failure, if any thread of control fails, after the -application has modified a database and has not subsequently flushed the -database to stable storage, the database may be left in a corrupted -state. In this case, the database should be handled as described -previously in the system failure case.</p> - <p>The third case is application failure for a Concurrent Data Store -application with a database environment. There are resources maintained -in database environments that may be left locked if a thread of control -exits without first closing all open Berkeley DB handles. Concurrent Data -Store applications with database environments have an additional option -for handling the unexpected exit of a thread of control, the -<a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method.</p> - <p>The <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> will return <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a> if the -database environment is unusable as a result of the thread of control -failure. (If a data structure mutex or a database write lock is left -held by thread of control failure, the application should not continue -to use the database environment, as subsequent use of the environment -is likely to result in threads of control convoying behind the held -locks.) The <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> call will release any database read -locks that have been left held by the exit of a thread of control. In -this case, the application can continue to use the database -environment.</p> - <p>A Concurrent Data Store application recovering from a thread of control -failure should call <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a>, and, if it returns success, -the application can continue. If <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> returns -<a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, the application should proceed as described for -the case of system failure.</p> + <p> + Applications where the potential for data loss is + unacceptable should consider the Berkeley DB Transactional + Data Store product, which offers standard transactional + durability guarantees, including recoverability after + failure. + </p> + <p> + Additionally, system failure requires that any persistent + database environment (that is, any database environment not + created using the <a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a> flag), be removed. Database + environments may be removed using the <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> method. If + the persistent database environment was backed by the + filesystem (that is, the environment was not created using the + <a href="../api_reference/C/envopen.html#envopen_DB_SYSTEM_MEM" class="olink">DB_SYSTEM_MEM</a> flag), the database environment may also be + safely removed by deleting the environment's files with + standard system utilities. + </p> + <p> + The second case is application failure for a Data Store + application, with or without a database environment, or + application failure for a Concurrent Data Store application + without a database environment: as in the case of system + failure, if any thread of control fails, after the application + has modified a database and has not subsequently flushed the + database to stable storage, the database may be left in a + corrupted state. In this case, the database should be handled + as described previously in the system failure case. + </p> + <p> + The third case is application failure for a Concurrent Data + Store application with a database environment. There are + resources maintained in database environments that may be left + locked if a thread of control exits without first closing all + open Berkeley DB handles. Concurrent Data Store applications + with database environments have an additional option for + handling the unexpected exit of a thread of control, the + <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method. + </p> + <p> + The <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> will return + <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a> + if the database environment is + unusable as a result of the thread of control failure. (If a + data structure mutex or a database write lock is left held by + thread of control failure, the application should not continue + to use the database environment, as subsequent use of the + environment is likely to result in threads of control + convoying behind the held locks.) The <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> call will + release any database read locks that have been left held by + the exit of a thread of control. In this case, the application + can continue to use the database environment. + </p> + <p> + Note that by default <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> only notifies the calling thread that + the database environment is unusable. You can optionally cause + <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> to broadcast this to other threads of control by using + the <code class="literal">--enable-failchk_broadcast</code> flag when you + compile your Berkeley DB library. If this option is turned on, + then all threads of control using the database environment will + return + <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a> + when they attempt to obtain a mutex lock. In this situation, a + <code class="literal">DB_EVENT_FAILCHK_PANIC</code> or <code class="literal">DB_EVENT_MUTEX_DIED</code> + event will also be raised. (You use <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> to examine events). + </p> + <p> + A Concurrent Data Store application recovering from a thread + of control failure should call <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a>, and, if it + returns success, the application can continue. If <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> + returns <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, + the application should proceed as + described for the case of system failure. + </p> </div> <div class="navfooter"> <hr /> @@ -113,9 +160,8 @@ the case of system failure.</p> <td width="40%" align="right"> <a accesskey="n" href="cam_app.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 10. - Berkeley DB Concurrent Data Store Applications - </td> + <td width="40%" align="left" valign="top">Chapter 10. Berkeley DB Concurrent Data Store + Applications </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/ch13s02.html b/docs/programmer_reference/ch13s02.html index 65bfe350..d66215d9 100644 --- a/docs/programmer_reference/ch13s02.html +++ b/docs/programmer_reference/ch13s02.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="xa.html">Prev</a> </td> - <th width="60%" align="center">Chapter 13. - Distributed Transactions - </th> + <th width="60%" align="center">Chapter 13. Distributed Transactions </th> <td width="20%" align="right"> <a accesskey="n" href="xa_build.html">Next</a></td> </tr> </table> @@ -34,35 +32,39 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="idp2523200"></a>Berkeley DB XA Implementation</h2> + <h2 class="title" style="clear: both"><a id="idp2294328"></a>Berkeley DB XA Implementation</h2> </div> </div> </div> <p> -Berkeley DB provides support for distributed transactions using the two-phase commit protocol - via its <a href="../api_reference/C/txnprepare.html" class="olink">DB_TXN->prepare()</a> interfaces. - The <a href="../api_reference/C/txnprepare.html" class="olink">DB_TXN->prepare()</a> method performs the first phase of a -two-phase commit, flushing the log to disk, and associating a global -transaction ID with the underlying Berkeley DB transaction. - This global transaction ID is used by the global transaction manager to -identify the Berkeley DB transaction, and will be returned by the -<a href="../api_reference/C/txnrecover.html" class="olink">DB_ENV->txn_recover()</a> method when it is called during recovery. - </p> + Berkeley DB provides support for distributed transactions + using the two-phase commit protocol via its <a href="../api_reference/C/txnprepare.html" class="olink">DB_TXN->prepare()</a> interfaces. + The <a href="../api_reference/C/txnprepare.html" class="olink">DB_TXN->prepare()</a> method performs the first phase of a + two-phase commit, flushing the log to disk, and associating a global + transaction ID with the underlying Berkeley DB transaction. + This global transaction ID is used by the global transaction manager to + identify the Berkeley DB transaction, and will be returned by the + <a href="../api_reference/C/txnrecover.html" class="olink">DB_ENV->txn_recover()</a> method when it is called during recovery. + </p> <p> -However, Berkeley DB does not perform distributed deadlock detection. -Instead, when being used as an XA resource manager, Berkeley DB acquires all locks -in a non-blocking mode. This results in pre-emptive abort of transactions that have the potential to deadlock. -While this can lead to more transactions being aborted than is strictly necessary, -it avoids system-wide hanging due to distributed deadlocks. -</p> - <p>When using distributed transactions, there is no way to perform -hot backups of multiple environments and guarantee that the backups -are globally transaction-consistent across these multiple environments. -If backups are desired, then all write transactions should be suspended; -that is, active write transactions must be allowed to complete and no -new write transactions should be begun. Once there are no active write -transactions, the logs may be copied for backup purposes and the backup -will be consistent across the multiple environments.</p> + However, Berkeley DB does not perform distributed deadlock + detection. Instead, when being used as an XA resource manager, + Berkeley DB acquires all locks in a non-blocking mode. This + results in pre-emptive abort of transactions that have the + potential to deadlock. While this can lead to more transactions + being aborted than is strictly necessary, + it avoids system-wide hanging due to distributed deadlocks. + </p> + <p> + When using distributed transactions, there is no way to perform + hot backups of multiple environments and guarantee that the backups + are globally transaction-consistent across these multiple environments. + If backups are desired, then all write transactions should be suspended; + that is, active write transactions must be allowed to complete and no + new write transactions should be begun. Once there are no active write + transactions, the logs may be copied for backup purposes and the backup + will be consistent across the multiple environments. + </p> </div> <div class="navfooter"> <hr /> @@ -75,9 +77,7 @@ will be consistent across the multiple environments.</p> <td width="40%" align="right"> <a accesskey="n" href="xa_build.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 13. - Distributed Transactions - </td> + <td width="40%" align="left" valign="top">Chapter 13. Distributed Transactions </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/csharp.html b/docs/programmer_reference/csharp.html index 6ba247ce..2f2b62d8 100644 --- a/docs/programmer_reference/csharp.html +++ b/docs/programmer_reference/csharp.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -48,78 +48,92 @@ </dt> </dl> </div> - <p>You can use Berkeley DB in your application through the C# API. To understand the application concepts relating to Berkeley DB, see the first few chapters of this manual. -For a general discussion on how to build Berkeley DB applications, see the Berkeley DB Getting Started Guides of C or C++. -You can also review the example code of C and C++ from the examples_c and examples_cxx directories. For a description of all the classes, functions, and enumerations of Berkeley DB C# API, see the <a class="ulink" href="http://download.oracle.com/docs/cd/E17076_02/html/csharp/Index.html" target="_top"> - Berkeley DB C# API Reference Guide. - </a></p> <p> - A separate Visual Studio solution is provided to build the Berkeley DB - C# classes, the examples, and the native support library. See - <a href="../installation/build_win_csharp.html#build_win_csharp.title" class="olink">Building the C# API</a> in the Berkeley DB Installation and Build Guide for more information. -</p> + You can use Berkeley DB in your application through the C# + API. To understand the application concepts relating to + Berkeley DB, see the first few chapters of this manual. For a + general discussion on how to build Berkeley DB applications, + see the Berkeley DB Getting Started Guides of C or C++. You + can also review the example code of C and C++ from the + examples/c and examples/cxx directories. For a description of + all the classes, functions, and enumerations of Berkeley DB C# + API, see the <a class="ulink" href="http://docs.oracle.com/cd/E17076_02/html/csharp/Index.html" target="_top"> Berkeley DB C# API Reference Guide. + </a> + </p> <p> - The C# API requires .NET framework version 2.0 or above, and expects that - it has already been installed on your system. For the sake of - discussion, we assume that the Berkeley DB source is in a directory - called db-<span class="emphasis"><em>VERSION</em></span>; for example, you downloaded a - Berkeley DB archive, and you did not change the top-level directory - name. The files related to C# are in four subdirectories of - db-<span class="emphasis"><em>VERSION</em></span>: csharp (the C# source files), - libdb_csharp (the C++ files that provide the "glue" between C# and - Berkeley DB,) examples_csharp (containing all example code) and - test\scr037 (containing NUnit tests for the API). -</p> + A separate Visual Studio solution is provided to build the + Berkeley DB C# classes, the examples, and the native support + library. See <a href="../installation/build_win_csharp.html#build_win_csharp.title" class="olink">Building the C# + API</a> in the Berkeley DB Installation and Build + Guide for more information. + </p> + <p> + The C# API requires .NET framework version 2.0 or above, + and expects that it has already been installed on your system. + For the sake of discussion, we assume that the Berkeley DB + source is in a directory called + db-<span class="emphasis"><em>VERSION</em></span>; for example, you + downloaded a Berkeley DB archive, and you did not change the + top-level directory name. The files related to C# are in four + subdirectories of db-<span class="emphasis"><em>VERSION</em></span>: csharp (the + C# source files), libdb_csharp (the C++ files that provide the + "glue" between C# and Berkeley DB,) examples/csharp + (containing all example code) and test\scr037 (containing + NUnit tests for the API). + </p> <p> - Building the C# API produces a managed assembly - <code class="filename">libdb_dotnet<span class="emphasis"><em>VERSION</em></span>.dll</code>, - containing the API, and two native libraries: - <code class="filename">libdb_csharp<span class="emphasis"><em>VERSION</em></span>.dll</code> and - <code class="filename">libdb<span class="emphasis"><em>VERSION</em></span>.dll</code>. (For all - three files, <span class="emphasis"><em>VERSION</em></span> is [MAJOR][MINOR], i.e. for - version 4.8 the managed assembly is - <code class="filename">libdb_dotnet48.dll</code>.) Following the existing - convention, native libraries are placed in either - <code class="filename">db-<span class="emphasis"><em>VERSION</em></span>\build_windows\Win32</code>or - <code class="filename">db-<span class="emphasis"><em>VERSION</em></span>\build_windows\x64</code>, - depending upon the platform being targeted. In all cases, the managed - assembly will be placed in - <code class="filename">db-<span class="emphasis"><em>VERSION</em></span>\build_windows\AnyCPU</code>. -</p> + Building the C# API produces a managed assembly + <code class="filename">libdb_dotnet<span class="emphasis"><em>VERSION</em></span>.dll</code>, + containing the API, and two native libraries: + <code class="filename">libdb_csharp<span class="emphasis"><em>VERSION</em></span>.dll</code> + and + <code class="filename">libdb<span class="emphasis"><em>VERSION</em></span>.dll</code>. + (For all three files, <span class="emphasis"><em>VERSION</em></span> is + [MAJOR][MINOR], i.e. for version 4.8 the managed assembly is + <code class="filename">libdb_dotnet48.dll</code>.) Following the + existing convention, native libraries are placed in either + <code class="filename">db-<span class="emphasis"><em>VERSION</em></span>\build_windows\Win32</code>or + <code class="filename">db-<span class="emphasis"><em>VERSION</em></span>\build_windows\x64</code>, + depending upon the platform being targeted. In all cases, the + managed assembly will be placed in + <code class="filename">db-<span class="emphasis"><em>VERSION</em></span>\build_windows\AnyCPU</code>. + </p> <p> - Because the C# API uses P/Invoke, for your application to use Berkeley - DB successfully, the .NET framework needs to be able to locate the - native libaries. This means the native libraries need to either be - copied to your application's directory, the Windows or System - directory, or the location of the libraries needs to be added to the - <code class="literal">PATH</code> environment variable. See the MSDN - documentation of the DllImport attribute and Dynamic-Link Library - Search Order for further information. -</p> - <p> - If you get the following exception when you run, the .NET platform probably - is unable to locate the native libraries: -</p> + Because the C# API uses P/Invoke, for your application to + use Berkeley DB successfully, the .NET framework needs to be + able to locate the native libaries. This means the native + libraries need to either be copied to your application's + directory, the Windows or System directory, or the location of + the libraries needs to be added to the <code class="literal">PATH</code> + environment variable. See the MSDN documentation of the + DllImport attribute and Dynamic-Link Library Search Order for + further information. + </p> + <p> + If you get the following exception when you run, the .NET + platform probably is unable to locate the native libraries: + </p> <pre class="programlisting">System.TypeInitializationException</pre> + <p> + To ensure that everything is running correctly, you may + want to try a simple test from the example programs in the + <code class="filename">db-<span class="emphasis"><em>VERSION</em></span>\examples/csharp</code> + directory. + </p> <p> - To ensure that everything is running correctly, you may want to try a - simple test from the example programs in the - <code class="filename">db-<span class="emphasis"><em>VERSION</em></span>\examples_csharp</code> - directory. -</p> - <p> - For example, the ex_access sample program will prompt for text input - lines, which are then stored in a Btree database named - <code class="filename">access.db</code>. It is designed to be run from either - the - <code class="filename">db-<span class="emphasis"><em>VERSION</em></span>\build_windows\Debug</code> - or - <code class="filename">db-<span class="emphasis"><em>VERSION</em></span>\build_windows\Release</code> - directory. Try giving it a few lines of input text and then a blank - line. Before it exits, you should see a list of the lines you entered - display with data items. This is a simple check to make sure the - fundamental configuration is working correctly. -</p> + For example, the ex_access sample program will prompt for + text input lines, which are then stored in a Btree database + named <code class="filename">access.db</code>. It is designed to be run + from either the + <code class="filename">db-<span class="emphasis"><em>VERSION</em></span>\build_windows\Debug</code> + or + <code class="filename">db-<span class="emphasis"><em>VERSION</em></span>\build_windows\Release</code> + directory. Try giving it a few lines of input text and then a + blank line. Before it exits, you should see a list of the + lines you entered display with data items. This is a simple + check to make sure the fundamental configuration is working + correctly. + </p> <div class="sect1" lang="en" xml:lang="en"> <div class="titlepage"> <div> @@ -128,9 +142,10 @@ You can also review the example code of C and C++ from the examples_c and exampl </div> </div> </div> - <p> - The Berkeley DB C# API has been tested with the Microsoft .NET Framework versions 2.0, 3.0, 3.5, and 4.0. -</p> + <p> + The Berkeley DB C# API has been tested with the + Microsoft .NET Framework versions 2.0, 3.0, 3.5, and 4.0. + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/dumpload.html b/docs/programmer_reference/dumpload.html index 5489304e..5597a5a9 100644 --- a/docs/programmer_reference/dumpload.html +++ b/docs/programmer_reference/dumpload.html @@ -14,13 +14,12 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 23. - Dumping and Reloading Databases - </th> + <th colspan="3" align="center">Chapter 23. Dumping and Reloading Databases + </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="ext_php.html">Prev</a> </td> @@ -34,9 +33,8 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="dumpload"></a>Chapter 23. - Dumping and Reloading Databases - </h2> + <h2 class="title"><a id="dumpload"></a>Chapter 23. Dumping and Reloading Databases + </h2> </div> </div> </div> @@ -70,12 +68,46 @@ </div> </div> </div> - <p>There are three utilities used for dumping and loading Berkeley DB databases: the <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility, the <a href="../api_reference/C/db_dump.html" class="olink">db_dump185</a> utility and the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility.</p> - <p>The <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility and the <a href="../api_reference/C/db_dump.html" class="olink">db_dump185</a> utility dump Berkeley DB databases into a flat-text representation of the data that can be read by <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility. The only difference between them is that the <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility reads Berkeley DB version 2 and greater database formats, whereas the <a href="../api_reference/C/db_dump.html" class="olink">db_dump185</a> utility reads Berkeley DB version 1.85 and 1.86 database formats.</p> - <p>The <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility reads either the output format used by the dump utilities or (optionally) a flat-text representation created using other tools, and stores it into a Berkeley DB database.</p> - <p>Dumping and reloading Hash databases that use user-defined hash functions will result in new databases that use the default hash function. Although using the default hash function may not be optimal for the new database, it will continue to work correctly.</p> - <p>Dumping and reloading Btree databases that use user-defined prefix or comparison functions will result in new databases that use the default prefix and comparison functions. In this case, it is quite likely that applications will be unable to retrieve records, and it is possible that the load process itself will fail.</p> - <p>The only available workaround for either Hash or Btree databases is to modify the sources for the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility to load the database using the correct hash, prefix, and comparison functions.</p> + <p> + There are three utilities used for dumping and loading + Berkeley DB databases: the <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility, the <a href="../api_reference/C/db_dump.html" class="olink">db_dump185</a> utility and the + <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility. + </p> + <p> + The <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility and the <a href="../api_reference/C/db_dump.html" class="olink">db_dump185</a> utility dump Berkeley DB + databases into a flat-text representation of the data that can + be read by <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility. The only difference between them is that + the <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility reads Berkeley DB version 2 and greater database + formats, whereas the <a href="../api_reference/C/db_dump.html" class="olink">db_dump185</a> utility reads Berkeley DB version + 1.85 and 1.86 database formats. + </p> + <p> + The <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility reads either the output format used by the + dump utilities or (optionally) a flat-text representation + created using other tools, and stores it into a Berkeley DB + database. + </p> + <p> + Dumping and reloading Hash databases that use user-defined + hash functions will result in new databases that use the + default hash function. Although using the default hash + function may not be optimal for the new database, it will + continue to work correctly. + </p> + <p> + Dumping and reloading Btree databases that use user-defined + prefix or comparison functions will result in new databases + that use the default prefix and comparison functions. In this + case, it is quite likely that applications will be unable to + retrieve records, and it is possible that the load process + itself will fail. + </p> + <p> + The only available workaround for either Hash or Btree + databases is to modify the sources for the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility to load + the database using the correct hash, prefix, and comparison + functions. + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/dumpload_format.html b/docs/programmer_reference/dumpload_format.html index 278fb1ce..21c94809 100644 --- a/docs/programmer_reference/dumpload_format.html +++ b/docs/programmer_reference/dumpload_format.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,8 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="dumpload.html">Prev</a> </td> - <th width="60%" align="center">Chapter 23. - Dumping and Reloading Databases - </th> + <th width="60%" align="center">Chapter 23. Dumping and Reloading Databases + </th> <td width="20%" align="right"> <a accesskey="n" href="dumpload_text.html">Next</a></td> </tr> </table> @@ -38,42 +37,92 @@ </div> </div> </div> - <p>There are two output formats used by the <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility and <a href="../api_reference/C/db_dump.html" class="olink">db_dump185</a> utility.</p> - <p>In both output formats, the first few lines of the output contain header -information describing the underlying access method, filesystem page size, -and other bookkeeping information.</p> - <p>The header information starts with a single line, VERSION=N, where N is -the version number of the dump output format.</p> - <p>The header information is then output in name=value pairs, where name may be any of the keywords listed in the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility manual page, and value will be its value. Although this header information can be manually edited before the database is reloaded, there is rarely any reason to do so because all of this information can also be specified or overridden by command-line arguments to the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility.</p> - <p>The header information ends with single line HEADER=END.</p> - <p>Following the header information are the key/data pairs from the -database. If the database being dumped is a Btree or Hash database, or -if the <span class="bold"><strong>-k</strong></span> option was specified, the output will be paired lines -of text where the first line of the pair is the key item, and the second -line of the pair is its corresponding data item. If the database being -dumped is a Queue or Recno database, and the <span class="bold"><strong>-k</strong></span> option was not -specified, the output will be lines of text where each line is the next -data item for the database. Each of these lines is preceded by a single -space.</p> - <p>If the <span class="bold"><strong>-p</strong></span> option was specified to the <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility or - <a href="../api_reference/C/db_dump.html" class="olink">db_dump185</a> utility, the key/data lines will consist of single characters representing any characters - from the database that are <span class="emphasis"><em>printing characters</em></span> and backslash - (<span class="bold"><strong>\</strong></span>) escaped characters for any that were not. Backslash - characters appearing in the output mean one of two things: if the backslash character precedes - another backslash character, it means that a literal backslash character occurred in the key or - data item. If the backslash character precedes any other character, the next two characters - must be interpreted as hexadecimal specification of a single character; for example, <span class="bold"><strong>\0a</strong></span> is a newline character in the ASCII character set.</p> - <p>Although some care should be exercised, it is perfectly reasonable to use standard text editors and tools to edit databases dumped using the <span class="bold"><strong>-p</strong></span> option before reloading them using the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility.</p> - <p>Note that the definition of a printing character may vary from system to -system, so database representations created using the <span class="bold"><strong>-p</strong></span> -option may be less portable than those created without it.</p> - <p>If the <span class="bold"><strong>-p</strong></span> option in not specified to <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility or <a href="../api_reference/C/db_dump.html" class="olink">db_dump185</a> utility, each output - line will consist of paired hexadecimal values; for example, the line <span class="bold"><strong>726f6f74</strong></span> is the string <span class="bold"><strong>root</strong></span> in the - ASCII character set.</p> - <p>In all output formats, the key and data items are ended by a single line -DATA=END.</p> - <p>Where multiple databases have been dumped from a file, the overall output -will repeat; that is, a new set of headers and a new set of data items.</p> + <p> + There are two output formats used by the <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility and + <a href="../api_reference/C/db_dump.html" class="olink">db_dump185</a> utility. + </p> + <p> + In both output formats, the first few lines of the output + contain header information describing the underlying access + method, filesystem page size, and other bookkeeping + information. + </p> + <p> + The header information starts with a single line, VERSION=N, + where N is the version number of the dump output + format. + </p> + <p> + The header information is then output in name=value pairs, + where name may be any of the keywords listed in the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility + manual page, and value will be its value. Although this header + information can be manually edited before the database is + reloaded, there is rarely any reason to do so because all of + this information can also be specified or overridden by + command-line arguments to the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility. + </p> + <p> + The header information ends with single line + HEADER=END. + </p> + <p> + Following the header information are the key/data pairs from + the database. If the database being dumped is a Btree or Hash + database, or if the <span class="bold"><strong>-k</strong></span> option + was specified, the output will be paired lines of text where + the first line of the pair is the key item, and the second + line of the pair is its corresponding data item. If the + database being dumped is a Queue or Recno database, and the + <span class="bold"><strong>-k</strong></span> option was not + specified, the output will be lines of text where each line is + the next data item for the database. Each of these lines is + preceded by a single space. + </p> + <p> + If the <span class="bold"><strong>-p</strong></span> option was + specified to the <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility or <a href="../api_reference/C/db_dump.html" class="olink">db_dump185</a> utility, the key/data lines + will consist of single characters representing any characters + from the database that are <span class="emphasis"><em>printing + characters</em></span> and backslash (<span class="bold"><strong>\</strong></span>) + escaped characters for any that were not. + Backslash characters appearing in the output mean one of two + things: if the backslash character precedes another backslash + character, it means that a literal backslash character + occurred in the key or data item. If the backslash character + precedes any other character, the next two characters must be + interpreted as hexadecimal specification of a single + character; for example, <span class="bold"><strong>\0a</strong></span> + is a newline character in the ASCII character set. + </p> + <p> + Although some care should be exercised, it is perfectly + reasonable to use standard text editors and tools to edit + databases dumped using the <span class="bold"><strong>-p</strong></span> + option before reloading them using the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility. + </p> + <p> + Note that the definition of a printing character may vary + from system to system, so database representations created + using the <span class="bold"><strong>-p</strong></span> option may be + less portable than those created without it. + </p> + <p> + If the <span class="bold"><strong>-p</strong></span> option in not + specified to <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility or <a href="../api_reference/C/db_dump.html" class="olink">db_dump185</a> utility, each output line will + consist of paired hexadecimal values; for example, the line + <span class="bold"><strong>726f6f74</strong></span> is the string + <span class="bold"><strong>root</strong></span> in the ASCII + character set. + </p> + <p> + In all output formats, the key and data items are ended by a + single line DATA=END. + </p> + <p> + Where multiple databases have been dumped from a file, the + overall output will repeat; that is, a new set of headers and + a new set of data items. + </p> </div> <div class="navfooter"> <hr /> @@ -86,9 +135,8 @@ will repeat; that is, a new set of headers and a new set of data items.</p> <td width="40%" align="right"> <a accesskey="n" href="dumpload_text.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 23. - Dumping and Reloading Databases - </td> + <td width="40%" align="left" valign="top">Chapter 23. Dumping and Reloading Databases + </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/dumpload_text.html b/docs/programmer_reference/dumpload_text.html index 884c016d..7d843252 100644 --- a/docs/programmer_reference/dumpload_text.html +++ b/docs/programmer_reference/dumpload_text.html @@ -9,12 +9,12 @@ <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> <link rel="up" href="dumpload.html" title="Chapter 23. Dumping and Reloading Databases" /> <link rel="prev" href="dumpload_format.html" title="Dump output formats" /> - <link rel="next" href="refs.html" title="Chapter 24. Additional References" /> + <link rel="next" href="refs.html" title="Chapter 24. Additional References" /> </head> <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,8 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="dumpload_format.html">Prev</a> </td> - <th width="60%" align="center">Chapter 23. - Dumping and Reloading Databases - </th> + <th width="60%" align="center">Chapter 23. Dumping and Reloading Databases + </th> <td width="20%" align="right"> <a accesskey="n" href="refs.html">Next</a></td> </tr> </table> @@ -38,10 +37,23 @@ </div> </div> </div> - <p>The <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility can be used to load text into databases. The <span class="bold"><strong>-T</strong></span> option permits nondatabase applications to create flat-text files that are then loaded into databases for fast, highly-concurrent access. For example, the following command loads the standard UNIX <code class="filename">/etc/passwd</code> file into a database, with the login name as the key item and the entire password entry as the data item:</p> + <p> + The <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility can be used to load text into databases. The + <span class="bold"><strong>-T</strong></span> option permits + nondatabase applications to create flat-text files that are + then loaded into databases for fast, highly-concurrent access. + For example, the following command loads the standard UNIX + <code class="filename">/etc/passwd</code> file into a database, + with the login name as the key item and the entire password + entry as the data item: + </p> <pre class="programlisting">awk -F: '{print $1; print $0}' < /etc/passwd |\ sed 's/\\/\\\\/g' | db_load -T -t hash passwd.db</pre> - <p>Note that backslash characters naturally occurring in the text are escaped to avoid interpretation as escape characters by the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility.</p> + <p> + Note that backslash characters naturally occurring in the + text are escaped to avoid interpretation as escape characters + by the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility. + </p> </div> <div class="navfooter"> <hr /> @@ -58,9 +70,7 @@ <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 24. - Additional References - </td> + <td width="40%" align="right" valign="top"> Chapter 24. Additional References</td> </tr> </table> </div> diff --git a/docs/programmer_reference/env.html b/docs/programmer_reference/env.html index a1b62389..92b20e6c 100644 --- a/docs/programmer_reference/env.html +++ b/docs/programmer_reference/env.html @@ -14,13 +14,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 9. - The Berkeley DB Environment - </th> + <th colspan="3" align="center">Chapter 9. The Berkeley DB Environment </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="arch_utilities.html">Prev</a> </td> @@ -34,9 +32,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="env"></a>Chapter 9. - The Berkeley DB Environment - </h2> + <h2 class="title"><a id="env"></a>Chapter 9. The Berkeley DB Environment </h2> </div> </div> </div> @@ -52,7 +48,8 @@ </dt> <dt> <span class="sect1"> - <a href="env_create.html">Creating a database environment</a> + <a href="env_create.html">Creating a database + environment</a> </span> </dt> <dt> @@ -62,7 +59,8 @@ </dt> <dt> <span class="sect1"> - <a href="env_open.html">Opening databases within the environment</a> + <a href="env_open.html">Opening databases within the + environment</a> </span> </dt> <dt> @@ -72,7 +70,8 @@ </dt> <dt> <span class="sect1"> - <a href="env_db_config.html">DB_CONFIG configuration file</a> + <a href="env_db_config.html">DB_CONFIG configuration + file</a> </span> </dt> <dt> @@ -84,17 +83,17 @@ <dl> <dt> <span class="sect2"> - <a href="env_naming.html#idp1570112">Specifying file naming to Berkeley DB</a> + <a href="env_naming.html#idp1149160">Specifying file naming to Berkeley DB</a> </span> </dt> <dt> <span class="sect2"> - <a href="env_naming.html#idp1584200">Filename resolution in Berkeley DB</a> + <a href="env_naming.html#idp1160456">Filename resolution in Berkeley DB</a> </span> </dt> <dt> <span class="sect2"> - <a href="env_naming.html#idp1605872">Examples</a> + <a href="env_naming.html#idp1182752">Examples</a> </span> </dt> </dl> @@ -134,35 +133,54 @@ </div> </div> </div> - <p>A Berkeley DB environment is an encapsulation of one or more databases, log -files and region files. Region files are the shared memory areas that -contain information about the database environment such as memory pool -cache pages. Only databases are byte-order independent and only -database files can be moved between machines of different byte orders. -Log files can be moved between machines of the same byte order. Region -files are usually unique to a specific machine and potentially to a -specific operating system release.</p> - <p>The simplest way to administer a Berkeley DB application environment is to -create a single <span class="bold"><strong>home</strong></span> directory that stores the files for the -applications that will share the environment. The environment home -directory must be created before any Berkeley DB applications are run. Berkeley DB -itself never creates the environment home directory. The environment can -then be identified by the name of that directory.</p> - <p>An environment may be shared by any number of processes, as well as by -any number of threads within those processes. It is possible for an -environment to include resources from other directories on the system, -and applications often choose to distribute resources to other -directories or disks for performance or other reasons. However, by -default, the databases, shared regions (the locking, logging, memory -pool, and transaction shared memory areas) and log files will be stored -in a single directory hierarchy.</p> - <p>It is important to realize that all applications sharing a database -environment implicitly trust each other. They have access to each -other's data as it resides in the shared regions, and they will share -resources such as buffer space and locks. At the same time, any -applications using the same databases <span class="bold"><strong>must</strong></span> share an environment -if consistency is to be maintained between them.</p> - <p>For more information on the operations supported by the database environment handle, see the <a href="../api_reference/C/env.html#envlist" class="olink">Database Environments and Related Methods</a> section in the <em class="citetitle">Berkeley DB C API Reference Guide.</em> </p> + <p> + A Berkeley DB environment is an encapsulation of one or more + databases, log files and region files. Region files are the + shared memory areas that contain information about the + database environment such as memory pool cache pages. Only + databases are byte-order independent and only database files + can be moved between machines of different byte orders. Log + files can be moved between machines of the same byte order. + Region files are usually unique to a specific machine and + potentially to a specific operating system release. + </p> + <p> + The simplest way to administer a Berkeley DB application + environment is to create a single <span class="bold"><strong>home</strong></span> + directory that stores the files for the + applications that will share the environment. The environment + home directory must be created before any Berkeley DB + applications are run. Berkeley DB itself never creates the + environment home directory. The environment can then be + identified by the name of that directory. + </p> + <p> + An environment may be shared by any number of processes, as + well as by any number of threads within those processes. It is + possible for an environment to include resources from other + directories on the system, and applications often choose to + distribute resources to other directories or disks for + performance or other reasons. However, by default, the + databases, shared regions (the locking, logging, memory pool, + and transaction shared memory areas) and log files will be + stored in a single directory hierarchy. + </p> + <p> + It is important to realize that all applications sharing a + database environment implicitly trust each other. They have + access to each other's data as it resides in the shared + regions, and they will share resources such as buffer space + and locks. At the same time, any applications using the same + databases <span class="bold"><strong>must</strong></span> share an + environment if consistency is to be maintained between + them. + </p> + <p> + For more information on the operations supported by the + database environment handle, see the <a href="../api_reference/C/env.html#envlist" class="olink">Database Environments and Related + Methods</a> section in the + <em class="citetitle">Berkeley DB C API Reference Guide.</em> + </p> </div> </div> <div class="navfooter"> @@ -178,7 +196,8 @@ if consistency is to be maintained between them.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Creating a database environment</td> + <td width="40%" align="right" valign="top"> Creating a database + environment</td> </tr> </table> </div> diff --git a/docs/programmer_reference/env_create.html b/docs/programmer_reference/env_create.html index 1b48e5d3..931f589a 100644 --- a/docs/programmer_reference/env_create.html +++ b/docs/programmer_reference/env_create.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Creating a database environment</th> + <th colspan="3" align="center">Creating a database + environment</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="env.html">Prev</a> </td> - <th width="60%" align="center">Chapter 9. - The Berkeley DB Environment - </th> + <th width="60%" align="center">Chapter 9. The Berkeley DB Environment </th> <td width="20%" align="right"> <a accesskey="n" href="env_size.html">Next</a></td> </tr> </table> @@ -34,75 +33,120 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="env_create"></a>Creating a database environment</h2> + <h2 class="title" style="clear: both"><a id="env_create"></a>Creating a database + environment</h2> </div> </div> </div> - <p>The Berkeley DB environment is created and described by the <a href="../api_reference/C/envcreate.html" class="olink">db_env_create()</a> -and <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> interfaces. In situations where customization is -desired, such as storing log files on a separate disk drive or selection -of a particular cache size, applications must describe the customization -by either creating an environment configuration file in the environment -home directory or by arguments passed to other <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handle methods.</p> - <p>Once an environment has been created, database files specified using -relative pathnames will be named relative to the home directory. Using -pathnames relative to the home directory allows the entire environment -to be easily moved, simplifying restoration and recovery of a database -in a different directory or on a different system.</p> - <p>Applications first obtain an environment handle using the -<a href="../api_reference/C/envcreate.html" class="olink">db_env_create()</a> method, then call the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method which creates -or joins the database environment. There are a number of options you -can set to customize <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> for your environment. These -options fall into four broad categories:</p> + <p> + The Berkeley DB environment is created and described by the + <a href="../api_reference/C/envcreate.html" class="olink">db_env_create()</a> and <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> interfaces. In situations where + customization is desired, such as storing log files on a + separate disk drive or selection of a particular cache size, + applications must describe the customization by either + creating an environment configuration file in the environment + home directory or by arguments passed to other <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handle + methods. + </p> + <p> + Once an environment has been created, database files + specified using relative pathnames will be named relative to + the home directory. Using pathnames relative to the home + directory allows the entire environment to be easily moved, + simplifying restoration and recovery of a database in a + different directory or on a different system. + </p> + <p> + Applications first obtain an environment handle using the + <a href="../api_reference/C/envcreate.html" class="olink">db_env_create()</a> method, then call the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method which + creates or joins the database environment. There are a number + of options you can set to customize <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> for your + environment. These options fall into four broad + categories: + </p> <div class="variablelist"> <dl> <dt> <span class="term">Subsystem Initialization:</span> </dt> - <dd>These flags indicate which Berkeley DB subsystems will be initialized for the -environment, and what operations will happen automatically when -databases are accessed within the environment. The flags include -<a href="../api_reference/C/envopen.html#envopen_DB_INIT_CDB" class="olink">DB_INIT_CDB</a>, <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOCK" class="olink">DB_INIT_LOCK</a>, <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOG" class="olink">DB_INIT_LOG</a>, <a href="../api_reference/C/envopen.html#envopen_DB_INIT_MPOOL" class="olink">DB_INIT_MPOOL</a>, and <a href="../api_reference/C/envopen.html#envopen_DB_INIT_TXN" class="olink">DB_INIT_TXN</a>. The <a href="../api_reference/C/envopen.html#envopen_DB_INIT_CDB" class="olink">DB_INIT_CDB</a> -flag does initialization for Berkeley DB Concurrent Data Store applications. (See -<a class="xref" href="cam.html#cam_intro" title="Concurrent Data Store introduction">Concurrent Data Store introduction</a> for more -information.) The rest of the flags initialize a single subsystem; that -is, when <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOCK" class="olink">DB_INIT_LOCK</a> is specified, applications reading and -writing databases opened in this environment will be using locking to -ensure that they do not overwrite each other's changes.</dd> + <dd> + These flags indicate which Berkeley DB + subsystems will be initialized for the environment, + and what operations will happen automatically when + databases are accessed within the environment. The + flags include <a href="../api_reference/C/envopen.html#envopen_DB_INIT_CDB" class="olink">DB_INIT_CDB</a>, <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOCK" class="olink">DB_INIT_LOCK</a>, + <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOG" class="olink">DB_INIT_LOG</a>, <a href="../api_reference/C/envopen.html#envopen_DB_INIT_MPOOL" class="olink">DB_INIT_MPOOL</a>, and <a href="../api_reference/C/envopen.html#envopen_DB_INIT_TXN" class="olink">DB_INIT_TXN</a>. The + <a href="../api_reference/C/envopen.html#envopen_DB_INIT_CDB" class="olink">DB_INIT_CDB</a> flag does initialization for Berkeley DB + Concurrent Data Store applications. (See <a class="xref" href="cam.html#cam_intro" title="Concurrent Data Store introduction">Concurrent Data Store + introduction</a> + for more information.) The rest of the flags + initialize a single subsystem; that is, when + <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOCK" class="olink">DB_INIT_LOCK</a> is specified, applications reading and + writing databases opened in this environment will be + using locking to ensure that they do not overwrite + each other's changes. + </dd> <dt> <span class="term">Recovery options:</span> </dt> - <dd>These flags, which include <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> and -<a href="../api_reference/C/envopen.html#envopen_DB_RECOVER_FATAL" class="olink">DB_RECOVER_FATAL</a>, indicate what recovery is to be performed on -the environment before it is opened for normal use.</dd> + <dd> + These flags, which include <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> and + <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER_FATAL" class="olink">DB_RECOVER_FATAL</a>, indicate what recovery is to be + performed on the environment before it is opened for + normal use. + </dd> <dt> <span class="term">Naming options:</span> </dt> - <dd>These flags, which include -<a href="../api_reference/C/envopen.html#envopen_DB_USE_ENVIRON" class="olink">DB_USE_ENVIRON</a> and <a href="../api_reference/C/envopen.html#envopen_DB_USE_ENVIRON_ROOT" class="olink">DB_USE_ENVIRON_ROOT</a>, modify how file naming happens in the -environment.</dd> + <dd> + These flags, which include <a href="../api_reference/C/envopen.html#envopen_DB_USE_ENVIRON" class="olink">DB_USE_ENVIRON</a> and + <a href="../api_reference/C/envopen.html#envopen_DB_USE_ENVIRON_ROOT" class="olink">DB_USE_ENVIRON_ROOT</a>, modify how file naming happens + in the environment. + </dd> <dt> <span class="term">Miscellaneous:</span> </dt> - <dd>Finally, there are a number of miscellaneous flags, for example, -<a href="../api_reference/C/dbopen.html#open_DB_CREATE" class="olink">DB_CREATE</a> which causes underlying files to be created as -necessary. See the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> manual pages for further -information.</dd> + <dd> + Finally, there are a number of miscellaneous + flags, for example, <a href="../api_reference/C/dbopen.html#open_DB_CREATE" class="olink">DB_CREATE</a> which causes + underlying files to be created as necessary. See the + <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> manual pages for further + information. + </dd> </dl> </div> - <p>Most applications either specify only the <a href="../api_reference/C/envopen.html#envopen_DB_INIT_MPOOL" class="olink">DB_INIT_MPOOL</a> flag or -they specify all four subsystem initialization flags -(<a href="../api_reference/C/envopen.html#envopen_DB_INIT_MPOOL" class="olink">DB_INIT_MPOOL</a>, <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOCK" class="olink">DB_INIT_LOCK</a>, <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOG" class="olink">DB_INIT_LOG</a>, and <a href="../api_reference/C/envopen.html#envopen_DB_INIT_TXN" class="olink">DB_INIT_TXN</a>). -The former configuration is for applications that -simply want to use the basic Access Method interfaces with a shared -underlying buffer pool, but don't care about recoverability after -application or system failure. The latter is for applications that need -recoverability. There are situations in which other combinations of -the initialization flags make sense, but they are rare.</p> - <p>The <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> flag is specified by applications that want to perform any necessary database recovery when they start running. That is, if there was a system or application failure the last time they ran, they want the databases to be made consistent before they start running again. It is not an error to specify this flag when no recovery needs to be done.</p> - <p>The <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER_FATAL" class="olink">DB_RECOVER_FATAL</a> flag is more special-purpose. It performs catastrophic database recovery, and normally requires that some initial arrangements be made; that is, archived log files be brought back into the filesystem. Applications should not normally specify this flag. Instead, under these rare conditions, the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility should be used.</p> - <p>The following is a simple example of a function that opens a database -environment for a transactional program.</p> + <p> + Most applications either specify only the <a href="../api_reference/C/envopen.html#envopen_DB_INIT_MPOOL" class="olink">DB_INIT_MPOOL</a> + flag or they specify all four subsystem initialization flags + (<a href="../api_reference/C/envopen.html#envopen_DB_INIT_MPOOL" class="olink">DB_INIT_MPOOL</a>, <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOCK" class="olink">DB_INIT_LOCK</a>, <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOG" class="olink">DB_INIT_LOG</a>, and + <a href="../api_reference/C/envopen.html#envopen_DB_INIT_TXN" class="olink">DB_INIT_TXN</a>). The former configuration is for applications + that simply want to use the basic Access Method interfaces + with a shared underlying buffer pool, but don't care about + recoverability after application or system failure. The latter + is for applications that need recoverability. There are + situations in which other combinations of the initialization + flags make sense, but they are rare. + </p> + <p> + The <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> flag is specified by applications that want + to perform any necessary database recovery when they start + running. That is, if there was a system or application failure + the last time they ran, they want the databases to be made + consistent before they start running again. It is not an error + to specify this flag when no recovery needs to be done. + </p> + <p> + The <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER_FATAL" class="olink">DB_RECOVER_FATAL</a> flag is more special-purpose. It + performs catastrophic database recovery, and normally requires + that some initial arrangements be made; that is, archived log + files be brought back into the filesystem. Applications should + not normally specify this flag. Instead, under these rare + conditions, the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility should be used. + </p> + <p> + The following is a simple example of a function that opens a + database environment for a transactional program. + </p> <a id="prog_env27"></a> <pre class="programlisting">DB_ENV * db_setup(char *home, char *data_dir, FILE *errfp, char *progname) @@ -130,8 +174,8 @@ db_setup(char *home, char *data_dir, FILE *errfp, char *progname) dbenv->err(dbenv, ret, "set_cachesize"); goto err; } - if ((ret = dbenv->set_data_dir(dbenv, data_dir)) != 0) { - dbenv->err(dbenv, ret, "set_data_dir: %s", data_dir); + if ((ret = dbenv->add_data_dir(dbenv, data_dir)) != 0) { + dbenv->err(dbenv, ret, "add_data_dir: %s", data_dir); goto err; } @@ -160,9 +204,7 @@ err: (void)dbenv->close(dbenv, 0); <td width="40%" align="right"> <a accesskey="n" href="env_size.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 9. - The Berkeley DB Environment - </td> + <td width="40%" align="left" valign="top">Chapter 9. The Berkeley DB Environment </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/env_db_config.html b/docs/programmer_reference/env_db_config.html index bd9add35..9f6d7c2f 100644 --- a/docs/programmer_reference/env_db_config.html +++ b/docs/programmer_reference/env_db_config.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">DB_CONFIG configuration file</th> + <th colspan="3" align="center">DB_CONFIG configuration + file</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="env_error.html">Prev</a> </td> - <th width="60%" align="center">Chapter 9. - The Berkeley DB Environment - </th> + <th width="60%" align="center">Chapter 9. The Berkeley DB Environment </th> <td width="20%" align="right"> <a accesskey="n" href="env_naming.html">Next</a></td> </tr> </table> @@ -34,40 +33,43 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="env_db_config"></a>DB_CONFIG configuration file</h2> + <h2 class="title" style="clear: both"><a id="env_db_config"></a>DB_CONFIG configuration + file</h2> </div> </div> </div> <p> - Almost all of the configuration information that can be specified to - <a href="../api_reference/C/env.html" class="olink">DB_ENV class</a> methods can also be specified using a configuration file. - If a file named DB_CONFIG exists in the database home directory, it - will be read for lines of the format - <span class="bold"><strong>NAME VALUE</strong></span>. -</p> - <p> - One or more whitespace characters are used to delimit the two parts of - the line, and trailing whitespace characters are discarded. All empty - lines or lines whose first character is a whitespace or hash - (<span class="bold"><strong>#</strong></span>) character will be ignored. Each - line must specify both the NAME and the VALUE of the pair. The - specific NAME VALUE pairs are documented in the manual for the - corresponding methods (for example, the <a href="../api_reference/C/envset_data_dir.html" class="olink">DB_ENV->set_data_dir()</a> documentation - includes NAME VALUE pair information Berkeley DB administrators can use - to configure locations for database files). -</p> - <p><a id="env_db_config.DB_CONFIG"></a> - The DB_CONFIG configuration file is intended to allow database - environment administrators to customize environments independent of - applications using the environment. For example, a database - administrator can move the database log and data files to a different - location without application recompilation. In addition, because the - DB_CONFIG file is read when the database environment is opened, it can - be used to overrule application configuration done before that time. - For example a database administrator could override the compiled-in - application cache size to a size more appropriate for a specific - machine. -</p> + Almost all of the configuration information that can be + specified to <a href="../api_reference/C/env.html" class="olink">DB_ENV class</a> methods can also be specified using + a configuration file. If a file named DB_CONFIG exists in the + database home directory, it will be read for lines of the + format <span class="bold"><strong>NAME VALUE</strong></span>. + </p> + <p> + One or more whitespace characters are used to delimit the + two parts of the line, and trailing whitespace characters are + discarded. All empty lines or lines whose first character is a + whitespace or hash (<span class="bold"><strong>#</strong></span>) + character will be ignored. Each line must specify both the + NAME and the VALUE of the pair. The specific NAME VALUE pairs + are documented in the manual for the corresponding methods + (for example, the <a href="../api_reference/C/envadd_data_dir.html" class="olink">DB_ENV->add_data_dir()</a> documentation includes + NAME VALUE pair information Berkeley DB administrators can use + to configure locations for database files). + </p> + <p><a id="env_db_config.DB_CONFIG"></a> + The DB_CONFIG configuration + file is intended to allow database environment administrators + to customize environments independent of applications using + the environment. For example, a database administrator can + move the database log and data files to a different location + without application recompilation. In addition, because the + DB_CONFIG file is read when the database environment is + opened, it can be used to overrule application configuration + done before that time. For example a database administrator + could override the compiled-in application cache size to a + size more appropriate for a specific machine. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/env_encrypt.html b/docs/programmer_reference/env_encrypt.html index 5c81dbed..6470fe9e 100644 --- a/docs/programmer_reference/env_encrypt.html +++ b/docs/programmer_reference/env_encrypt.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="env_security.html">Prev</a> </td> - <th width="60%" align="center">Chapter 9. - The Berkeley DB Environment - </th> + <th width="60%" align="center">Chapter 9. The Berkeley DB Environment </th> <td width="20%" align="right"> <a accesskey="n" href="env_remote.html">Next</a></td> </tr> </table> @@ -39,63 +37,65 @@ </div> </div> <p> - Berkeley DB optionally supports encryption using the Rijndael/AES - (also known as the Advanced Encryption Standard and Federal - Information Processing Standard (FIPS) 197) algorithm for - encryption or decryption. The algorithm is configured to use a - 128-bit key. Berkeley DB uses a 16-byte initialization vector - generated using the Mersenne Twister. All encrypted information is - additionally checksummed using the SHA1 Secure Hash Algorithm, - using a 160-bit message digest. + Berkeley DB optionally supports encryption using the + Rijndael/AES (also known as the Advanced Encryption Standard + and Federal Information Processing Standard (FIPS) 197) + algorithm for encryption or decryption. The algorithm is + configured to use a 128-bit key. Berkeley DB uses a 16-byte + initialization vector generated using the Mersenne Twister. + All encrypted information is additionally checksummed using + the SHA1 Secure Hash Algorithm, using a 160-bit message + digest. </p> <p> - The encryption support provided with Berkeley DB is intended to - protect applications from an attacker obtaining physical access to - the media on which a Berkeley DB database is stored, or an attacker - compromising a system on which Berkeley DB is running but who is - unable to read system or process memory on that system. - <span class="bold"><strong> - The encryption support provided with Berkeley DB will not - protect applications from attackers able to read system memory - on the system where Berkeley DB is running. + The encryption support provided with Berkeley DB is + intended to protect applications from an attacker obtaining + physical access to the media on which a Berkeley DB database + is stored, or an attacker compromising a system on which + Berkeley DB is running but who is unable to read system or + process memory on that system. <span class="bold"><strong> The + encryption support provided with Berkeley DB will not + protect applications from attackers able to read system + memory on the system where Berkeley DB is running. </strong></span> </p> - <p> + <p> To encrypt a database, you must configure the database for encryption prior to creating it. If you are using a database environment, you must also configure the environment for - encryption. In order to create an encrypted database within an - environment, you: + encryption. In order to create an encrypted database within an + environment, you: </p> <div class="orderedlist"> <ol type="1"> <li> - <p> - Configure the environment for encryption using the - <a href="../api_reference/C/envset_encrypt.html" class="olink">DB_ENV->set_encrypt()</a> method. + <p> + Configure the environment for encryption using the + <a href="../api_reference/C/envset_encrypt.html" class="olink">DB_ENV->set_encrypt()</a> method. </p> </li> <li> <p> - Open the database environment. + Open the database environment. </p> </li> <li> - <p> - Specify the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_ENCRYPT" class="olink">DB_ENCRYPT</a> flag to the database handle. + <p> + Specify the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_ENCRYPT" class="olink">DB_ENCRYPT</a> flag to the database + handle. </p> </li> <li> - <p> + <p> Open the database. </p> </li> </ol> </div> - <p> - Once you have done that, all of the databases that you create in - the environment are encrypted/decrypted by the password you specify - using the <a href="../api_reference/C/envset_encrypt.html" class="olink">DB_ENV->set_encrypt()</a> method. + <p> + Once you have done that, all of the databases that you + create in the environment are encrypted/decrypted by the + password you specify using the <a href="../api_reference/C/envset_encrypt.html" class="olink">DB_ENV->set_encrypt()</a> method. </p> <p> For databases not created in an environment: @@ -103,13 +103,14 @@ <div class="orderedlist"> <ol type="1"> <li> - <p> - Specify the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_ENCRYPT" class="olink">DB_ENCRYPT</a> flag to the database handle. + <p> + Specify the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_ENCRYPT" class="olink">DB_ENCRYPT</a> flag to the database + handle. </p> </li> <li> <p> - Call the <a href="../api_reference/C/dbset_encrypt.html" class="olink">DB->set_encrypt()</a> method. + Call the <a href="../api_reference/C/dbset_encrypt.html" class="olink">DB->set_encrypt()</a> method. </p> </li> <li> @@ -119,89 +120,100 @@ </li> </ol> </div> - <p> + <p> Note that databases cannot be converted to an encrypted - format after they have been created without dumping and re-creating - them. Finally, encrypted databases cannot be read on systems with - a different endianness than the system that created the encrypted - database. + format after they have been created without dumping and + re-creating them. Finally, encrypted databases cannot be read + on systems with a different endianness than the system that + created the encrypted database. </p> <p> - Each encrypted database environment (including all its encrypted - databases) is encrypted using a single password and a single - algorithm. Applications wanting to provide a finer granularity of - database access must either use multiple database environments or - implement additional access controls outside of Berkeley DB. + When a database is encrypted, its log files are also encrypted, so + accessing the logs also requires the encryption key. By default, + logs are placed in the same directory as the environment. When + using the SQL API, the logs are placed in the journal directory. + Encrypted log files should never be simply deleted. For + instructions on how to properly remove log files see, + <a class="xref" href="transapp_logfile.html" title="Log file removal">Log file removal</a>. </p> <p> + Each encrypted database environment (including all its + encrypted databases) is encrypted using a single password and + a single algorithm. Applications wanting to provide a finer + granularity of database access must either use multiple + database environments or implement additional access controls + outside of Berkeley DB. + </p> + <p> The only encrypted parts of a database environment are its - databases and its log files. Specifically, the - <a class="xref" href="env_region.html" title="Shared memory regions">Shared memory regions</a> supporting - the database environment are not encrypted. For this reason, it - may be possible for an attacker to read some or all of an encrypted - database by reading the on-disk files that back these shared memory - regions. To prevent such attacks, applications may want to use - in-memory filesystem support (on systems that support it), or the - <a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a> or <a href="../api_reference/C/envopen.html#envopen_DB_SYSTEM_MEM" class="olink">DB_SYSTEM_MEM</a> flags to the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method, to - place the shared memory regions in memory that is never written to - a disk. As some systems page system memory to a backing disk, it - is important to consider the specific operating system running on - the machine as well. Finally, when backing database environment - shared regions with the filesystem, Berkeley DB can be configured - to overwrite the shared regions before removing them by specifying - the <a href="../api_reference/C/envset_flags.html#set_flags_DB_OVERWRITE" class="olink">DB_OVERWRITE</a> flag. This option is only effective in the - presence of fixed-block filesystems, journaling or logging - filesystems will require operating system support and probably - modification of the Berkeley DB sources. + databases and its log files. Specifically, the <a class="xref" href="env_region.html" title="Shared memory regions">Shared memory regions</a> + supporting the database environment are not encrypted. For + this reason, it may be possible for an attacker to read some + or all of an encrypted database by reading the on-disk files + that back these shared memory regions. To prevent such + attacks, applications may want to use in-memory filesystem + support (on systems that support it), or the <a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a> or + <a href="../api_reference/C/envopen.html#envopen_DB_SYSTEM_MEM" class="olink">DB_SYSTEM_MEM</a> flags to the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method, to place the + shared memory regions in memory that is never written to a + disk. As some systems page system memory to a backing disk, it + is important to consider the specific operating system running + on the machine as well. Finally, when backing database + environment shared regions with the filesystem, Berkeley DB + can be configured to overwrite the shared regions before + removing them by specifying the <a href="../api_reference/C/envset_flags.html#set_flags_DB_OVERWRITE" class="olink">DB_OVERWRITE</a> flag. This + option is only effective in the presence of fixed-block + filesystems, journaling or logging filesystems will require + operating system support and probably modification of the + Berkeley DB sources. </p> <p> - While all user data is encrypted, parts of the databases and log - files in an encrypted environment are maintained in an unencrypted - state. Specifically, log record headers are not encrypted, only - the actual log records. Additionally, database internal page - header fields are not encrypted. These page header fields includes - information such as the page's <a href="../api_reference/C/lsn.html" class="olink">DB_LSN</a> number and position in the - database's sort order. + While all user data is encrypted, parts of the databases + and log files in an encrypted environment are maintained in an + unencrypted state. Specifically, log record headers are not + encrypted, only the actual log records. Additionally, database + internal page header fields are not encrypted. These page + header fields includes information such as the page's <a href="../api_reference/C/lsn.html" class="olink">DB_LSN</a> + number and position in the database's sort order. </p> - <p> - Log records distributed by a replication master to replicated - clients are transmitted to the clients in unencrypted form. If - encryption is desired in a replicated application, the use of a - secure transport is strongly suggested. + <p> + Log records distributed by a replication master to + replicated clients are transmitted to the clients in + unencrypted form. If encryption is desired in a replicated + application, the use of a secure transport is strongly + suggested. </p> <p> - We gratefully acknowledge: + We gratefully acknowledge: </p> <div class="itemizedlist"> <ul type="disc"> <li> - Vincent Rijmen, Antoon Bosselaers and Paulo Barreto for writing - the Rijndael/AES code used in Berkeley DB. + Vincent Rijmen, Antoon Bosselaers and Paulo Barreto + for writing the Rijndael/AES code used in Berkeley DB. </li> <li> - Steve Reid and James H. Brown for writing the SHA1 checksum - code used in Berkeley DB. + Steve Reid and James H. Brown for writing the SHA1 + checksum code used in Berkeley DB. </li> <li> - Makoto Matsumoto and Takuji Nishimura for writing the Mersenne - Twister code used in Berkeley DB. + Makoto Matsumoto and Takuji Nishimura for writing + the Mersenne Twister code used in Berkeley DB. </li> <li> - Adam Stubblefield for integrating the Rijndael/AES, SHA1 - checksum and Mersenne Twister code into Berkeley DB. + Adam Stubblefield for integrating the Rijndael/AES, + SHA1 checksum and Mersenne Twister code into Berkeley DB. </li> </ul> </div> - <p> + <p> Berkeley DB 11g Release 2 supports encryption using Intel's Performance Primitive (IPP) on Linux. This works only on Intel - processors. To use Berkeley DB with IPP encryption, you must have - IPP installed along with the cryptography extension. The IPP - performance is higher in most cases compared to the current AES - implementation. See - <a href="../installation/build_unix_conf.html#build_unix_conf.--with-cryptography" class="olink">--with-cryptography</a> - for more information. See the - <a class="ulink" href="http://software.intel.com/en-us/articles/intel-integrated-performance-primitives-documentation/" target="_top"> + processors. To use Berkeley DB with IPP encryption, you must + have IPP installed along with the cryptography extension. The + IPP performance is higher in most cases compared to the + current AES implementation. See <a href="../installation/build_unix_conf.html#build_unix_conf.--with-cryptography" class="olink">--with-cryptography</a> + for more information. See the + <a class="ulink" href="https://software.intel.com/en-us/articles/intel-integrated-performance-primitives-documentation/" target="_top"> Intel Documenation</a> for more information on IPP. </p> </div> diff --git a/docs/programmer_reference/env_error.html b/docs/programmer_reference/env_error.html index 8170aebc..ba1e49a5 100644 --- a/docs/programmer_reference/env_error.html +++ b/docs/programmer_reference/env_error.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="env_open.html">Prev</a> </td> - <th width="60%" align="center">Chapter 9. - The Berkeley DB Environment - </th> + <th width="60%" align="center">Chapter 9. The Berkeley DB Environment </th> <td width="20%" align="right"> <a accesskey="n" href="env_db_config.html">Next</a></td> </tr> </table> @@ -38,12 +36,17 @@ </div> </div> </div> - <p>Berkeley DB offers programmatic support for displaying error return values. -The <a href="../api_reference/C/envstrerror.html" class="olink">db_strerror()</a> function returns a pointer to the error -message corresponding to any Berkeley DB error return. This is similar to the -ANSI C strerror interface, but can handle both system error returns and -Berkeley DB-specific return values.</p> - <p>For example:</p> + <p> + Berkeley DB offers programmatic support for displaying error + return values. The <a href="../api_reference/C/envstrerror.html" class="olink">db_strerror()</a> function returns a pointer to + the error message corresponding to any Berkeley DB error + return. This is similar to the ANSI C strerror interface, but + can handle both system error returns and Berkeley DB-specific + return values. + </p> + <p> + For example: + </p> <a id="prog_env29"></a> <pre class="programlisting">int ret; if ((ret = dbenv->set_cachesize(dbenv, 0, 32 * 1024, 1)) != 0) { @@ -51,15 +54,28 @@ if ((ret = dbenv->set_cachesize(dbenv, 0, 32 * 1024, 1)) != 0) { return (1); } </pre> - <p>There are also two additional error methods: <a href="../api_reference/C/enverr.html" class="olink">DB_ENV->err()</a> and -<code class="methodname">DB_ENV->errx()</code>. These methods work like the ANSI C printf function, -taking a printf-style format string and argument list, and writing a -message constructed from the format string and arguments.</p> - <p>The <a href="../api_reference/C/enverr.html" class="olink">DB_ENV->err()</a> function appends the standard error string to the -constructed message; the <code class="methodname">DB_ENV->errx()</code> function does not.</p> - <p>Error messages can be configured always to include a prefix (for -example, the program name) using the <a href="../api_reference/C/envset_errpfx.html" class="olink">DB_ENV->set_errpfx()</a> method.</p> - <p>These functions provide simpler ways of displaying Berkeley DB error messages:</p> + <p> + There are also two additional error methods: <a href="../api_reference/C/enverr.html" class="olink">DB_ENV->err()</a> and + <code class="methodname">DB_ENV->errx()</code>. These methods + work like the ANSI C printf function, taking a printf-style + format string and argument list, and writing a message + constructed from the format string and arguments. + </p> + <p> + The <a href="../api_reference/C/enverr.html" class="olink">DB_ENV->err()</a> function appends the standard error string to + the constructed message; the + <code class="methodname">DB_ENV->errx()</code> function does + not. + </p> + <p> + Error messages can be configured always to include a prefix + (for example, the program name) using the <a href="../api_reference/C/envset_errpfx.html" class="olink">DB_ENV->set_errpfx()</a> + method. + </p> + <p> + These functions provide simpler ways of displaying Berkeley + DB error messages: + </p> <a id="prog_env30"></a> <pre class="programlisting">int ret; ... @@ -73,9 +89,12 @@ if ((ret = dbenv->open(dbenv, home, session_id); return (1); }</pre> - <p>For example, if the program was called "my_app", and it tried to open -an environment home directory in "/tmp/home" and the open call returned -a permission error, the error messages shown would look like this:</p> + <p> + For example, if the program was called "my_app", and it + tried to open an environment home directory in "/tmp/home" and + the open call returned a permission error, the error messages + shown would look like this: + </p> <pre class="programlisting">my_app: open: /tmp/home: Permission denied. my_app: contact your system administrator: session ID was 2</pre> </div> @@ -90,11 +109,13 @@ my_app: contact your system administrator: session ID was 2</pre> <td width="40%" align="right"> <a accesskey="n" href="env_db_config.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Opening databases within the environment </td> + <td width="40%" align="left" valign="top">Opening databases within the + environment </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> DB_CONFIG configuration file</td> + <td width="40%" align="right" valign="top"> DB_CONFIG configuration + file</td> </tr> </table> </div> diff --git a/docs/programmer_reference/env_faq.html b/docs/programmer_reference/env_faq.html index 7775c3d3..be1a33aa 100644 --- a/docs/programmer_reference/env_faq.html +++ b/docs/programmer_reference/env_faq.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="env_remote.html">Prev</a> </td> - <th width="60%" align="center">Chapter 9. - The Berkeley DB Environment - </th> + <th width="60%" align="center">Chapter 9. The Berkeley DB Environment </th> <td width="20%" align="right"> <a accesskey="n" href="cam.html">Next</a></td> </tr> </table> @@ -42,40 +40,50 @@ <ol type="1"> <li> <p> - <span class="bold"><strong>I'm using multiple processes to access an Berkeley DB database - environment; is there any way to ensure that two processes don't run transactional - recovery at the same time, or that all processes have exited the database - environment so that recovery can be run?</strong></span> - </p> + <span class="bold"><strong>I'm using multiple processes to + access an Berkeley DB database environment; is + there any way to ensure that two processes don't + run transactional recovery at the same time, or + that all processes have exited the database + environment so that recovery can be + run?</strong></span> + </p> <p> - See <a class="xref" href="transapp_fail.html" title="Handling failure in Transactional Data Store applications">Handling failure in Transactional Data Store applications</a> and - <a class="xref" href="transapp_app.html" title="Architecting Transactional Data Store applications">Architecting Transactional Data Store applications</a> for a full - discussion of this topic. - </p> + See <a class="xref" href="transapp_fail.html" title="Handling failure in Transactional Data Store applications">Handling failure in Transactional Data Store applications</a> and <a class="xref" href="transapp_app.html" title="Architecting Transactional Data Store applications">Architecting Transactional Data + Store applications</a> for a full + discussion of this topic. + </p> </li> <li> <p> - <span class="bold"><strong>How can I associate application information with a <a href="../api_reference/C/db.html" class="olink">DB</a> or - <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handle?</strong></span> - </p> + <span class="bold"><strong>How can I associate application + information with a <a href="../api_reference/C/db.html" class="olink">DB</a> or <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> + handle?</strong></span> + </p> <p> - In the C API, the <a href="../api_reference/C/db.html" class="olink">DB</a> and <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> structures each contain an "app_private" field intended - to be used to reference application-specific information. See the <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a> and - <a href="../api_reference/C/envcreate.html" class="olink">db_env_create()</a> documentation for more information. - </p> + In the C API, the <a href="../api_reference/C/db.html" class="olink">DB</a> and <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> structures each + contain an "app_private" field intended to be used to + reference application-specific information. See the + <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a> and <a href="../api_reference/C/envcreate.html" class="olink">db_env_create()</a> documentation for more + information. + </p> <p> - In the C++ or Java APIs, the easiest way to associate application-specific data with a - handle is to subclass the <a href="../api_reference/CXX/db.html" class="olink">Db</a> or <a href="../api_reference/CXX/env.html" class="olink">DbEnv</a>, for example subclassing - <a href="../api_reference/CXX/db.html" class="olink">Db</a> to get MyDb. Objects of type MyDb will still have the Berkeley DB API - methods available on them, and you can put any extra data or methods you want into the - MyDb class. If you are using "callback" APIs that take <a href="../api_reference/CXX/db.html" class="olink">Db</a> or - <a href="../api_reference/CXX/env.html" class="olink">DbEnv</a> arguments (for example, - <a href="../api_reference/CXX/dbset_bt_compare.html" class="olink">Db::set_bt_compare()</a>) - these will always be called with the <a href="../api_reference/CXX/db.html" class="olink">Db</a> or <a href="../api_reference/CXX/env.html" class="olink">DbEnv</a> objects you - create. So if you always use MyDb objects, you will be able to take the first argument - to the callback function and cast it to a MyDb (in C++, cast it to (MyDb*)). That will - allow you to access your data members or methods. - </p> + In the C++ or Java APIs, the easiest way to + associate application-specific data with a handle is + to subclass the <a href="../api_reference/CXX/db.html" class="olink">Db</a> or <a href="../api_reference/CXX/env.html" class="olink">DbEnv</a>, for + example subclassing <a href="../api_reference/CXX/db.html" class="olink">Db</a> to get MyDb. + Objects of type MyDb will still have the Berkeley DB + API methods available on them, and you can put any + extra data or methods you want into the MyDb class. If + you are using "callback" APIs that take <a href="../api_reference/CXX/db.html" class="olink">Db</a> + or <a href="../api_reference/CXX/env.html" class="olink">DbEnv</a> arguments (for example, <a href="../api_reference/CXX/dbset_bt_compare.html" class="olink">Db::set_bt_compare()</a>) + these will always be called with the <a href="../api_reference/CXX/db.html" class="olink">Db</a> or <a href="../api_reference/CXX/env.html" class="olink">DbEnv</a> + objects you create. So if you always use MyDb objects, + you will be able to take the first argument to the + callback function and cast it to a MyDb (in C++, cast + it to (MyDb*)). That will allow you to access your + data members or methods. + </p> </li> </ol> </div> @@ -95,9 +103,8 @@ <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 10. - Berkeley DB Concurrent Data Store Applications - </td> + <td width="40%" align="right" valign="top"> Chapter 10. Berkeley DB Concurrent Data Store + Applications </td> </tr> </table> </div> diff --git a/docs/programmer_reference/env_naming.html b/docs/programmer_reference/env_naming.html index e05a12e7..c5353bc2 100644 --- a/docs/programmer_reference/env_naming.html +++ b/docs/programmer_reference/env_naming.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="env_db_config.html">Prev</a> </td> - <th width="60%" align="center">Chapter 9. - The Berkeley DB Environment - </th> + <th width="60%" align="center">Chapter 9. The Berkeley DB Environment </th> <td width="20%" align="right"> <a accesskey="n" href="env_region.html">Next</a></td> </tr> </table> @@ -42,143 +40,152 @@ <dl> <dt> <span class="sect2"> - <a href="env_naming.html#idp1570112">Specifying file naming to Berkeley DB</a> + <a href="env_naming.html#idp1149160">Specifying file naming to Berkeley DB</a> </span> </dt> <dt> <span class="sect2"> - <a href="env_naming.html#idp1584200">Filename resolution in Berkeley DB</a> + <a href="env_naming.html#idp1160456">Filename resolution in Berkeley DB</a> </span> </dt> <dt> <span class="sect2"> - <a href="env_naming.html#idp1605872">Examples</a> + <a href="env_naming.html#idp1182752">Examples</a> </span> </dt> </dl> </div> <p> - One of the most important tasks of the database environment is to - structure file naming within Berkeley DB. Cooperating applications (or - multiple invocations of the same application) must agree on the - location of the database environment, log files and other files used by - the Berkeley DB subsystems, and, of course, the database files. - Although it is possible to specify full pathnames to all Berkeley DB - methods, this is cumbersome and requires applications be recompiled - when database files are moved. -</p> - <p> - Applications are normally expected to specify a single directory home - for the database environment. This can be done easily in the call to - <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> by specifying a value for the - <span class="bold"><strong>db_home</strong></span> argument. There are more - complex configurations in which it may be desirable to override - <span class="bold"><strong>db_home</strong></span> or provide supplementary path - information. -</p> + One of the most important tasks of the database environment + is to structure file naming within Berkeley DB. Cooperating + applications (or multiple invocations of the same application) + must agree on the location of the database environment, log + files and other files used by the Berkeley DB subsystems, and, + of course, the database files. Although it is possible to + specify full pathnames to all Berkeley DB methods, this is + cumbersome and requires applications be recompiled when + database files are moved. + </p> + <p> + Applications are normally expected to specify a single + directory home for the database environment. This can be done + easily in the call to <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> by specifying a value for the + <span class="bold"><strong>db_home</strong></span> argument. There + are more complex configurations in which it may be desirable + to override <span class="bold"><strong>db_home</strong></span> or + provide supplementary path information. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1570112"></a>Specifying file naming to Berkeley DB</h3> + <h3 class="title"><a id="idp1149160"></a>Specifying file naming to Berkeley DB</h3> </div> </div> </div> <p> - The following list describes the possible ways in which file naming - information may be specified to the Berkeley DB library. The - specific circumstances and order in which these ways are applied - are described in a subsequent paragraph. - </p> + The following list describes the possible ways in which + file naming information may be specified to the Berkeley + DB library. The specific circumstances and order in which + these ways are applied are described in a subsequent + paragraph. + </p> <div class="variablelist"> <dl> <dt> <span class="term">db_home</span> </dt> <dd> - If the <span class="bold"><strong>db_home</strong></span> argument to - <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> is non-NULL, its value may be used as the - database home, and files named relative to its path. - </dd> + If the <span class="bold"><strong>db_home</strong></span> argument + to <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> is + non-NULL, its value may be used as the database + home, and files named relative to its path. + </dd> <dt> <span class="term">DB_HOME</span> </dt> <dd> <p> - If the DB_HOME environment variable is set when - <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> is called, its value may be used as the - database home, and files named relative to its - path. - </p> + If the DB_HOME environment variable is set + when <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> is called, its value may be + used as the database home, and files named + relative to its path. + </p> <p> - The DB_HOME environment variable is intended to permit - users and system administrators to override application - and installation defaults. For example: - </p> + The DB_HOME environment variable is + intended to permit users and system + administrators to override application and + installation defaults. For example: + </p> <pre class="programlisting">env DB_HOME=/database/my_home application</pre> - <p> - Application writers are encouraged to support the - <span class="bold"><strong>-h</strong></span> option found in the - supporting Berkeley DB utilities to let users specify a - database home. - </p> + <p> + Application writers are encouraged to + support the <span class="bold"><strong>-h</strong></span> + option found in the + supporting Berkeley DB utilities to let users + specify a database home. + </p> </dd> <dt> <span class="term"><a href="../api_reference/C/env.html" class="olink">DB_ENV</a> methods</span> </dt> <dd> - <p> - There are four <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> methods that affect file naming: - </p> + <p> + There are four <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> methods that affect + file naming: + </p> <div class="itemizedlist"> <ul type="disc"> <li> - <p> - The <a href="../api_reference/C/envadd_data_dir.html" class="olink">DB_ENV->add_data_dir()</a> method specifies a - directory to search for database files. - </p> + <p> + The <a href="../api_reference/C/envadd_data_dir.html" class="olink">DB_ENV->add_data_dir()</a> method + specifies a directory to search for + database files. + </p> </li> <li> - <p> - The <a href="../api_reference/C/envset_lg_dir.html" class="olink">DB_ENV->set_lg_dir()</a> method specifies a - directory in which to create logging files. - </p> + <p> + The <a href="../api_reference/C/envset_lg_dir.html" class="olink">DB_ENV->set_lg_dir()</a> method + specifies a directory in which to + create logging files. + </p> </li> <li> <p> - The <a href="../api_reference/C/envset_tmp_dir.html" class="olink">DB_ENV->set_tmp_dir()</a> method specifies a - directory in which to create backing temporary - files. - </p> + The <a href="../api_reference/C/envset_tmp_dir.html" class="olink">DB_ENV->set_tmp_dir()</a> method + specifies a directory in which to + create backing temporary files. + </p> </li> <li> <p> - The <a href="../api_reference/C/envset_metadata_dir.html" class="olink">DB_ENV->set_metadata_dir()</a> method specifies the - directory in which to create persistent - metadata files used by the environment. - </p> + The <a href="../api_reference/C/envset_metadata_dir.html" class="olink">DB_ENV->set_metadata_dir()</a> method + specifies the directory in which to + create persistent metadata files used + by the environment. + </p> </li> </ul> </div> - <p> - These methods are intended to permit applications to - customize a file locations for an environment. For example, - an application writer can place data files and log - files in different directories or instantiate a new log - directory each time the application runs. - </p> + <p> + These methods are intended to permit + applications to customize a file locations for + an environment. For example, an application + writer can place data files and log files in + different directories or instantiate a new log + directory each time the application runs. + </p> </dd> <dt> <span class="term"> - <a class="link" href="env_db_config.html" title="DB_CONFIG configuration file">DB_CONFIG</a> - </span> + <a class="link" href="env_db_config.html" title="DB_CONFIG configuration file">DB_CONFIG</a> + </span> </dt> - <dd> - The same information specified to the <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> methods may - also be specified using the - <a class="link" href="env_db_config.html" title="DB_CONFIG configuration file">DB_CONFIG</a> - configuration file. - </dd> + <dd> + The same information specified to the <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> + methods may also be specified using the <a class="link" href="env_db_config.html" title="DB_CONFIG configuration file">DB_CONFIG</a> + configuration file. + </dd> </dl> </div> </div> @@ -186,140 +193,146 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1584200"></a>Filename resolution in Berkeley DB</h3> + <h3 class="title"><a id="idp1160456"></a>Filename resolution in Berkeley DB</h3> </div> </div> </div> <p> - The following list describes the specific circumstances and order - in which the different ways of specifying file naming information - are applied. Berkeley DB filename processing proceeds sequentially - through the following steps: - </p> + The following list describes the specific circumstances + and order in which the different ways of specifying file + naming information are applied. Berkeley DB filename + processing proceeds sequentially through the following + steps: + </p> <div class="variablelist"> <dl> <dt> <span class="term">absolute pathnames</span> </dt> <dd> - <p> - If the filename specified to a Berkeley DB function is - an <span class="emphasis"><em>absolute pathname</em></span>, that - filename is used without modification by Berkeley DB. - </p> - <p> - On UNIX systems, an absolute pathname is defined as any - pathname that begins with a leading slash - (<span class="bold"><strong>/</strong></span>). - </p> - <p> - On Windows systems, an absolute pathname is any - pathname that begins with a leading slash or leading - backslash (<span class="bold"><strong>\</strong></span>); or any - pathname beginning with a single alphabetic character, - a colon and a leading slash or backslash (for example, - <code class="filename">C:/tmp</code>). - </p> + <p> + If the filename specified to a Berkeley DB + function is an <span class="emphasis"><em>absolute + pathname</em></span>, that filename is used + without modification by Berkeley DB. </p> + <p> + On UNIX systems, an absolute pathname is + defined as any pathname that begins with a + leading slash (<span class="bold"><strong>/</strong></span>). + </p> + <p> + On Windows systems, an absolute pathname is + any pathname that begins with a leading slash + or leading backslash (<span class="bold"><strong>\</strong></span>); + or any pathname beginning + with a single alphabetic character, a colon + and a leading slash or backslash (for example, + <code class="filename">C:/tmp</code>). + </p> </dd> <dt> <span class="term"><a href="../api_reference/C/env.html" class="olink">DB_ENV</a> methods, DB_CONFIG</span> </dt> - <dd> - If a relevant configuration string (for example, - set_data_dir), is specified either by calling a <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> - method or as a line in the - <a class="link" href="env_db_config.html" title="DB_CONFIG configuration file">DB_CONFIG</a> configuration - file, the value is prepended to the filename. If the - resulting filename is an absolute pathname, the filename is - used without further modification by Berkeley - DB. - </dd> + <dd> + If a relevant configuration string (for + example, add_data_dir), is specified either by + calling a <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> method or as a line in the <a class="link" href="env_db_config.html" title="DB_CONFIG configuration file">DB_CONFIG</a> + configuration file, the value is prepended to the + filename. If the resulting filename is an absolute + pathname, the filename is used without further + modification by Berkeley DB. + </dd> <dt> <span class="term">db_home</span> </dt> - <dd> - If the application specified a non-NULL - <span class="bold"><strong>db_home</strong></span> argument to - <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a>, its value is prepended to the filename. If the - resulting filename is an absolute pathname, the filename is - used without further modification by Berkeley - DB. - </dd> + <dd> + If the application specified a non-NULL + <span class="bold"><strong>db_home</strong></span> + argument to <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a>, its value is prepended to + the filename. If the resulting filename is an + absolute pathname, the filename is used without + further modification by Berkeley DB. + </dd> <dt> <span class="term">DB_HOME</span> </dt> <dd> - If the <span class="bold"><strong>db_home</strong></span> argument is - NULL, the DB_HOME environment variable was set, and the - application has set the appropriate <a href="../api_reference/C/envopen.html#envopen_DB_USE_ENVIRON" class="olink">DB_USE_ENVIRON</a> or - <a href="../api_reference/C/envopen.html#envopen_DB_USE_ENVIRON_ROOT" class="olink">DB_USE_ENVIRON_ROOT</a> flags, its value is prepended to the - filename. If the resulting filename is an absolute - pathname, the filename is used without further modification - by Berkeley DB. - </dd> + If the <span class="bold"><strong>db_home</strong></span> argument is NULL, the + DB_HOME environment variable was set, and the + application has set the appropriate + <a href="../api_reference/C/envopen.html#envopen_DB_USE_ENVIRON" class="olink">DB_USE_ENVIRON</a> or <a href="../api_reference/C/envopen.html#envopen_DB_USE_ENVIRON_ROOT" class="olink">DB_USE_ENVIRON_ROOT</a> flags, + its value is prepended to the filename. If the + resulting filename is an absolute pathname, the + filename is used without further modification by + Berkeley DB. + </dd> <dt> <span class="term">default</span> </dt> <dd> - Finally, all filenames are interpreted relative to the - current working directory of the process. - </dd> + Finally, all filenames are interpreted + relative to the current working directory of the + process. + </dd> </dl> </div> <p> - The common model for a Berkeley DB environment is one in which only - the DB_HOME environment variable, or the - <span class="bold"><strong>db_home</strong></span> argument is specified. In - this case, all data filenames are relative to that directory, and - all files created by the Berkeley DB subsystems will be created in - that directory. - </p> + The common model for a Berkeley DB environment is one + in which only the DB_HOME environment variable, or the + <span class="bold"><strong>db_home</strong></span> argument is + specified. In this case, all data filenames are relative + to that directory, and all files created by the Berkeley + DB subsystems will be created in that directory. + </p> <p> - The more complex model for a transaction environment might be one - in which a database home is specified, using either the DB_HOME - environment variable or the - <span class="bold"><strong>db_home</strong></span> argument to <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a>; and - then the data directory and logging directory are set to the - relative pathnames of directories underneath the environment - home. - </p> + The more complex model for a transaction environment + might be one in which a database home is specified, using + either the DB_HOME environment variable or the <span class="bold"><strong>db_home</strong></span> argument to <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a>; + and then the data directory and logging directory are set + to the relative pathnames of directories underneath the + environment home. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1605872"></a>Examples</h3> + <h3 class="title"><a id="idp1182752"></a>Examples</h3> </div> </div> </div> - <p> - Store all files in the directory <code class="filename">/a/database</code>: - </p> + <p> + Store all files in the directory + <code class="filename">/a/database</code>: + </p> <pre class="programlisting">dbenv->open(dbenv, "/a/database", flags, mode);</pre> - <p> - Create temporary backing files in - <code class="filename">/b/temporary</code>, and all other files in - <code class="filename">/a/database</code>: - </p> + <p> + Create temporary backing files in + <code class="filename">/b/temporary</code>, and all other files + in <code class="filename">/a/database</code>: + </p> <pre class="programlisting">dbenv->set_tmp_dir(dbenv, "/b/temporary"); dbenv->open(dbenv, "/a/database", flags, mode);</pre> - <p> - Store data files in <code class="filename">/a/database/datadir</code>, log - files in <code class="filename">/a/database/logdir</code>, and all other - files in the directory <code class="filename">/a/database</code>: - </p> + <p> + Store data files in + <code class="filename">/a/database/datadir</code>, log files in + <code class="filename">/a/database/logdir</code>, and all other + files in the directory <code class="filename">/a/database</code>: + </p> <pre class="programlisting">dbenv->set_lg_dir(dbenv, "logdir"); -dbenv->set_data_dir(dbenv, "datadir"); +dbenv->add_data_dir(dbenv, "datadir"); dbenv->open(dbenv, "/a/database", flags, mode);</pre> - <p> - Store data files in <code class="filename">/a/database/data1</code> and - <code class="filename">/b/data2</code>, and all other files in the directory - <code class="filename">/a/database</code>. Any data files that are created - will be created in <code class="filename">/b/data2</code>, because it is the - first data file directory specified: - </p> - <pre class="programlisting">dbenv->set_data_dir(dbenv, "/b/data2"); -dbenv->set_data_dir(dbenv, "data1"); + <p> + Store data files in + <code class="filename">/a/database/data1</code> and + <code class="filename">/b/data2</code>, and all other files in + the directory <code class="filename">/a/database</code>. Any data + files that are created will be created in + <code class="filename">/b/data2</code>, because it is the first + data file directory specified: + </p> + <pre class="programlisting">dbenv->add_data_dir(dbenv, "/b/data2"); +dbenv->add_data_dir(dbenv, "data1"); dbenv->open(dbenv, "/a/database", flags, mode);</pre> </div> </div> @@ -334,7 +347,8 @@ dbenv->open(dbenv, "/a/database", flags, mode);</pre> <td width="40%" align="right"> <a accesskey="n" href="env_region.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">DB_CONFIG configuration file </td> + <td width="40%" align="left" valign="top">DB_CONFIG configuration + file </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/env_open.html b/docs/programmer_reference/env_open.html index d4b3f19f..8b36f064 100644 --- a/docs/programmer_reference/env_open.html +++ b/docs/programmer_reference/env_open.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Opening databases within the environment</th> + <th colspan="3" align="center">Opening databases within the + environment</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="env_size.html">Prev</a> </td> - <th width="60%" align="center">Chapter 9. - The Berkeley DB Environment - </th> + <th width="60%" align="center">Chapter 9. The Berkeley DB Environment </th> <td width="20%" align="right"> <a accesskey="n" href="env_error.html">Next</a></td> </tr> </table> @@ -34,21 +33,29 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="env_open"></a>Opening databases within the environment</h2> + <h2 class="title" style="clear: both"><a id="env_open"></a>Opening databases within the + environment</h2> </div> </div> </div> - <p>Once the environment has been created, database handles may be created -and then opened within the environment. This is done by calling the -<a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a> function and specifying the appropriate environment -as an argument.</p> - <p>File naming, database operations, and error handling will all be done as -specified for the environment. For example, if the <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOCK" class="olink">DB_INIT_LOCK</a> -or <a href="../api_reference/C/envopen.html#envopen_DB_INIT_CDB" class="olink">DB_INIT_CDB</a> flags were specified when the environment was -created or joined, database operations will automatically perform all -necessary locking operations for the application.</p> - <p>The following is a simple example of opening two databases within a -database environment:</p> + <p> + Once the environment has been created, database handles may + be created and then opened within the environment. This is + done by calling the <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a> function and specifying the + appropriate environment as an argument. + </p> + <p> + File naming, database operations, and error handling will + all be done as specified for the environment. For example, if + the <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOCK" class="olink">DB_INIT_LOCK</a> or <a href="../api_reference/C/envopen.html#envopen_DB_INIT_CDB" class="olink">DB_INIT_CDB</a> flags were specified when + the environment was created or joined, database operations + will automatically perform all necessary locking operations + for the application. + </p> + <p> + The following is a simple example of opening two databases + within a database environment: + </p> <a id="prog_env28"></a> <pre class="programlisting">DB_ENV *dbenv; DB *dbp1, *dbp2; @@ -56,7 +63,7 @@ database environment:</p> dbenv = NULL; dbp1 = dbp2 = NULL; - /* + /* * Create an environment and initialize it for additional error * reporting. */ diff --git a/docs/programmer_reference/env_region.html b/docs/programmer_reference/env_region.html index 24658d3a..43c820f5 100644 --- a/docs/programmer_reference/env_region.html +++ b/docs/programmer_reference/env_region.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="env_naming.html">Prev</a> </td> - <th width="60%" align="center">Chapter 9. - The Berkeley DB Environment - </th> + <th width="60%" align="center">Chapter 9. The Berkeley DB Environment </th> <td width="20%" align="right"> <a accesskey="n" href="env_security.html">Next</a></td> </tr> </table> @@ -38,92 +36,104 @@ </div> </div> </div> - <p> - Each of the Berkeley DB subsystems within an environment is described by - one or more regions, or chunks of memory. The regions contain all of - the per-process and per-thread shared information (including mutexes), - that comprise a Berkeley DB environment. These regions are created in - one of three types of memory, depending on the flags specified to the - <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method: + <p> + Each of the Berkeley DB subsystems within an environment is + described by one or more regions, or chunks of memory. The + regions contain all of the per-process and per-thread shared + information (including mutexes), that comprise a Berkeley DB + environment. These regions are created in one of three types + of memory, depending on the flags specified to the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> + method: </p> <div class="orderedlist"> <ol type="1"> <li> - <p> - If the <a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a> flag is specified to the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method, - regions are created in per-process heap memory; that is, memory - returned by <code class="literal">malloc</code>(3). + <p> + If the <a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a> flag is specified to the + <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method, regions are created in per-process + heap memory; that is, memory returned by + <code class="literal">malloc</code>(3). </p> <p> - If this flag is specified, then you cannot open more than a - single handle for the environment. For example, if - both a server application and Berkeley DB utilities (for - example, the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility, the <a href="../api_reference/C/db_checkpoint.html" class="olink">db_checkpoint</a> utility or the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility) - are expected to access the environment, the <a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a> flag - should not be specified because the second attempt to open + If this flag is specified, then you cannot open + more than a single handle for the environment. For + example, if both a server application and Berkeley DB + utilities (for example, the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility, the + <a href="../api_reference/C/db_checkpoint.html" class="olink">db_checkpoint</a> utility or the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility) are expected to + access the environment, the <a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a> flag should + not be specified because the second attempt to open the environment will fail. </p> </li> <li> - <p> - If the <a href="../api_reference/C/envopen.html#envopen_DB_SYSTEM_MEM" class="olink">DB_SYSTEM_MEM</a> flag is specified to <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a>, shared - regions are created in system memory rather than files. This is - an alternative mechanism for sharing the Berkeley DB environment - among multiple processes and multiple threads within processes. + <p> + If the <a href="../api_reference/C/envopen.html#envopen_DB_SYSTEM_MEM" class="olink">DB_SYSTEM_MEM</a> flag is specified to + <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a>, shared regions are created in system memory + rather than files. This is an alternative mechanism + for sharing the Berkeley DB environment among multiple + processes and multiple threads within processes. </p> - <p> - The system memory used by Berkeley DB is potentially useful past - the lifetime of any particular process. Therefore, additional - cleanup may be necessary after an application fails because - there may be no way for Berkeley DB to ensure that system - resources backing the shared memory regions are returned to the - system. + <p> + The system memory used by Berkeley DB is + potentially useful past the lifetime of any particular + process. Therefore, additional cleanup may be + necessary after an application fails because there may + be no way for Berkeley DB to ensure that system + resources backing the shared memory regions are + returned to the system. </p> - <p> - The system memory that is used is architecture-dependent. For - example, on systems supporting X/Open-style shared memory - interfaces, such as UNIX systems, the - <code class="literal">shmget</code>(2) and related System V IPC interfaces - are used. Additionally, VxWorks systems use system memory. In - these cases, an initial segment ID must be specified by the - application to ensure that applications do not overwrite each - other's database environments, so that the number of segments - created does not grow without bounds. See the <a href="../api_reference/C/envset_shm_key.html" class="olink">DB_ENV->set_shm_key()</a> - method for more information. + <p> + The system memory that is used is + architecture-dependent. For example, on systems + supporting X/Open-style shared memory interfaces, such + as UNIX systems, the <code class="literal">shmget</code>(2) and + related System V IPC interfaces are used. + Additionally, VxWorks systems use system memory. In + these cases, an initial segment ID must be specified + by the application to ensure that applications do not + overwrite each other's database environments, so that + the number of segments created does not grow without + bounds. See the <a href="../api_reference/C/envset_shm_key.html" class="olink">DB_ENV->set_shm_key()</a> method for more + information. </p> - <p> - On Windows platforms, the use of the <a href="../api_reference/C/envopen.html#envopen_DB_SYSTEM_MEM" class="olink">DB_SYSTEM_MEM</a> flag is - problematic because the operating system uses reference counting - to clean up shared objects in the paging file automatically. In - addition, the default access permissions for shared objects are - different from files, which may cause problems when an - environment is accessed by multiple processes running as - different users. See the <a href="../installation/build_win_notes.html" class="olink">Windows Notes</a> section in the - Berkeley DB Installation and Build Guide for more information. + <p> + On Windows platforms, the use of the + <a href="../api_reference/C/envopen.html#envopen_DB_SYSTEM_MEM" class="olink">DB_SYSTEM_MEM</a> flag is problematic because the + operating system uses reference counting to clean up + shared objects in the paging file automatically. In + addition, the default access permissions for shared + objects are different from files, which may cause + problems when an environment is accessed by multiple + processes running as different users. See the + <a href="../installation/build_win_notes.html" class="olink">Windows Notes</a> section in the Berkeley DB Installation and Build Guide for + more information. </p> </li> <li> - If no memory-related flags are specified to <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a>, memory backed - by the filesystem is used to store the regions. On UNIX systems, - the Berkeley DB library will use the POSIX mmap interface. If mmap - is not available, the UNIX shmget interfaces may be used instead, if - they are available. + If no memory-related flags are specified to + <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a>, memory backed by the filesystem is used to + store the regions. On UNIX systems, the Berkeley DB + library will use the POSIX mmap interface. If mmap is not + available, the UNIX shmget interfaces may be used instead, + if they are available. </li> </ol> </div> <p> - Any files created in the filesystem to back the regions are created in - the environment home directory specified to the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> call. These - files are named __db.### (for example, __db.001, __db.002 and so on). - When region files are backed by the filesystem, one file per region is - created. When region files are backed by system memory, a single file - will still be created because there must be a well-known name in the - filesystem so that multiple processes can locate the system shared - memory that is being used by the environment.</p> - <p>Statistics - about the shared memory regions in the environment can be displayed - using the <span class="bold"><strong>-e</strong></span> option to the - <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility. + Any files created in the filesystem to back the regions are + created in the environment home directory specified to the + <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> call. These files are named __db.### (for example, + __db.001, __db.002 and so on). When region files are backed by + the filesystem, one file per region is created. When region + files are backed by system memory, a single file will still be + created because there must be a well-known name in the + filesystem so that multiple processes can locate the system + shared memory that is being used by the environment. + </p> + <p> + Statistics about the shared memory regions in the + environment can be displayed using the <span class="bold"><strong> + -e</strong></span> option to the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility. </p> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/env_remote.html b/docs/programmer_reference/env_remote.html index 197c3f7a..fd257dff 100644 --- a/docs/programmer_reference/env_remote.html +++ b/docs/programmer_reference/env_remote.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="env_encrypt.html">Prev</a> </td> - <th width="60%" align="center">Chapter 9. - The Berkeley DB Environment - </th> + <th width="60%" align="center">Chapter 9. The Berkeley DB Environment </th> <td width="20%" align="right"> <a accesskey="n" href="env_faq.html">Next</a></td> </tr> </table> @@ -38,54 +36,74 @@ </div> </div> </div> - <p>When Berkeley DB database environment shared memory regions are backed by the -filesystem, it is a common application error to create database -environments backed by remote filesystems such as the Network File -System (NFS), Windows network shares (SMB/CIFS) or the Andrew File -System (AFS). Remote filesystems rarely support mapping files into -process memory, and even more rarely support correct semantics for -mutexes if the mapping succeeds. For this reason, we recommend database -environment directories be created in a local filesystem.</p> - <p>For remote filesystems that do allow remote files to be mapped into -process memory, database environment directories accessed via remote -filesystems cannot be used simultaneously from multiple clients (that -is, from multiple computers). No commercial remote filesystem of which -we're aware supports coherent, distributed shared memory for -remote-mounted files. As a result, different machines will see -different versions of these shared region files, and the behavior is -undefined.</p> - <p>Databases, log files, and temporary files may be placed on remote -filesystems, as long as the remote filesystem fully supports standard -POSIX filesystem semantics (although the application may incur a -performance penalty for doing so). Further, read-only databases on -remote filesystems can be accessed from multiple systems simultaneously. -However, it is difficult (or impossible) for modifiable databases on -remote filesystems to be accessed from multiple systems simultaneously. -The reason is the Berkeley DB library caches modified database pages, and when -those modified pages are written to the backing file is not entirely -under application control. If two systems were to write database pages -to the remote filesystem at the same time, database corruption could -result. If a system were to write a database page back to the remote -filesystem at the same time as another system read a page, a core dump -in the reader could result.</p> + <p> + When Berkeley DB database environment shared memory regions + are backed by the filesystem, it is a common application error + to create database environments backed by remote filesystems + such as the Network File System (NFS), Windows network shares + (SMB/CIFS) or the Andrew File System (AFS). Remote filesystems + rarely support mapping files into process memory, and even + more rarely support correct semantics for mutexes if the + mapping succeeds. For this reason, we recommend database + environment directories be created in a local + filesystem. + </p> + <p> + For remote filesystems that do allow remote files to be + mapped into process memory, database environment directories + accessed via remote filesystems cannot be used simultaneously + from multiple clients (that is, from multiple computers). No + commercial remote filesystem of which we're aware supports + coherent, distributed shared memory for remote-mounted files. + As a result, different machines will see different versions of + these shared region files, and the behavior is + undefined. + </p> + <p> + Databases, log files, and temporary files may be placed on + remote filesystems, as long as the remote filesystem fully + supports standard POSIX filesystem semantics (although the + application may incur a performance penalty for doing so). + Further, read-only databases on remote filesystems can be + accessed from multiple systems simultaneously. However, it is + difficult (or impossible) for modifiable databases on remote + filesystems to be accessed from multiple systems + simultaneously. The reason is the Berkeley DB library caches + modified database pages, and when those modified pages are + written to the backing file is not entirely under application + control. If two systems were to write database pages to the + remote filesystem at the same time, database corruption could + result. If a system were to write a database page back to the + remote filesystem at the same time as another system read a + page, a core dump in the reader could result. + </p> <div class="variablelist"> <dl> <dt> <span class="term">FreeBSD note:</span> </dt> - <dd>Some historic FreeBSD releases will return ENOLCK from fsync and close -calls on NFS-mounted filesystems, even though the call has succeeded. -To support Berkeley DB on these releases, the Berkeley DB code should be modified -to ignore ENOLCK errors, or no Berkeley DB files should be placed on -NFS-mounted filesystems on these systems. Note that current FreeBSD -releases do not suffer from this problem.</dd> + <dd> + Some historic FreeBSD releases will return + ENOLCK from fsync and close calls on NFS-mounted + filesystems, even though the call has succeeded. To + support Berkeley DB on these releases, the Berkeley DB + code should be modified to ignore ENOLCK errors, or no + Berkeley DB files should be placed on NFS-mounted + filesystems on these systems. Note that current + FreeBSD releases do not suffer from this + problem. + </dd> <dt> <span class="term">Linux note:</span> </dt> - <dd>Some historic Linux releases do not support complete semantics for the -POSIX fsync call on NFS-mounted filesystems. No Berkeley DB files should be -placed on NFS-mounted filesystems on these systems. Note that current -Linux releases do not suffer from this problem.</dd> + <dd> + Some historic Linux releases do not support + complete semantics for the POSIX fsync call on + NFS-mounted filesystems. No Berkeley DB files should + be placed on NFS-mounted filesystems on these systems. + Note that current Linux releases do not suffer from + this problem. + </dd> </dl> </div> </div> diff --git a/docs/programmer_reference/env_security.html b/docs/programmer_reference/env_security.html index 4e18322f..fac00c76 100644 --- a/docs/programmer_reference/env_security.html +++ b/docs/programmer_reference/env_security.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="env_region.html">Prev</a> </td> - <th width="60%" align="center">Chapter 9. - The Berkeley DB Environment - </th> + <th width="60%" align="center">Chapter 9. The Berkeley DB Environment </th> <td width="20%" align="right"> <a accesskey="n" href="env_encrypt.html">Next</a></td> </tr> </table> @@ -38,62 +36,86 @@ </div> </div> </div> - <p>The following are security issues that should be considered when writing -Berkeley DB applications:</p> + <p> + The following are security issues that should be considered + when writing Berkeley DB applications: + </p> <div class="variablelist"> <dl> <dt> <span class="term">Database environment permissions</span> </dt> - <dd>The directory used as the Berkeley DB database environment should have its -permissions set to ensure that files in the environment are not accessible -to users without appropriate permissions. Applications that add to the -user's permissions (for example, UNIX setuid or setgid applications), -must be carefully checked to not permit illegal use of those permissions -such as general file access in the environment directory.</dd> + <dd> + The directory used as the Berkeley DB database + environment should have its permissions set to ensure + that files in the environment are not accessible to + users without appropriate permissions. Applications + that add to the user's permissions (for example, UNIX + setuid or setgid applications), must be carefully + checked to not permit illegal use of those permissions + such as general file access in the environment + directory. + </dd> <dt> <span class="term">Environment variables</span> </dt> - <dd>Setting the <a href="../api_reference/C/envopen.html#envopen_DB_USE_ENVIRON" class="olink">DB_USE_ENVIRON</a> -and <a href="../api_reference/C/envopen.html#envopen_DB_USE_ENVIRON_ROOT" class="olink">DB_USE_ENVIRON_ROOT</a> flags -and allowing the use of environment variables during file naming can be -dangerous. Setting those flags in Berkeley DB applications with additional -permissions (for example, UNIX setuid or setgid applications) could -potentially allow users to read and write databases to which they would -not normally have access.</dd> + <dd> + Setting the <a href="../api_reference/C/envopen.html#envopen_DB_USE_ENVIRON" class="olink">DB_USE_ENVIRON</a> and + <a href="../api_reference/C/envopen.html#envopen_DB_USE_ENVIRON_ROOT" class="olink">DB_USE_ENVIRON_ROOT</a> flags and allowing the use of + environment variables during file naming can be + dangerous. Setting those flags in Berkeley DB + applications with additional permissions (for example, + UNIX setuid or setgid applications) could potentially + allow users to read and write databases to which they + would not normally have access. + </dd> <dt> <span class="term">File permissions</span> </dt> - <dd>By default, Berkeley DB always creates files readable and writable by the owner -and the group (that is, S_IRUSR, S_IWUSR, S_IRGRP and S_IWGRP; or octal mode -0660 on historic UNIX systems). The group ownership of created files is -based on the system and directory defaults, and is not further specified -by Berkeley DB.</dd> + <dd> + By default, Berkeley DB always creates files + readable and writable by the owner and the group (that + is, S_IRUSR, S_IWUSR, S_IRGRP and S_IWGRP; or octal + mode 0660 on historic UNIX systems). The group + ownership of created files is based on the system and + directory defaults, and is not further specified by + Berkeley DB. + </dd> <dt> <span class="term">Temporary backing files</span> </dt> - <dd>If an unnamed database is created and the cache is too small to hold -the database in memory, Berkeley DB will create a temporary physical file to -enable it to page the database to disk as needed. In this case, -environment variables such as <span class="bold"><strong>TMPDIR</strong></span> may be used to specify -the location of that temporary file. Although temporary backing files -are created readable and writable by the owner only (S_IRUSR and -S_IWUSR, or octal mode 0600 on historic UNIX systems), some filesystems -may not sufficiently protect temporary files created in random -directories from improper access. To be absolutely safe, applications -storing sensitive data in unnamed databases should use the -<a href="../api_reference/C/envset_tmp_dir.html" class="olink">DB_ENV->set_tmp_dir()</a> method to specify a temporary directory with -known permissions.</dd> + <dd> + If an unnamed database is created and the cache + is too small to hold the database in memory, Berkeley + DB will create a temporary physical file to enable it + to page the database to disk as needed. In this case, + environment variables such as <span class="bold"><strong>TMPDIR</strong></span> + may be used to specify the + location of that temporary file. Although temporary + backing files are created readable and writable by the + owner only (S_IRUSR and S_IWUSR, or octal mode 0600 on + historic UNIX systems), some filesystems may not + sufficiently protect temporary files created in random + directories from improper access. To be absolutely + safe, applications storing sensitive data in unnamed + databases should use the <a href="../api_reference/C/envset_tmp_dir.html" class="olink">DB_ENV->set_tmp_dir()</a> method to + specify a temporary directory with known + permissions. + </dd> <dt> <span class="term">Tcl API</span> </dt> - <dd>The Berkeley DB Tcl API does not attempt to avoid evaluating input as Tcl -commands. For this reason, it may be dangerous to pass unreviewed user -input through the Berkeley DB Tcl API, as the input may subsequently be -evaluated as a Tcl command. Additionally, the Berkeley DB Tcl API -initialization routine resets process' effective user and group IDs to -the real user and group IDs, to minimize the effectiveness of a Tcl -injection attack.</dd> + <dd> + The Berkeley DB Tcl API does not attempt to + avoid evaluating input as Tcl commands. For this + reason, it may be dangerous to pass unreviewed user + input through the Berkeley DB Tcl API, as the input + may subsequently be evaluated as a Tcl command. + Additionally, the Berkeley DB Tcl API initialization + routine resets process' effective user and group IDs + to the real user and group IDs, to minimize the + effectiveness of a Tcl injection attack. + </dd> </dl> </div> </div> diff --git a/docs/programmer_reference/env_size.html b/docs/programmer_reference/env_size.html index 78f1398d..de849860 100644 --- a/docs/programmer_reference/env_size.html +++ b/docs/programmer_reference/env_size.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="env_create.html">Prev</a> </td> - <th width="60%" align="center">Chapter 9. - The Berkeley DB Environment - </th> + <th width="60%" align="center">Chapter 9. The Berkeley DB Environment </th> <td width="20%" align="right"> <a accesskey="n" href="env_open.html">Next</a></td> </tr> </table> @@ -40,15 +38,16 @@ </div> <p> The Berkeley DB environment allocates memory to hold shared - structures, either in shared regions or in process data space (if the - <a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a> flag is specified). There are three distinct memory - regions: + structures, either in shared regions or in process data space + (if the <a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a> flag is specified). There are three + distinct memory regions: </p> <div class="itemizedlist"> <ul type="disc"> <li> <p> - The memory pool (also known as the database page cache), + The memory pool (also known as the database page + cache), </p> </li> <li> @@ -57,32 +56,32 @@ </p> </li> <li> - <p> - the main region which holds all other shared structures. + <p> + the main region which holds all other shared + structures. </p> </li> </ul> </div> <p> - The shared structures in the main region are used by the lock, - transaction, logging, thread and replicatoin subsystems. + The shared structures in the main region are used by the + lock, transaction, logging, thread and replicatoin subsystems. </p> <p> - Determining the amount of space allocated for each of these shared - structures is dependent upon the structure in question. The sizing - of the memory pool is discussed in - <a class="xref" href="mp_config.html" title="Configuring the memory pool">Configuring the memory pool</a>. The amount - of memory needed for mutexes is calculated from the number of - mutexes needed by various subsystems and can be adjusted using the - <a href="../api_reference/C/mutexset_increment.html" class="olink">DB_ENV->mutex_set_increment()</a> method. + Determining the amount of space allocated for each of these + shared structures is dependent upon the structure in question. + The sizing of the memory pool is discussed in <a class="xref" href="mp_config.html" title="Configuring the memory pool">Configuring the memory pool</a>. The + amount of memory needed for mutexes is calculated from the + number of mutexes needed by various subsystems and can be + adjusted using the <a href="../api_reference/C/mutexset_increment.html" class="olink">DB_ENV->mutex_set_increment()</a> method. </p> <p> - For applications using shared memory (that is, they do not specify - <a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a>), a maximum memory size for the main region must be - specified or left to default. The maximum memory size is specified - using the <a href="../api_reference/C/envset_memory_max.html" class="olink">DB_ENV->set_memory_max()</a> method. + For applications using shared memory (that is, they do not + specify <a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a>), a maximum memory size for the main + region must be specified or left to default. The maximum + memory size is specified using the <a href="../api_reference/C/envset_memory_max.html" class="olink">DB_ENV->set_memory_max()</a> method. </p> - <p> + <p> The amount of memory needed by an application is dependent on the resources that the application uses. For a very rough estimate, add all of the following together: @@ -90,37 +89,40 @@ <div class="orderedlist"> <ol type="1"> <li> - <p> - The environment has an overhead of about 80 kilobytes - without statistics enabled or 250 kilobytes with statistics - enabled. + <p> + The environment has an overhead of about 80 + kilobytes without statistics enabled or 250 kilobytes + with statistics enabled. </p> </li> <li> <p> - Identify the amount of space you require for your locks: + Identify the amount of space you require for your + locks: </p> <div class="orderedlist"> <ol type="a"> <li> - <p> - Estimate the number of threads of control that will - simultaneously access the environment. + <p> + Estimate the number of threads of control + that will simultaneously access the + environment. </p> </li> <li> - <p> - Estimate the number of concurrency locks that, on average, - will be required by each thread. For information on - sizing concurrency locks, see - <a class="xref" href="lock_max.html" title="Configuring locking: sizing the system">Configuring locking: sizing the system</a>. + <p> + Estimate the number of concurrency locks + that, on average, will be required by each + thread. For information on sizing concurrency + locks, see <a class="xref" href="lock_max.html" title="Configuring locking: sizing the system">Configuring locking: sizing the + system</a>. </p> </li> <li> - <p> - Multiply these two numbers, then multiply by 1/2 to - arrive at the number of kilobytes required to - service your locks. + <p> + Multiply these two numbers, then multiply + by 1/2 to arrive at the number of kilobytes + required to service your locks. </p> </li> </ol> @@ -128,44 +130,45 @@ </li> <li> <p> - Estimate the number of open database handles you will use - at any given time. For each database handle, there is an - overhead of about 1/2 kilobyte. + Estimate the number of open database handles you + will use at any given time. For each database handle, + there is an overhead of about 1/2 kilobyte. </p> </li> <li> <p> - Add 1 kilobyte for each active transaction. + Add 1 kilobyte for each active transaction. </p> </li> </ol> </div> <p> Note that these are very rough guidelines. It is best to - overestimate the needs of your applications, because if the memory - allocation is exhausted the application must be shutdown to - increase the allocation. + overestimate the needs of your applications, because if the + memory allocation is exhausted the application must be + shutdown to increase the allocation. </p> <p> The estimate for maximum memory need not be exact. In most - situations there is little penalty for over estimating. For systems - using memory mapped files for the shared environment, this only - allocates the address space in the process to hold the maximum - memory. The backing file will only be extended as needed. For - systems running with <a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a> specified, the maximum memory - serves only as a limit and memory is allocated from the process - data space as needed. No maximum need be set for private - environments. + situations there is little penalty for over estimating. For + systems using memory mapped files for the shared environment, + this only allocates the address space in the process to hold + the maximum memory. The backing file will only be extended as + needed. For systems running with <a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a> specified, the + maximum memory serves only as a limit and memory is allocated + from the process data space as needed. No maximum need be set + for private environments. </p> - <p> - For locking and thread information, groups of objects are allocated - when needed so that there is less contention in the allocator - during performance critical operations. Once allocated to a - particular use, this memory will only be used for that structure. - To avoid runtime contention, or to ensure a minimum number of a - particular type of object, the <a href="../api_reference/C/envset_memory_init.html" class="olink">DB_ENV->set_memory_init()</a> method can be - used. This method can set the initial numbers of particular types - of structures to allocate at environment creation time. + <p> + For locking and thread information, groups of objects are + allocated when needed so that there is less contention in the + allocator during performance critical operations. Once + allocated to a particular use, this memory will only be used + for that structure. To avoid runtime contention, or to ensure + a minimum number of a particular type of object, the + <a href="../api_reference/C/envset_memory_init.html" class="olink">DB_ENV->set_memory_init()</a> method can be used. This method can set + the initial numbers of particular types of structures to + allocate at environment creation time. </p> </div> <div class="navfooter"> @@ -179,11 +182,13 @@ <td width="40%" align="right"> <a accesskey="n" href="env_open.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Creating a database environment </td> + <td width="40%" align="left" valign="top">Creating a database + environment </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Opening databases within the environment</td> + <td width="40%" align="right" valign="top"> Opening databases within the + environment</td> </tr> </table> </div> diff --git a/docs/programmer_reference/ext.html b/docs/programmer_reference/ext.html index 31d38bd9..c73bea92 100644 --- a/docs/programmer_reference/ext.html +++ b/docs/programmer_reference/ext.html @@ -14,13 +14,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 22. - Berkeley DB Extensions - </th> + <th colspan="3" align="center">Chapter 22. Berkeley DB Extensions </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="tcl_faq.html">Prev</a> </td> @@ -34,9 +32,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="ext"></a>Chapter 22. - Berkeley DB Extensions - </h2> + <h2 class="title"><a id="ext"></a>Chapter 22. Berkeley DB Extensions </h2> </div> </div> </div> @@ -70,64 +66,110 @@ </div> </div> </div> - <p>A mod_db4 Apache module is included in the Berkeley DB distribution, providing -a safe framework for running Berkeley DB applications in an Apache 1.3 -environment. Apache natively provides no interface for communication -between threads or processes, so the mod_db4 module exists to provide -this communication.</p> - <p>In general, it is dangerous to run Berkeley DB in a multiprocess system -without some facility to coordinate database recovery between processes -sharing the database environment after application or system failure. -Failure to run recovery after failure can include process hangs and an -inability to access the database environment. The mod_db4 Apache module -oversees the proper management of Berkeley DB database environment resources. -Developers building applications using Berkeley DB as the storage manager -within an Apache module should employ this technique for proper resource -management.</p> - <p>Specifically, mod_db4 provides the following facilities:</p> + <p> + A mod_db4 Apache module is included in the Berkeley DB + distribution, providing a safe framework for running Berkeley + DB applications in an Apache 1.3 environment. Apache natively + provides no interface for communication between threads or + processes, so the mod_db4 module exists to provide this + communication. + </p> + <p> + In general, it is dangerous to run Berkeley DB in a + multiprocess system without some facility to coordinate + database recovery between processes sharing the database + environment after application or system failure. Failure to + run recovery after failure can include process hangs and an + inability to access the database environment. The mod_db4 + Apache module oversees the proper management of Berkeley DB + database environment resources. Developers building + applications using Berkeley DB as the storage manager within + an Apache module should employ this technique for proper + resource management. + </p> + <p> + Specifically, mod_db4 provides the following + facilities: + </p> <div class="orderedlist"> <ol type="1"> - <li>New constructors for <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> and <a href="../api_reference/C/db.html" class="olink">DB</a> handles, which install -replacement open/close methods.</li> - <li>Transparent caching of open <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> and <a href="../api_reference/C/db.html" class="olink">DB</a> handles.</li> - <li>Reference counting on all structures, allowing the module to detect the -initial opening of any managed database and automatically perform recovery.</li> - <li>Automatic detection of unexpected failures (segfaults, or a module -actually calling exit() and avoiding shut down phases), and automatic -termination of all child processes with open database resources to -attempt consistency.</li> + <li> + New constructors for <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> and <a href="../api_reference/C/db.html" class="olink">DB</a> handles, which + install replacement open/close methods. + </li> + <li> + Transparent caching of open <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> and <a href="../api_reference/C/db.html" class="olink">DB</a> + handles. + </li> + <li> + Reference counting on all structures, allowing the + module to detect the initial opening of any managed + database and automatically perform recovery. + </li> + <li> + Automatic detection of unexpected failures + (segfaults, or a module actually calling exit() and + avoiding shut down phases), and automatic termination of + all child processes with open database resources to + attempt consistency. + </li> </ol> </div> - <p>mod_db4 is designed to be used as an alternative interface to Berkeley DB. To -have another Apache module (for example, mod_foo) use mod_db4, do not -link mod_foo against the Berkeley DB library. In your mod_foo makefile, you -should:</p> + <p> + mod_db4 is designed to be used as an alternative interface + to Berkeley DB. To have another Apache module (for example, + mod_foo) use mod_db4, do not link mod_foo against the Berkeley + DB library. In your mod_foo makefile, you should: + </p> <pre class="programlisting">#include "mod_db4_export.h"</pre> - <p>and add your Apache include directory to your CPPFLAGS.</p> - <p>In mod_foo, to create a mod_db4 managed <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handle, use the -following:</p> + <p> + and add your Apache include directory to your + CPPFLAGS. + </p> + <p> + In mod_foo, to create a mod_db4 managed <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handle, use + the following: + </p> <pre class="programlisting">int mod_db4_db_env_create(DB_ENV **dbenvp, u_int32_t flags);</pre> - <p>which takes identical arguments to <a href="../api_reference/C/envcreate.html" class="olink">db_env_create()</a>.</p> - <p>To create a mod_db4 managed <a href="../api_reference/C/db.html" class="olink">DB</a> handle, use the following:</p> + <p> + which takes identical arguments to <a href="../api_reference/C/envcreate.html" class="olink">db_env_create()</a>. + </p> + <p> + To create a mod_db4 managed <a href="../api_reference/C/db.html" class="olink">DB</a> handle, use the + following: + </p> <pre class="programlisting">int mod_db4_db_create(DB **dbp, DB_ENV *dbenv, u_int32_t flags);</pre> - <p>which takes identical arguments to <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a>.</p> - <p>Otherwise the API is completely consistent with the standard Berkeley DB -API.</p> - <p>The mod_db4 module requires the Berkeley DB library be compiled with C++ -extensions and the MM library. (The MM library provides an abstraction -layer which allows related processes to share data easily. On systems -where shared memory or other inter-process communication mechanisms are -not available, the MM library emulates them using temporary files. MM -is used in several operating systems to provide shared memory pools to -Apache modules.)</p> - <p>To build this apache module, perform the following steps:</p> + <p> + which takes identical arguments to <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a>. + </p> + <p> + Otherwise the API is completely consistent with the standard + Berkeley DB API. + </p> + <p> + The mod_db4 module requires the Berkeley DB library be + compiled with C++ extensions and the MM library. (The MM + library provides an abstraction layer which allows related + processes to share data easily. On systems where shared memory + or other inter-process communication mechanisms are not + available, the MM library emulates them using temporary files. + MM is used in several operating systems to provide shared + memory pools to Apache modules.) + </p> + <p> + To build this apache module, perform the following + steps: + </p> <pre class="programlisting">% ./configure --with-apxs=[path to the apxs utility] \ - --with-db4=[Berkeley DB library installation directory] \ - --with-mm=[libmm installation directory] + --with-db4=[Berkeley DB library installation directory] \ + --with-mm=[libmm installation directory] % make % make install</pre> - <p>Post-installation, modules can use this extension via the functions -documented in $APACHE_INCLUDEDIR/mod_db4_export.h.</p> + <p> + Post-installation, modules can use this extension via the + functions documented in + $APACHE_INCLUDEDIR/mod_db4_export.h. + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/ext_perl.html b/docs/programmer_reference/ext_perl.html index af72b9f7..263c231c 100644 --- a/docs/programmer_reference/ext_perl.html +++ b/docs/programmer_reference/ext_perl.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="ext.html">Prev</a> </td> - <th width="60%" align="center">Chapter 22. - Berkeley DB Extensions - </th> + <th width="60%" align="center">Chapter 22. Berkeley DB Extensions </th> <td width="20%" align="right"> <a accesskey="n" href="ext_php.html">Next</a></td> </tr> </table> @@ -38,26 +36,37 @@ </div> </div> </div> - <p>The original Perl module for Berkeley DB was DB_File, which was written to -interface to Berkeley DB version 1.85. The newer Perl module for Berkeley DB is -BerkeleyDB, which was written to interface to version 2.0 and subsequent -releases. Because Berkeley DB version 2.X has a compatibility API for version -1.85, you can (and should!) build DB_File using version 2.X of Berkeley DB, -although DB_File will still only support the 1.85 functionality.</p> - <p>DB_File is distributed with the standard Perl source distribution (look -in the directory "ext/DB_File"). You can find both DB_File and BerkeleyDB -on CPAN, the Comprehensive Perl Archive Network of mirrored FTP sites. -The master CPAN site is -<a class="ulink" href="ftp://ftp.funet.fi/" target="_top">ftp://ftp.funet.fi/</a>.</p> - <p>Versions of both BerkeleyDB and DB_File that are known to work correctly -with each release of Berkeley DB are included in the distributed Berkeley DB source -tree, in the subdirectories <code class="filename">perl.BerkeleyDB</code> and -<code class="filename">perl.DB_File</code>. Each of those directories contains a -<code class="filename">README</code> file with instructions on installing and using those -modules.</p> - <p>The Perl interface is not maintained by Oracle. Questions about the -DB_File and BerkeleyDB modules are best asked on the Usenet newsgroup -comp.lang.perl.modules.</p> + <p> + The original Perl module for Berkeley DB was DB_File, which + was written to interface to Berkeley DB version 1.85. The + newer Perl module for Berkeley DB is BerkeleyDB, which was + written to interface to version 2.0 and subsequent releases. + Because Berkeley DB version 2.X has a compatibility API for + version 1.85, you can (and should!) build DB_File using + version 2.X of Berkeley DB, although DB_File will still only + support the 1.85 functionality. + </p> + <p> + DB_File is distributed with the standard Perl source + distribution (look in the directory "ext/DB_File"). You can + find both DB_File and BerkeleyDB on CPAN, the Comprehensive + Perl Archive Network of mirrored FTP sites. The master CPAN + site is <a class="ulink" href="ftp://ftp.funet.fi/" target="_top">ftp://ftp.funet.fi/</a>. + </p> + <p> + Versions of both BerkeleyDB and DB_File that are known to + work correctly with each release of Berkeley DB are included + in the distributed Berkeley DB source tree, in the + subdirectories <code class="filename">perl.BerkeleyDB</code> and + <code class="filename">perl.DB_File</code>. Each of those + directories contains a <code class="filename">README</code> file with + instructions on installing and using those modules. + </p> + <p> + The Perl interface is not maintained by Oracle. Questions + about the DB_File and BerkeleyDB modules are best asked on the + Usenet newsgroup comp.lang.perl.modules. + </p> </div> <div class="navfooter"> <hr /> @@ -70,9 +79,7 @@ comp.lang.perl.modules.</p> <td width="40%" align="right"> <a accesskey="n" href="ext_php.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 22. - Berkeley DB Extensions - </td> + <td width="40%" align="left" valign="top">Chapter 22. Berkeley DB Extensions </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/ext_php.html b/docs/programmer_reference/ext_php.html index 11263863..a88dcb5e 100644 --- a/docs/programmer_reference/ext_php.html +++ b/docs/programmer_reference/ext_php.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="ext_perl.html">Prev</a> </td> - <th width="60%" align="center">Chapter 22. - Berkeley DB Extensions - </th> + <th width="60%" align="center">Chapter 22. Berkeley DB Extensions </th> <td width="20%" align="right"> <a accesskey="n" href="dumpload.html">Next</a></td> </tr> </table> @@ -38,18 +36,24 @@ </div> </div> </div> - <p>A PHP 4 extension for this release of Berkeley DB is included in the -distribution package. It can either either link directly against the -installed Berkeley DB library (which is necessary for running in a -non-Apache/mod_php4 environment), or against mod_db4, which provides -additional safety when running under Apache/mod_php4.</p> <p> - For installation instructions, see the <code class="filename">INSTALL</code> file that - resides in the <code class="literal">lang/php_db4</code> directory in your - Berkeley DB distribution. -</p> - <p>The PHP extension provides the following classes, which mirror the -standard Berkeley DB C++ API.</p> + A PHP 4 extension for this release of Berkeley DB is + included in the distribution package. It can either either + link directly against the installed Berkeley DB library (which + is necessary for running in a non-Apache/mod_php4 + environment), or against mod_db4, which provides additional + safety when running under Apache/mod_php4. + </p> + <p> + For installation instructions, see the + <code class="filename">INSTALL</code> file that resides in the + <code class="literal">lang/php_db4</code> directory in your + Berkeley DB distribution. + </p> + <p> + The PHP extension provides the following classes, which + mirror the standard Berkeley DB C++ API. + </p> <pre class="programlisting">class Db4Env { function Db4Env($flags = 0) {} function close($flags = 0) {} @@ -60,7 +64,7 @@ standard Berkeley DB C++ API.</p> DB_INIT_LOG | DB_INIT_MPOOL | DB_INIT_TXN, $mode = 0666) {} function remove($home, $flags = 0) {} - function set_data_dir($directory) {} + function add_data_dir($directory) {} function txn_begin($parent_txn = null, $flags = 0) {} function txn_checkpoint($kbytes, $minutes, $flags = 0) {} } @@ -101,14 +105,24 @@ class Db4Cursor { function pget($key, &$primary_key, $flags = 0) {} function put($key, $data, $flags = 0) {} }</pre> - <p>The PHP extension attempts to be "smart" for you by:</p> + <p> + The PHP extension attempts to be "smart" for you by: + </p> <div class="orderedlist"> <ol type="1"> - <li>Auto-committing operations on transactional databases if no explicit -Db4Txn object is specified.</li> - <li>Performing reference and dependency checking to insure that all -resources are closed in the correct order.</li> - <li>Supplying default values for flags.</li> + <li> + Auto-committing operations on transactional + databases if no explicit Db4Txn object is + specified. + </li> + <li> + Performing reference and dependency checking to + insure that all resources are closed in the correct + order. + </li> + <li> + Supplying default values for flags. + </li> </ol> </div> </div> @@ -127,9 +141,8 @@ resources are closed in the correct order.</li> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 23. - Dumping and Reloading Databases - </td> + <td width="40%" align="right" valign="top"> Chapter 23. Dumping and Reloading Databases + </td> </tr> </table> </div> diff --git a/docs/programmer_reference/general_am_conf.html b/docs/programmer_reference/general_am_conf.html index 729e1439..73f3909f 100644 --- a/docs/programmer_reference/general_am_conf.html +++ b/docs/programmer_reference/general_am_conf.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_conf_logrec.html">Prev</a> </td> - <th width="60%" align="center">Chapter 2. - Access Method Configuration - </th> + <th width="60%" align="center">Chapter 2. Access Method Configuration </th> <td width="20%" align="right"> <a accesskey="n" href="bt_conf.html">Next</a></td> </tr> </table> @@ -62,15 +60,17 @@ </dt> <dt> <span class="sect2"> - <a href="general_am_conf.html#am_conf_malloc">Non-local memory allocation</a> + <a href="general_am_conf.html#am_conf_malloc">Non-local memory + allocation</a> </span> </dt> </dl> </div> - <p> - There are a series of configuration tasks which are common to all - access methods. They are described in the following sections. -</p> + <p> + There are a series of configuration tasks which are common + to all access methods. They are described in the following + sections. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> @@ -80,83 +80,95 @@ </div> </div> <p> - The size of the pages used in the underlying database can be specified - by calling the <a href="../api_reference/C/dbset_pagesize.html" class="olink">DB->set_pagesize()</a> method. The minimum page size is 512 - bytes and the maximum page size is 64K bytes, and must be a power of - two. If no page size is specified by the application, a page size is - selected based on the underlying filesystem I/O block size. (A page - size selected in this way has a lower limit of 512 bytes and an upper - limit of 16K bytes.) -</p> + The size of the pages used in the underlying database can + be specified by calling the <a href="../api_reference/C/dbset_pagesize.html" class="olink">DB->set_pagesize()</a> method. The + minimum page size is 512 bytes and the maximum page size is + 64K bytes, and must be a power of two. If no page size is + specified by the application, a page size is selected based on + the underlying filesystem I/O block size. (A page size + selected in this way has a lower limit of 512 bytes and an + upper limit of 16K bytes.) + </p> + <p> + There are several issues to consider when selecting a + pagesize: overflow record sizes, locking, I/O efficiency, and + recoverability. + </p> <p> - There are several issues to consider when selecting a pagesize: overflow - record sizes, locking, I/O efficiency, and recoverability. -</p> + First, the page size implicitly sets the size of an + overflow record. Overflow records are key or data items that + are too large to fit on a normal database page because of + their size, and are therefore stored in overflow pages. + Overflow pages are pages that exist outside of the normal + database structure. For this reason, there is often a + significant performance penalty associated with retrieving or + modifying overflow records. Selecting a page size that is too + small, and which forces the creation of large numbers of + overflow pages, can seriously impact the performance of an + application. + </p> + <p> + Second, in the Btree, Hash and Recno access methods, the + finest-grained lock that Berkeley DB acquires is for a page. + (The Queue access method generally acquires record-level locks + rather than page-level locks.) Selecting a page size that is + too large, and which causes threads or processes to wait + because other threads of control are accessing or modifying + records on the same page, can impact the performance of your + application. + </p> <p> - First, the page size implicitly sets the size of an overflow record. - Overflow records are key or data items that are too large to fit on a - normal database page because of their size, and are therefore stored in - overflow pages. Overflow pages are pages that exist outside of the - normal database structure. For this reason, there is often a - significant performance penalty associated with retrieving or modifying - overflow records. Selecting a page size that is too small, and which - forces the creation of large numbers of overflow pages, can seriously - impact the performance of an application. -</p> + Third, the page size specifies the granularity of I/O from + the database to the operating system. Berkeley DB will give a + page-sized unit of bytes to the operating system to be + scheduled for reading/writing from/to the disk. For many + operating systems, there is an internal <span class="bold"><strong> + block size</strong></span> which is used as the granularity of + I/O from the operating system to the disk. Generally, it will + be more efficient for Berkeley DB to write filesystem-sized + blocks to the operating system and for the operating system to + write those same blocks to the disk. + </p> + <p> + Selecting a database page size smaller than the filesystem + block size may cause the operating system to coalesce or + otherwise manipulate Berkeley DB pages and can impact the + performance of your application. When the page size is smaller + than the filesystem block size and a page written by Berkeley + DB is not found in the operating system's cache, the operating + system may be forced to read a block from the disk, copy the + page into the block it read, and then write out the block to + disk, rather than simply writing the page to disk. + Additionally, as the operating system is reading more data + into its buffer cache than is strictly necessary to satisfy + each Berkeley DB request for a page, the operating system + buffer cache may be wasting memory. + </p> <p> - Second, in the Btree, Hash and Recno access methods, the finest-grained - lock that Berkeley DB acquires is for a page. (The Queue access method - generally acquires record-level locks rather than page-level locks.) - Selecting a page size that is too large, and which causes threads or - processes to wait because other threads of control are accessing or - modifying records on the same page, can impact the performance of your - application. -</p> - <p> - Third, the page size specifies the granularity of I/O from the database - to the operating system. Berkeley DB will give a page-sized unit of - bytes to the operating system to be scheduled for reading/writing - from/to the disk. For many operating systems, there is an internal - <span class="bold"><strong>block size</strong></span> which is used as the - granularity of I/O from the operating system to the disk. Generally, - it will be more efficient for Berkeley DB to write filesystem-sized - blocks to the operating system and for the operating system to write - those same blocks to the disk. -</p> - <p> - Selecting a database page size smaller than the filesystem block size - may cause the operating system to coalesce or otherwise manipulate - Berkeley DB pages and can impact the performance of your application. - When the page size is smaller than the filesystem block size and a page - written by Berkeley DB is not found in the operating system's cache, - the operating system may be forced to read a block from the disk, copy - the page into the block it read, and then write out the block to disk, - rather than simply writing the page to disk. Additionally, as the - operating system is reading more data into its buffer cache than is - strictly necessary to satisfy each Berkeley DB request for a page, the - operating system buffer cache may be wasting memory. -</p> - <p> - Alternatively, selecting a page size larger than the filesystem block - size may cause the operating system to read more data than necessary. - On some systems, reading filesystem blocks sequentially may cause the - operating system to begin performing read-ahead. If requesting a - single database page implies reading enough filesystem blocks to - satisfy the operating system's criteria for read-ahead, the operating - system may do more I/O than is required. -</p> - <p> - Fourth, when using the Berkeley DB Transactional Data Store product, - the page size may affect the errors from which your database can - recover See <a class="xref" href="transapp_reclimit.html" title="Berkeley DB recoverability">Berkeley DB recoverability</a> for more information. -</p> + Alternatively, selecting a page size larger than the + filesystem block size may cause the operating system to read + more data than necessary. On some systems, reading filesystem + blocks sequentially may cause the operating system to begin + performing read-ahead. If requesting a single database page + implies reading enough filesystem blocks to satisfy the + operating system's criteria for read-ahead, the operating + system may do more I/O than is required. + </p> + <p> + Fourth, when using the Berkeley DB Transactional Data Store + product, the page size may affect the errors from which your + database can recover See <a class="xref" href="transapp_reclimit.html" title="Berkeley DB recoverability">Berkeley DB recoverability</a> for more information. + </p> <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> <h3 class="title">Note</h3> - <p> -The <a href="../api_reference/C/db_tuner.html" class="olink">db_tuner</a> utility suggests a page size for btree databases that optimizes cache -efficiency and storage space requirements. This utility works only when given a pre-populated database. -So, it is useful when tuning an existing application and not when first implementing an application. -</p> + <p> + The <a href="../api_reference/C/db_tuner.html" class="olink">db_tuner</a> utility suggests a page size for btree databases + that optimizes cache efficiency and storage space + requirements. This utility works only when given a + pre-populated database. So, it is useful when tuning an + existing application and not when first implementing an + application. + </p> </div> </div> <div class="sect2" lang="en" xml:lang="en"> @@ -167,51 +179,74 @@ So, it is useful when tuning an existing application and not when first implemen </div> </div> </div> - <p>The size of the cache used for the underlying database can be specified -by calling the <a href="../api_reference/C/dbset_cachesize.html" class="olink">DB->set_cachesize()</a> method. -Choosing a cache size is, unfortunately, an art. Your cache must be at -least large enough for your working set plus some overlap for unexpected -situations.</p> - <p>When using the Btree access method, you must have a cache big enough for -the minimum working set for a single access. This will include a root -page, one or more internal pages (depending on the depth of your tree), -and a leaf page. If your cache is any smaller than that, each new page -will force out the least-recently-used page, and Berkeley DB will re-read the -root page of the tree anew on each database request.</p> - <p>If your keys are of moderate size (a few tens of bytes) and your pages -are on the order of 4KB to 8KB, most Btree applications will be only -three levels. For example, using 20 byte keys with 20 bytes of data -associated with each key, a 8KB page can hold roughly 400 keys (or 200 -key/data pairs), so a fully populated three-level Btree will hold 32 -million key/data pairs, and a tree with only a 50% page-fill factor will -still hold 16 million key/data pairs. We rarely expect trees to exceed -five levels, although Berkeley DB will support trees up to 255 levels.</p> - <p>The rule-of-thumb is that cache is good, and more cache is better. -Generally, applications benefit from increasing the cache size up to a -point, at which the performance will stop improving as the cache size -increases. When this point is reached, one of two things have happened: -either the cache is large enough that the application is almost never -having to retrieve information from disk, or, your application is doing -truly random accesses, and therefore increasing size of the cache doesn't -significantly increase the odds of finding the next requested information -in the cache. The latter is fairly rare -- almost all applications show -some form of locality of reference.</p> - <p>That said, it is important not to increase your cache size beyond the -capabilities of your system, as that will result in reduced performance. -Under many operating systems, tying down enough virtual memory will cause -your memory and potentially your program to be swapped. This is -especially likely on systems without unified OS buffer caches and virtual -memory spaces, as the buffer cache was allocated at boot time and so -cannot be adjusted based on application requests for large amounts of -virtual memory.</p> - <p>For example, even if accesses are truly random within a Btree, your -access pattern will favor internal pages to leaf pages, so your cache -should be large enough to hold all internal pages. In the steady state, -this requires at most one I/O per operation to retrieve the appropriate -leaf page.</p> - <p>You can use the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility to monitor the effectiveness of -your cache. The following output is excerpted from the output of that -utility's <span class="bold"><strong>-m</strong></span> option:</p> + <p> + The size of the cache used for the underlying database can + be specified by calling the <a href="../api_reference/C/dbset_cachesize.html" class="olink">DB->set_cachesize()</a> method. Choosing + a cache size is, unfortunately, an art. Your cache must be at + least large enough for your working set plus some overlap for + unexpected situations. + </p> + <p> + When using the Btree access method, you must have a cache + big enough for the minimum working set for a single access. + This will include a root page, one or more internal pages + (depending on the depth of your tree), and a leaf page. If + your cache is any smaller than that, each new page will force + out the least-recently-used page, and Berkeley DB will re-read + the root page of the tree anew on each database + request. + </p> + <p> + If your keys are of moderate size (a few tens of bytes) and + your pages are on the order of 4KB to 8KB, most Btree + applications will be only three levels. For example, using 20 + byte keys with 20 bytes of data associated with each key, a + 8KB page can hold roughly 400 keys (or 200 key/data pairs), so + a fully populated three-level Btree will hold 32 million + key/data pairs, and a tree with only a 50% page-fill factor + will still hold 16 million key/data pairs. We rarely expect + trees to exceed five levels, although Berkeley DB will support + trees up to 255 levels. + </p> + <p> + The rule-of-thumb is that cache is good, and more cache is + better. Generally, applications benefit from increasing the + cache size up to a point, at which the performance will stop + improving as the cache size increases. When this point is + reached, one of two things have happened: either the cache is + large enough that the application is almost never having to + retrieve information from disk, or, your application is doing + truly random accesses, and therefore increasing size of the + cache doesn't significantly increase the odds of finding the + next requested information in the cache. The latter is fairly + rare -- almost all applications show some form of locality of + reference. + </p> + <p> + That said, it is important not to increase your cache size + beyond the capabilities of your system, as that will result in + reduced performance. Under many operating systems, tying down + enough virtual memory will cause your memory and potentially + your program to be swapped. This is especially likely on + systems without unified OS buffer caches and virtual memory + spaces, as the buffer cache was allocated at boot time and so + cannot be adjusted based on application requests for large + amounts of virtual memory. + </p> + <p> + For example, even if accesses are truly random within a + Btree, your access pattern will favor internal pages to leaf + pages, so your cache should be large enough to hold all + internal pages. In the steady state, this requires at most one + I/O per operation to retrieve the appropriate leaf + page. + </p> + <p> + You can use the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility to monitor the effectiveness of + your cache. The following output is excerpted from the output + of that utility's <span class="bold"><strong>-m</strong></span> + option: + </p> <pre class="programlisting">prompt: db_stat -m 131072 Cache size (128K). 4273 Requested pages found in the cache (97%). @@ -223,13 +258,15 @@ utility's <span class="bold"><strong>-m</strong></span> option:</p> 13 Dirty pages forced from the cache. 0 Dirty buffers written by trickle-sync thread. 130 Current clean buffer count. -4 Current dirty buffer count. -</pre> - <p>The statistics for this cache say that there have been 4,273 requests of -the cache, and only 116 of those requests required an I/O from disk. This -means that the cache is working well, yielding a 97% cache hit rate. The -<a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility will present these statistics both for the cache -as a whole and for each file within the cache separately.</p> +4 Current dirty buffer count.</pre> + <p> + The statistics for this cache say that there have been 4,273 + requests of the cache, and only 116 of those requests required + an I/O from disk. This means that the cache is working well, + yielding a 97% cache hit rate. The <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility will present + these statistics both for the cache as a whole and for each + file within the cache separately. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> @@ -239,25 +276,30 @@ as a whole and for each file within the cache separately.</p> </div> </div> </div> - <p>Database files created by Berkeley DB can be created in either little- or -big-endian formats. The byte order used for the underlying database -is specified by calling the <a href="../api_reference/C/dbset_lorder.html" class="olink">DB->set_lorder()</a> method. If no order -is selected, the native format of the machine on which the database is -created will be used.</p> - <p>Berkeley DB databases are architecture independent, and any format database can -be used on a machine with a different native format. In this case, as -each page that is read into or written from the cache must be converted -to or from the host format, and databases with non-native formats will -incur a performance penalty for the run-time conversion.</p> <p> - <span class="bold"> - <strong>It is important to note that the Berkeley DB access methods do no data -conversion for application specified data. Key/data pairs written on a -little-endian format architecture will be returned to the application -exactly as they were written when retrieved on a big-endian format -architecture.</strong> - </span> - </p> + Database files created by Berkeley DB can be created in + either little- or big-endian formats. The byte order used for + the underlying database is specified by calling the + <a href="../api_reference/C/dbset_lorder.html" class="olink">DB->set_lorder()</a> method. If no order is selected, the native + format of the machine on which the database is created will be + used. + </p> + <p> + Berkeley DB databases are architecture independent, and any + format database can be used on a machine with a different + native format. In this case, as each page that is read into or + written from the cache must be converted to or from the host + format, and databases with non-native formats will incur a + performance penalty for the run-time conversion. + </p> + <p> + <span class="bold"><strong>It is important to note that the + Berkeley DB access methods do no data conversion for + application specified data. Key/data pairs written on a + little-endian format architecture will be returned to the + application exactly as they were written when retrieved on + a big-endian format architecture.</strong></span> + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> @@ -268,105 +310,123 @@ architecture.</strong> </div> </div> <p> - The Btree and Hash access methods support the creation of multiple data - items for a single key item. By default, multiple data items are not - permitted, and each database store operation will overwrite any - previous data item for that key. To configure Berkeley DB for - duplicate data items, call the <a href="../api_reference/C/dbset_flags.html" class="olink">DB->set_flags()</a> method with the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_DUP" class="olink">DB_DUP</a> - flag. Only one copy of the key will be stored for each set of - duplicate data items. If the Btree access method comparison routine - returns that two keys compare equally, it is undefined which of the two - keys will be stored and returned from future database operations. -</p> + The Btree and Hash access methods support the creation of + multiple data items for a single key item. By default, + multiple data items are not permitted, and each database store + operation will overwrite any previous data item for that key. + To configure Berkeley DB for duplicate data items, call the + <a href="../api_reference/C/dbset_flags.html" class="olink">DB->set_flags()</a> method with the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_DUP" class="olink">DB_DUP</a> flag. Only one copy of + the key will be stored for each set of duplicate data items. + If the Btree access method comparison routine returns that two + keys compare equally, it is undefined which of the two keys + will be stored and returned from future database operations. + </p> <p> - By default, Berkeley DB stores duplicates in the order in which they - were added, that is, each new duplicate data item will be stored after - any already existing data items. This default behavior can be - overridden by using the <a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a> method and one of the <a href="../api_reference/C/dbcput.html#dbcput_DB_AFTER" class="olink">DB_AFTER</a>, - <a href="../api_reference/C/dbcput.html#dbcput_DB_BEFORE" class="olink">DB_BEFORE</a>, <a href="../api_reference/C/dbcput.html#dbcput_DB_KEYFIRST" class="olink">DB_KEYFIRST</a> or <a href="../api_reference/C/dbcput.html#dbcput_DB_KEYLAST" class="olink">DB_KEYLAST</a> flags. Alternatively, - Berkeley DB may be configured to sort duplicate data items. -</p> + By default, Berkeley DB stores duplicates in the order in + which they were added, that is, each new duplicate data item + will be stored after any already existing data items. This + default behavior can be overridden by using the <a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a> + method and one of the <a href="../api_reference/C/dbcput.html#dbcput_DB_AFTER" class="olink">DB_AFTER</a>, <a href="../api_reference/C/dbcput.html#dbcput_DB_BEFORE" class="olink">DB_BEFORE</a>, <a href="../api_reference/C/dbcput.html#dbcput_DB_KEYFIRST" class="olink">DB_KEYFIRST</a> + or <a href="../api_reference/C/dbcput.html#dbcput_DB_KEYLAST" class="olink">DB_KEYLAST</a> flags. Alternatively, Berkeley DB may be + configured to sort duplicate data items. + </p> <p> - When stepping through the database sequentially, duplicate data items - will be returned individually, as a key/data pair, where the key item - only changes after the last duplicate data item has been returned. For - this reason, duplicate data items cannot be accessed using the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> - method, as it always returns the first of the duplicate data items. - Duplicate data items should be retrieved using a Berkeley DB cursor - interface such as the <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method. -</p> + When stepping through the database sequentially, duplicate + data items will be returned individually, as a key/data pair, + where the key item only changes after the last duplicate data + item has been returned. For this reason, duplicate data items + cannot be accessed using the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> method, as it always + returns the first of the duplicate data items. Duplicate data + items should be retrieved using a Berkeley DB cursor interface + such as the <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method. + </p> + <p> + There is a flag that permits applications to request the + following data item only if it <span class="bold"><strong>is</strong></span> + a duplicate data item of the current entry, + see <a href="../api_reference/C/dbcget.html#dbcget_DB_NEXT_DUP" class="olink">DB_NEXT_DUP</a> for more information. There is a flag that + permits applications to request the following data item only + if it <span class="bold"><strong>is not</strong></span> a duplicate data + item of the current entry, see <a href="../api_reference/C/dbcget.html#dbcget_DB_NEXT_NODUP" class="olink">DB_NEXT_NODUP</a> and + <a href="../api_reference/C/dbcget.html#dbcget_DB_PREV_NODUP" class="olink">DB_PREV_NODUP</a> for more information. + </p> + <p> + It is also possible to maintain duplicate records in sorted + order. Sorting duplicates will significantly increase + performance when searching them and performing equality joins + — both of which are common operations when using + secondary indices. To configure Berkeley DB to sort duplicate + data items, the application must call the <a href="../api_reference/C/dbset_flags.html" class="olink">DB->set_flags()</a> method + with the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_DUPSORT" class="olink">DB_DUPSORT</a> flag. Note that <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_DUPSORT" class="olink">DB_DUPSORT</a> + automatically turns on the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_DUP" class="olink">DB_DUP</a> flag for you, so you do + not have to also set that flag; however, it is not an error to + also set <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_DUP" class="olink">DB_DUP</a> when configuring for sorted duplicate + records. + </p> <p> - There is a flag that permits applications to request the following data - item only if it <span class="bold"><strong>is</strong></span> a duplicate data - item of the current entry, see <a href="../api_reference/C/dbcget.html#dbcget_DB_NEXT_DUP" class="olink">DB_NEXT_DUP</a> for more information. - There is a flag that permits applications to request the following data - item only if it <span class="bold"><strong>is not</strong></span> a duplicate - data item of the current entry, see <a href="../api_reference/C/dbcget.html#dbcget_DB_NEXT_NODUP" class="olink">DB_NEXT_NODUP</a> and <a href="../api_reference/C/dbcget.html#dbcget_DB_PREV_NODUP" class="olink">DB_PREV_NODUP</a> - for more information. -</p> + When configuring sorted duplicate records, you can also + specify a custom comparison function using the + <a href="../api_reference/C/dbset_dup_compare.html" class="olink">DB->set_dup_compare()</a> method. If the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_DUPSORT" class="olink">DB_DUPSORT</a> flag is given, + but no comparison routine is specified, then Berkeley DB + defaults to the same lexicographical sorting used for Btree + keys, with shorter items collating before longer items. + </p> <p> - It is also possible to maintain duplicate records in sorted order. - Sorting duplicates will significantly increase performance when - searching them and performing equality joins — both of which are - common operations when using secondary indices. To configure Berkeley - DB to sort duplicate data items, the application must call the - <a href="../api_reference/C/dbset_flags.html" class="olink">DB->set_flags()</a> method with the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_DUPSORT" class="olink">DB_DUPSORT</a> flag. Note that <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_DUPSORT" class="olink">DB_DUPSORT</a> - automatically turns on the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_DUP" class="olink">DB_DUP</a> flag for you, so you do not - have to also set that flag; however, it is not an error to also set <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_DUP" class="olink">DB_DUP</a> - when configuring for sorted duplicate records. -</p> + If the duplicate data items are unsorted, applications may + store identical duplicate data items, or, for those that just + like the way it sounds, <span class="emphasis"><em>duplicate + duplicates</em></span>. + </p> <p> - When configuring sorted duplicate records, you can also specify a - custom comparison function using the <a href="../api_reference/C/dbset_dup_compare.html" class="olink">DB->set_dup_compare()</a> method. If - the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_DUPSORT" class="olink">DB_DUPSORT</a> flag is given, but no comparison routine is specified, - then Berkeley DB defaults to the same lexicographical sorting used for - Btree keys, with shorter items collating before longer items. -</p> + <span class="bold"><strong>It is an error to attempt to store + identical duplicate data items when duplicates are being + stored in a sorted order.</strong></span> Any such attempt + results in the error message "Duplicate data items are not + supported with sorted data" with a + <code class="literal">DB_KEYEXIST</code> return code. + </p> <p> - If the duplicate data items are unsorted, applications may store - identical duplicate data items, or, for those that just like the way it - sounds, <span class="emphasis"><em>duplicate duplicates</em></span>. -</p> - <p> - <span class="bold"><strong>It is an error to attempt to store identical - duplicate data items when duplicates are being stored in a sorted - order.</strong></span> Any such attempt results in the - error message "Duplicate data items are not supported with sorted - data" with a <code class="literal">DB_KEYEXIST</code> return code. -</p> - <p> - Note that you can suppress the error message "Duplicate data items are - not supported with sorted data" by using the <a href="../api_reference/C/dbput.html#put_DB_NODUPDATA" class="olink">DB_NODUPDATA</a> flag. Use - of this flag does not change the database's basic behavior; storing - duplicate data items in a database configured for sorted duplicates is - still an error and so you will continue to receive the - <code class="literal">DB_KEYEXIST</code> return code if you try to do that. -</p> - <p> - For further information on how searching and insertion behaves in the - presence of duplicates (sorted or not), see the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a>, - <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> and <a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a> documentation. -</p> + Note that you can suppress the error message "Duplicate + data items are not supported with sorted data" by using the + <a href="../api_reference/C/dbput.html#put_DB_NODUPDATA" class="olink">DB_NODUPDATA</a> flag. Use of this flag does not change the + database's basic behavior; storing duplicate data items in a + database configured for sorted duplicates is still an error + and so you will continue to receive the + <code class="literal">DB_KEYEXIST</code> return code if you try to + do that. + </p> + <p> + For further information on how searching and insertion + behaves in the presence of duplicates (sorted or not), see the + <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a>, <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> and <a href="../api_reference/C/dbcput.html" class="olink">DBC->put()</a> documentation. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="am_conf_malloc"></a>Non-local memory allocation</h3> + <h3 class="title"><a id="am_conf_malloc"></a>Non-local memory + allocation</h3> </div> </div> </div> - <p>Berkeley DB allocates memory for returning key/data pairs and statistical -information which becomes the responsibility of the application. -There are also interfaces where an application will allocate memory -which becomes the responsibility of Berkeley DB.</p> - <p>On systems in which there may be multiple library versions of the -standard allocation routines (notably Windows NT), transferring memory -between the library and the application will fail because the Berkeley DB -library allocates memory from a different heap than the application -uses to free it, or vice versa. To avoid this problem, the -<a href="../api_reference/C/envset_alloc.html" class="olink">DB_ENV->set_alloc()</a> and <a href="../api_reference/C/dbset_alloc.html" class="olink">DB->set_alloc()</a> methods can be used to -give Berkeley DB references to the application's allocation routines.</p> + <p> + Berkeley DB allocates memory for returning key/data pairs + and statistical information which becomes the responsibility + of the application. There are also interfaces where an + application will allocate memory which becomes the + responsibility of Berkeley DB. + </p> + <p> + On systems in which there may be multiple library versions + of the standard allocation routines (notably Windows NT), + transferring memory between the library and the application + will fail because the Berkeley DB library allocates memory + from a different heap than the application uses to free it, or + vice versa. To avoid this problem, the <a href="../api_reference/C/envset_alloc.html" class="olink">DB_ENV->set_alloc()</a> and + <a href="../api_reference/C/dbset_alloc.html" class="olink">DB->set_alloc()</a> methods can be used to give Berkeley DB + references to the application's allocation routines. + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/group_membership.html b/docs/programmer_reference/group_membership.html index c77bbc1c..64b8ec73 100644 --- a/docs/programmer_reference/group_membership.html +++ b/docs/programmer_reference/group_membership.html @@ -3,29 +3,28 @@ <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> - <title>Managing Replication Manager Group Membership</title> + <title>Managing Replication Manager group membership</title> <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> <link rel="up" href="rep.html" title="Chapter 12. Berkeley DB Replication" /> <link rel="prev" href="rep_newsite.html" title="Connecting to a new site" /> - <link rel="next" href="rep_filename.html" title="Managing Replication Files" /> + <link rel="next" href="rep_partview.html" title="Replication views" /> </head> <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Managing Replication Manager Group Membership</th> + <th colspan="3" align="center">Managing Replication Manager + group membership</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_newsite.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> - <td width="20%" align="right"> <a accesskey="n" href="rep_filename.html">Next</a></td> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> + <td width="20%" align="right"> <a accesskey="n" href="rep_partview.html">Next</a></td> </tr> </table> <hr /> @@ -34,7 +33,8 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="group_membership"></a>Managing Replication Manager Group Membership</h2> + <h2 class="title" style="clear: both"><a id="group_membership"></a>Managing Replication Manager + group membership</h2> </div> </div> </div> @@ -42,220 +42,240 @@ <dl> <dt> <span class="sect2"> - <a href="group_membership.html#group_mem_add">Adding Sites to a Replication Group</a> + <a href="group_membership.html#group_mem_add">Adding sites to a replication + group</a> </span> </dt> <dt> <span class="sect2"> - <a href="group_membership.html#group_mem_remove">Removing Sites from a Replication Group</a> + <a href="group_membership.html#group_mem_remove">Removing sites from a + replication group</a> </span> </dt> <dt> <span class="sect2"> - <a href="group_membership.html#group_mem_primordialstartup">Primordial Startups</a> + <a href="group_membership.html#group_mem_primordialstartup">Primordial startups</a> </span> </dt> <dt> <span class="sect2"> - <a href="group_membership.html#group_mem_upgrade">Upgrading Groups</a> + <a href="group_membership.html#group_mem_upgrade">Upgrading groups</a> </span> </dt> </dl> </div> - <p> + <p> A replication group is a collection of two or more database - environments which are configured to replicate with one another. - When operating normally, a replication group consists of a master - site and one or more read-only sites. + environments which are configured to replicate with one + another. When operating normally, a replication group consists + of a master site and one or more read-only sites. </p> <p> - For Replication Manager applications, the sites comprising the - replication group are recorded in an internal database, so even if - a group member is not available, it counts towards the group's - total site count. This matters for certain replication activities, - such as holding elections and acknowledging replication messages - that require some number of sites to participate in these - activities. Replicated applications will often require all sites, - or a majority of sites, to participate before the activity can be - completed. + For Replication Manager applications, the sites comprising + the replication group are recorded in an internal group + membership database, so even if a group member is not + available, it counts towards the group's total site count. + This matters for certain replication activities, such as + holding elections and acknowledging replication messages that + require some number of sites to participate in these + activities. Replicated applications will often require all + sites, or a majority of sites, to participate before the + activity can be completed. </p> <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> <h3 class="title">Note</h3> - <p> - If you are configuring your application to keep replication - metadata in-memory by specifying the <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_INMEM" class="olink">DB_REP_CONF_INMEM</a> flag to the - <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method, then the internal database containing group - site information is not stored persistently on disk. This - severely limits Replication Manager's ability to automatically - manage group membership. For more information, including some - work-arounds, see - <a class="xref" href="rep_filename.html" title="Managing Replication Files">Managing Replication Files</a>. + <p> + If you are configuring your application to keep + replication metadata in-memory by specifying the + <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_INMEM" class="olink">DB_REP_CONF_INMEM</a> flag to the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method, then + the internal group membership database is not stored + persistently on disk. This severely limits Replication + Manager's ability to automatically manage group + membership. For more information, including some + work-arounds, see <a class="xref" href="rep_filename.html" title="Managing replication directories and files">Managing replication directories and files</a>. </p> </div> <p> - Because Replication Manager tracks group members, there are some - administrative activities that you should know about when using BDB - replication. + Because Replication Manager tracks group members, there are + some administrative activities that you should know about when + using Berkeley DB replication. </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="group_mem_add"></a>Adding Sites to a Replication Group</h3> + <h3 class="title"><a id="group_mem_add"></a>Adding sites to a replication + group</h3> </div> </div> </div> - <p> - To add a site to a replication group, you merely start up the - site such that it knows where at least one site in the group is - located. The site new site then joins the group. When this - happens, the new site is recorded in the Replication Manager's - group member database. + <p> + To add a site to a replication group, you merely start + up the site such that it knows where at least one site in + the group is located. The new site then joins the group. + When this happens, the new site is recorded in the group + membership database. </p> - <p> - Note that when you are starting the very first site in the - group for the very first time (called <span class="emphasis"><em>the primordial - start up</em></span>), there are no other existing sites to - help the new site join the group. In fact, a primordial start up - actually creates the group. For this reason, there are some - slight differences on how to perform a primordial start up. For - a description of this, see - <a class="xref" href="group_membership.html#group_mem_primordialstartup" title="Primordial Startups">Primordial Startups</a>. + <p> + Note that when you are starting the very first site in + the group for the very first time (called <span class="emphasis"><em>the + primordial start up</em></span>), there are no other + existing sites to help the new site join the group. In + fact, a primordial start up actually creates the group. + For this reason, there are some slight differences on how + to perform a primordial start up. For a description of + this, see <a class="xref" href="group_membership.html#group_mem_primordialstartup" title="Primordial startups">Primordial startups</a>. </p> - <p> + <p> When you add a site to a replication group, you use the - following general procedure: + following general procedure: </p> <div class="itemizedlist"> <ul type="disc"> <li> <p> - Make sure your replication group is operating well - enough that write activity can occur. + Make sure your replication group is operating + well enough that write activity can occur. </p> </li> <li> <p> - Create and open the environment such that it is configured to - use replication. + Create and open the environment such that it is + configured to use replication. </p> </li> <li> <p> Use <a href="../api_reference/C/repmgr_site.html" class="olink">DB_ENV->repmgr_site()</a> to obtain a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle. - Configure this handle for the local site's host and - port information when you create the handle. Then, use - <a href="../api_reference/C/dbsite_set_config.html" class="olink">DB_SITE->set_config()</a> to indicate that this is the local - site by setting the <code class="literal">DB_LOCAL_SITE</code> - parameter. + Configure this handle for the local site's host + and port information when you create the handle. + Then, use <a href="../api_reference/C/dbsite_set_config.html" class="olink">DB_SITE->set_config()</a> to indicate that + this is the local site by setting the + <code class="literal">DB_LOCAL_SITE</code> parameter. </p> </li> <li> - <p> - Use <a href="../api_reference/C/repmgr_site.html" class="olink">DB_ENV->repmgr_site()</a> to obtain a second <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle. - Configure this handle with the host and port - information for a site that already belongs to the - replication group. Then, use <a href="../api_reference/C/dbsite_set_config.html" class="olink">DB_SITE->set_config()</a> to - indicate this site is a "helper" site by setting the - <code class="literal">DB_BOOTSTRAP_HELPER</code> parameter. By - configuring a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle in this way, your new - site will know how to contact the replication group so - that it can join the group. + <p> + Use <a href="../api_reference/C/repmgr_site.html" class="olink">DB_ENV->repmgr_site()</a> to obtain a second <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> + handle. Configure this handle with the host and + port information for a site that already belongs + to the replication group. Then, use + <a href="../api_reference/C/dbsite_set_config.html" class="olink">DB_SITE->set_config()</a> to indicate this site is a + "helper" site by setting the + <code class="literal">DB_BOOTSTRAP_HELPER</code> + parameter. By configuring a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle in + this way, your new site will know how to contact + the replication group so that it can join the + group. </p> </li> <li> <p> Start replication as normal by configuring an - acknowledgement policy, setting the site's replication - priority, and then calling <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a>. + acknowledgement policy, setting the site's + replication priority, and then calling + <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a>. </p> </li> </ul> </div> <p> - Note that on subsequent start-ups of your replication code, any - helper site information you might provide is ignored because - the Replication Manager reads the group membership database - in order to obtain this information. + Note that on subsequent start-ups of your replication + code, any helper site information you might provide is + ignored because the Replication Manager reads the group + membership database in order to obtain this information. </p> <p> - Also, be aware that if the new site cannot be added to the - group for some reason (because a master site is not available, - or because insufficient replicas are running to acknowledge the - new site), the attempt to start the new site via <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a> - will fail and return <code class="literal">DB_REP_UNAVAIL</code>. You can - then pause and retry the start up attempt until it completes - successfully. + Also, be aware that if the new site cannot be added to + the group for some reason (because a master site is not + available, or because insufficient replicas are running to + acknowledge the new site), the attempt to start the new + site via <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a> will fail and return + <code class="literal">DB_REP_UNAVAIL</code>. You can then pause + and retry the start up attempt until it completes + successfully. </p> <p> - You must use the exact same host string and port number to refer - to a given site throughout your application and on each of its - sites. + You must use the exact same host string and port number + to refer to a given site throughout your application and + on each of its sites. </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="group_mem_remove"></a>Removing Sites from a Replication Group</h3> + <h3 class="title"><a id="group_mem_remove"></a>Removing sites from a + replication group</h3> </div> </div> </div> <p> - Elections and message acknowledgements require knowledge of the - total number of sites in the group. If a site is shut down, or - is otherwise unable to communicate with the rest of the group, - it still counts towards the total number of sites in the group. - In most cases, this is the desirable behavior. + Elections and message acknowledgements require + knowledge of the total number of sites in the group. If a + site is shut down, or is otherwise unable to communicate + with the rest of the group, it still counts towards the + total number of sites in the group. In most cases, this is + the desirable behavior. </p> <p> - However, if you are shutting down a site permanently, then you - should remove that site from the group. You might also want to - remove a site from the group if you are shutting it down - temporarily, but nevertheless for a very long period of time - (days or weeks). In either case, you remove a site from the - group by: + However, if you are shutting down a site permanently, + then you should remove that site from the group. You might + also want to remove a site from the group if you are + shutting it down temporarily, but nevertheless for a very + long period of time (days or weeks). In either case, you + remove a site from the group by: </p> <div class="itemizedlist"> <ul type="disc"> <li> - <p> - Make sure your replication group is operating well - enough that write activity can occur. + <p> + Make sure your replication group is operating + well enough that write activity can occur. </p> </li> <li> <p> - On one of the sites in your replication group (this - does not have to be the master site), use <a href="../api_reference/C/repmgr_site.html" class="olink">DB_ENV->repmgr_site()</a> - to obtain a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle. Configure this handle with - the host and port information of the site that you want - to remove. + On one of the sites in your replication group + (this does not have to be the master site), use + <a href="../api_reference/C/repmgr_site.html" class="olink">DB_ENV->repmgr_site()</a> to obtain a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle. + Configure this handle with the host and port + information of the site that you want to remove. </p> - <p> - Note that this step can occur at any site — - including the site that you are removing from the - group. + <p> + Note that this step can occur at any site + — including the site that you are removing + from the group. </p> </li> <li> + <p> + Call the <a href="../api_reference/C/dbsite_remove.html" class="olink">DB_SITE->remove()</a> method. This removes + the identified site from the group membership + database. If this action is not performed on the + master site, the client sends a request to the + master to perform the operation and awaits + confirmation. + </p> <p> - Call the <a href="../api_reference/C/dbsite_remove.html" class="olink">DB_SITE->remove()</a> method. This removes the - identified site from the replication group database. If - this action is not performed on the master site, the - client sends a request to the master to perform the - operation and awaits confirmation. + A client removing itself can close its database + environment any time after the DB_SITE->remove() + method returns. A site that has been removed by + another site can close its database environment + any time after the + <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_LOCAL_SITE_REMOVED" class="olink">DB_EVENT_REP_LOCAL_SITE_REMOVED</a> event is fired. </p> </li> </ul> </div> <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> <h3 class="title">Note</h3> - <p> - Upon completing the above procedure, DO NOT call the - <a href="../api_reference/C/dbsite_close.html" class="olink">DB_SITE->close()</a> method. After removing (or even attempting - to remove) a site from the group using a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle, - the handle must never be accessed again. + <p> + Upon completing the above procedure, DO NOT call + the <a href="../api_reference/C/dbsite_close.html" class="olink">DB_SITE->close()</a> method. After removing (or even + attempting to remove) a site from the group using a + <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle, the handle must never be accessed + again. </p> </div> </div> @@ -263,29 +283,31 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="group_mem_primordialstartup"></a>Primordial Startups</h3> + <h3 class="title"><a id="group_mem_primordialstartup"></a>Primordial startups</h3> </div> </div> </div> <p> - If you have never started a site in a replication group before, - then the replication group membership database does not exist. - In this situation, you must start the site and declare it to be - the group creator. This causes the site to become the master, - create the group membership database, and create a replication - group of size 1. After that, subsequent sites can add - themselves to the group as described in - <a class="xref" href="group_membership.html#group_mem_add" title="Adding Sites to a Replication Group">Adding Sites to a Replication Group</a>. + If you have never started a site in a replication group + before, then the group membership database does not exist. + In this situation, you must start an initial site and + declare it to be the group creator. This causes the site + to become the master, create the group membership + database, and create a replication group of size 1. After + that, subsequent sites can add themselves to the group as + described in <a class="xref" href="group_membership.html#group_mem_add" title="Adding sites to a replication group">Adding sites to a replication + group</a>. </p> <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> <h3 class="title">Note</h3> <p> - It is never incorrect to declare a site the group creator. - This is true even well-after the replication group has been - established. This is because group creator information is - ignored on any site start-up, except for the primoridial - start-up; that is, a start-up where the group membership - database does not exist. + It is never incorrect to declare a site the group + creator. This is true even well-after the replication + group has been established. This is because group + creator information is ignored on any site start-up, + except for the primoridial start-up; that is, a + start-up where the group membership database does not + exist. </p> </div> <p> @@ -294,23 +316,23 @@ <div class="itemizedlist"> <ul type="disc"> <li> - <p> - Create and open the environment such that it is configured to - use replication. + <p> + Create and open the environment such that it is + configured to use replication. </p> </li> <li> - <p> + <p> Use <a href="../api_reference/C/repmgr_site.html" class="olink">DB_ENV->repmgr_site()</a> to obtain a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle. - Configure this handle for the local site's host and - port information when you create the handle. Then, use - <a href="../api_reference/C/dbsite_set_config.html" class="olink">DB_SITE->set_config()</a> to indicate that this is the group - creator site by setting the <code class="literal">DB_GROUP_CREATOR</code> - parameter. + Configure this handle for the local site's host + and port information when you create the handle. + Then, use <a href="../api_reference/C/dbsite_set_config.html" class="olink">DB_SITE->set_config()</a> to indicate that + this is the group creator site by setting the + <code class="literal">DB_GROUP_CREATOR</code> parameter. </p> </li> <li> - <p> + <p> Start replication as normal by configuring acknowledgement policies, setting replication priorities for the site, and then calling @@ -324,37 +346,39 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="group_mem_upgrade"></a>Upgrading Groups</h3> + <h3 class="title"><a id="group_mem_upgrade"></a>Upgrading groups</h3> </div> </div> </div> - <p> - Prior to the Berkeley DB 11.2.5.2 release, replication group - membership was managed differently than in the way it is - described in the previous sections. For this reason, when you - upgrade from older releases of Berkeley DB to 11.2.5.2 or - later, the upgrade procedure is different than when upgrading - between other releases. + <p> + Prior to the Berkeley DB 11.2.5.2 release, replication + group membership was managed differently than in the way + it is described in the previous sections. For this reason, + when you upgrade from older releases of Berkeley DB to + 11.2.5.2 or later, the upgrade procedure is different than + when upgrading between other releases. </p> - <p> - To perform an upgrade that takes you from the old way of - managing group membership to the new way of managing group - membership (pre-11.2.5.2 to 11.2.5.2 and later), do the - following: + <p> + To perform an upgrade that takes you from the old way + of managing group membership to the new way of managing + group membership (pre-11.2.5.2 to 11.2.5.2 and later), do + the following: </p> <div class="itemizedlist"> <ul type="disc"> <li> <p> - Update your replication code to use the new <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> - handle and related methods. Recompile and thoroughly - test your code to make sure it is production-ready. + Update your replication code to use the new + <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle and related methods. Recompile and + thoroughly test your code to make sure it is + production-ready. </p> </li> <li> <p> - Do the following one production machine at a time. Make - sure to do this at the master site LAST. + Do the following one production machine at a + time. Make sure to do this at the master site + LAST. </p> <div class="orderedlist"> <ol type="1"> @@ -369,50 +393,53 @@ </p> </li> <li> - <p> - Configure a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle for the local site. - Use <a href="../api_reference/C/dbsite_set_config.html" class="olink">DB_SITE->set_config()</a> to indicate that this - is a legacy site by setting the - <code class="literal">DB_LEGACY</code> parameter. + <p> + Configure a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle for the + local site. Use <a href="../api_reference/C/dbsite_set_config.html" class="olink">DB_SITE->set_config()</a> to + indicate that this is a legacy site by + setting the <code class="literal">DB_LEGACY</code> + parameter. </p> </li> <li> <p> - Configure a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle for <span class="emphasis"><em>every - other site</em></span> in the replication - group. Set the <code class="literal">DB_LEGACY</code> - parameter for each of these handles. + Configure a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle for + <span class="emphasis"><em>every other site</em></span> + in the replication group. Set the + <code class="literal">DB_LEGACY</code> parameter + for each of these handles. </p> <p> - Please pay careful attention to this step. To - repeat: a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle MUST be configured - for EVERY site in the replication group. + Please pay careful attention to this + step. To repeat: a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle MUST be + configured for EVERY site in the + replication group. </p> </li> <li> <p> - Start replication. The site is upgraded at this - point. + Start replication. The site is upgraded + at this point. </p> </li> </ol> </div> <p> Once you have performed this procedure for each - production site, making sure to upgrade the master only - after every other site has been upgraded, you are done - upgrading your replicated application to use the - current group membership mechanism. + production site, making sure to upgrade the master + only after every other site has been upgraded, you + are done upgrading your replicated application to + use the current group membership mechanism. </p> </li> </ul> </div> - <p> - On subsequent restarts of your replication code, you do not - need to specify the <code class="literal">DB_LEGACY</code> parameter, nor - do you need to identify all of the replication group members. - However, it is not an error if you do specify this information - on subsequent start ups. + <p> + On subsequent restarts of your replication code, you do + not need to specify the <code class="literal">DB_LEGACY</code> + parameter, nor do you need to identify all of the + replication group members. However, it is not an error if + you do specify this information on subsequent start ups. </p> </div> </div> @@ -424,14 +451,14 @@ <td width="20%" align="center"> <a accesskey="u" href="rep.html">Up</a> </td> - <td width="40%" align="right"> <a accesskey="n" href="rep_filename.html">Next</a></td> + <td width="40%" align="right"> <a accesskey="n" href="rep_partview.html">Next</a></td> </tr> <tr> <td width="40%" align="left" valign="top">Connecting to a new site </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Managing Replication Files</td> + <td width="40%" align="right" valign="top"> Replication views</td> </tr> </table> </div> diff --git a/docs/programmer_reference/hash_conf.html b/docs/programmer_reference/hash_conf.html index 51f31bf2..89879261 100644 --- a/docs/programmer_reference/hash_conf.html +++ b/docs/programmer_reference/hash_conf.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Hash access method specific configuration</th> + <th colspan="3" align="center">Hash access method specific + configuration</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="bt_conf.html">Prev</a> </td> - <th width="60%" align="center">Chapter 2. - Access Method Configuration - </th> + <th width="60%" align="center">Chapter 2. Access Method Configuration </th> <td width="20%" align="right"> <a accesskey="n" href="heap_conf.html">Next</a></td> </tr> </table> @@ -34,7 +33,8 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="hash_conf"></a>Hash access method specific configuration</h2> + <h2 class="title" style="clear: both"><a id="hash_conf"></a>Hash access method specific + configuration</h2> </div> </div> </div> @@ -57,10 +57,11 @@ </dt> </dl> </div> - <p> - There are a series of configuration tasks which you can perform when - using the Hash access method. They are described in the following sections. -</p> + <p> + There are a series of configuration tasks which you can + perform when using the Hash access method. They are described + in the following sections. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> @@ -69,15 +70,21 @@ </div> </div> </div> - <p>The density, or page fill factor, is an approximation of the number of -keys allowed to accumulate in any one bucket, determining when the hash -table grows or shrinks. If you know the average sizes of the keys and -data in your data set, setting the fill factor can enhance performance. -A reasonable rule to use to compute fill factor is:</p> + <p> + The density, or page fill factor, is an approximation of the + number of keys allowed to accumulate in any one bucket, + determining when the hash table grows or shrinks. If you know + the average sizes of the keys and data in your data set, + setting the fill factor can enhance performance. A reasonable + rule to use to compute fill factor is: + </p> <pre class="programlisting">(pagesize - 32) / (average_key_size + average_data_size + 8)</pre> - <p>The desired density within the hash table can be specified by calling -the <a href="../api_reference/C/dbset_h_ffactor.html" class="olink">DB->set_h_ffactor()</a> method. If no density is specified, one will -be selected dynamically as pages are filled.</p> + <p> + The desired density within the hash table can be specified + by calling the <a href="../api_reference/C/dbset_h_ffactor.html" class="olink">DB->set_h_ffactor()</a> method. If no density is + specified, one will be selected dynamically as pages are + filled. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> @@ -87,23 +94,32 @@ be selected dynamically as pages are filled.</p> </div> </div> </div> - <p>The database hash determines in which bucket a particular key will reside. -The goal of hashing keys is to distribute keys equally across the database -pages, therefore it is important that the hash function work well with -the specified keys so that the resulting bucket usage is relatively -uniform. A hash function that does not work well can effectively turn -into a sequential list.</p> - <p>No hash performs equally well on all possible data sets. It is possible -that applications may find that the default hash function performs poorly -with a particular set of keys. The distribution resulting from the hash -function can be checked using the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility. By comparing the -number of hash buckets and the number of keys, one can decide if the entries -are hashing in a well-distributed manner.</p> - <p>The hash function for the hash table can be specified by calling the -<a href="../api_reference/C/dbset_h_hash.html" class="olink">DB->set_h_hash()</a> method. If no hash function is specified, a default -function will be used. Any application-specified hash function must -take a reference to a <a href="../api_reference/C/db.html" class="olink">DB</a> object, a pointer to a byte string and -its length, as arguments and return an unsigned, 32-bit hash value.</p> + <p> + The database hash determines in which bucket a particular + key will reside. The goal of hashing keys is to distribute + keys equally across the database pages, therefore it is + important that the hash function work well with the specified + keys so that the resulting bucket usage is relatively uniform. + A hash function that does not work well can effectively turn + into a sequential list. + </p> + <p> + No hash performs equally well on all possible data sets. It + is possible that applications may find that the default hash + function performs poorly with a particular set of keys. The + distribution resulting from the hash function can be checked + using the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility. By comparing the number of hash buckets + and the number of keys, one can decide if the entries are + hashing in a well-distributed manner. + </p> + <p> + The hash function for the hash table can be specified by + calling the <a href="../api_reference/C/dbset_h_hash.html" class="olink">DB->set_h_hash()</a> method. If no hash function is + specified, a default function will be used. Any + application-specified hash function must take a reference to a + <a href="../api_reference/C/db.html" class="olink">DB</a> object, a pointer to a byte string and its length, as + arguments and return an unsigned, 32-bit hash value. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> @@ -113,16 +129,23 @@ its length, as arguments and return an unsigned, 32-bit hash value.</p> </div> </div> </div> - <p>When setting up the hash database, knowing the expected number of elements -that will be stored in the hash table is useful. This value can be used -by the Hash access method implementation to more accurately construct the -necessary number of buckets that the database will eventually require.</p> - <p>The anticipated number of elements in the hash table can be specified by -calling the <a href="../api_reference/C/dbset_h_nelem.html" class="olink">DB->set_h_nelem()</a> method. If not specified, or set too low, -hash tables will expand gracefully as keys are entered, although a slight -performance degradation may be noticed. In order for the estimated number -of elements to be a useful value to Berkeley DB, the <a href="../api_reference/C/dbset_h_ffactor.html" class="olink">DB->set_h_ffactor()</a> method -must also be called to set the page fill factor.</p> + <p> + When setting up the hash database, knowing the expected + number of elements that will be stored in the hash table is + useful. This value can be used by the Hash access method + implementation to more accurately construct the necessary + number of buckets that the database will eventually + require. + </p> + <p> + The anticipated number of elements in the hash table can be + specified by calling the <a href="../api_reference/C/dbset_h_nelem.html" class="olink">DB->set_h_nelem()</a> method. If not + specified, or set too low, hash tables will expand gracefully + as keys are entered, although a slight performance degradation + may be noticed. In order for the estimated number of elements + to be a useful value to Berkeley DB, the <a href="../api_reference/C/dbset_h_ffactor.html" class="olink">DB->set_h_ffactor()</a> + method must also be called to set the page fill factor. + </p> </div> </div> <div class="navfooter"> @@ -140,7 +163,8 @@ must also be called to set the page fill factor.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Heap access method specific configuration</td> + <td width="40%" align="right" valign="top"> Heap access method specific + configuration</td> </tr> </table> </div> diff --git a/docs/programmer_reference/heap_conf.html b/docs/programmer_reference/heap_conf.html index e22e5a00..aaca8090 100644 --- a/docs/programmer_reference/heap_conf.html +++ b/docs/programmer_reference/heap_conf.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Heap access method specific configuration</th> + <th colspan="3" align="center">Heap access method specific + configuration</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="hash_conf.html">Prev</a> </td> - <th width="60%" align="center">Chapter 2. - Access Method Configuration - </th> + <th width="60%" align="center">Chapter 2. Access Method Configuration </th> <td width="20%" align="right"> <a accesskey="n" href="rq_conf.html">Next</a></td> </tr> </table> @@ -34,35 +33,41 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="heap_conf"></a>Heap access method specific configuration</h2> + <h2 class="title" style="clear: both"><a id="heap_conf"></a>Heap access method specific + configuration</h2> </div> </div> </div> <p> - Configuring the Heap access method is fairly simple. Beyond the general - configuration required for any access method, you can configure how large - the database will become, as well as the amount by which the database grows. + Configuring the Heap access method is fairly simple. Beyond + the general configuration required for any access method, you + can configure how large the database will become, as well as + the amount by which the database grows. </p> <p> - If you provide no configuration relative to the heap size, then the - database will grow without bound. Whether this is desirable depends on - how much disk space is available to your application. + If you provide no configuration relative to the heap size, + then the database will grow without bound. Whether this is + desirable depends on how much disk space is available to your + application. </p> - <p> - You can limit the size of the on-disk database file by using the - <a href="../api_reference/C/dbset_heapsize.html" class="olink">DB->set_heapsize()</a> method. If the size specified on this method is - reached, then further attempts to insert/update records will fail - with a <code class="literal">DB_HEAP_FULL</code> error message. + <p> + You can limit the size of the on-disk database file by + using the <a href="../api_reference/C/dbset_heapsize.html" class="olink">DB->set_heapsize()</a> method. If the size specified on + this method is reached, then further attempts to insert/update + records will fail with a <code class="literal">DB_HEAP_FULL</code> error + message. </p> - <p> - Heap databases are organized into regions, and each region is a constant - size. The size of the region in a heap database is limited by the page - size, the first page of the region contains a bitmap describing the - available space on the remaining pages in the region. When the database - experiences write contention, a region is added to reduce contention. - This means heap databases can grow in size very quickly. In order to - control the amount by which the database increases, the size of the region - is configurable via <a href="../api_reference/C/dbset_heap_regionsize.html" class="olink">DB->set_heap_regionsize()</a>. + <p> + Heap databases are organized into regions, and each region + is a constant size. The size of the region in a heap database + is limited by the page size, the first page of the region + contains a bitmap describing the available space on the + remaining pages in the region. When the database experiences + write contention, a region is added to reduce contention. This + means heap databases can grow in size very quickly. In order + to control the amount by which the database increases, the + size of the region is configurable via + <a href="../api_reference/C/dbset_heap_regionsize.html" class="olink">DB->set_heap_regionsize()</a>. </p> </div> <div class="navfooter"> @@ -76,7 +81,8 @@ <td width="40%" align="right"> <a accesskey="n" href="rq_conf.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Hash access method specific configuration </td> + <td width="40%" align="left" valign="top">Hash access method specific + configuration </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/index.html b/docs/programmer_reference/index.html index fbccbb13..f0df26da 100644 --- a/docs/programmer_reference/index.html +++ b/docs/programmer_reference/index.html @@ -12,7 +12,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -30,11 +30,11 @@ <div class="titlepage"> <div> <div> - <h1 class="title"><a id="idm1386312"></a>Berkeley DB Programmer's Reference Guide</h1> + <h1 class="title"><a id="idm614064"></a>Berkeley DB Programmer's Reference Guide</h1> </div> <div> <div class="legalnotice"> - <a id="idm1958888"></a> + <a id="idm1383864"></a> <p class="legalnotice-title"> <b>Legal Notice</b> </p> @@ -64,7 +64,7 @@ <p> To obtain a copy of this document's original source code, please submit a request to the Oracle Technology Network forum at: - <a class="ulink" href="http://forums.oracle.com/forums/forum.jspa?forumID=271" target="_top">http://forums.oracle.com/forums/forum.jspa?forumID=271</a> + <a class="ulink" href="https://forums.oracle.com/forums/forum.jspa?forumID=271" target="_top">https://forums.oracle.com/forums/forum.jspa?forumID=271</a> </p> @@ -72,7 +72,7 @@ </div> </div> <div> - <p class="pubdate">5/11/2012</p> + <p class="pubdate">2/17/2015</p> </div> </div> <hr /> @@ -112,48 +112,48 @@ </dd> <dt> <span class="chapter"> - <a href="intro.html">1. - Introduction - </a> + <a href="intro.html">1. Introduction </a> </span> </dt> <dd> <dl> <dt> <span class="sect1"> - <a href="intro.html#intro_data">An introduction to data management</a> + <a href="intro.html#intro_data">An introduction to data + management</a> </span> </dt> <dt> <span class="sect1"> - <a href="intro_terrain.html">Mapping the terrain: theory and practice</a> + <a href="intro_terrain.html">Mapping the terrain: theory and + practice</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="intro_terrain.html#idm1895840">Data access and data management</a> + <a href="intro_terrain.html#idm3285464">Data access and data management</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_terrain.html#idm2229408">Relational databases</a> + <a href="intro_terrain.html#idm2046400">Relational databases</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_terrain.html#idm2389408">Object-oriented databases</a> + <a href="intro_terrain.html#idm3331136">Object-oriented databases</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_terrain.html#idm2511776">Network databases</a> + <a href="intro_terrain.html#idm2845328">Network databases</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_terrain.html#idm1916248">Clients and servers</a> + <a href="intro_terrain.html#idm1493864">Clients and servers</a> </span> </dt> </dl> @@ -167,17 +167,17 @@ <dl> <dt> <span class="sect2"> - <a href="intro_dbis.html#idm1665072">Data Access Services</a> + <a href="intro_dbis.html#idm2881808">Data Access Services</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_dbis.html#idm1554168">Data management services</a> + <a href="intro_dbis.html#idm2939256">Data management services</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_dbis.html#idm157888">Design</a> + <a href="intro_dbis.html#idm2483664">Design</a> </span> </dt> </dl> @@ -191,22 +191,22 @@ <dl> <dt> <span class="sect2"> - <a href="intro_dbisnot.html#idm1802280">Berkeley DB is not a relational database</a> + <a href="intro_dbisnot.html#idm1825416">Berkeley DB is not a relational database</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_dbisnot.html#idm2288920">Berkeley DB is not an object-oriented database</a> + <a href="intro_dbisnot.html#idm1180680">Berkeley DB is not an object-oriented database</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_dbisnot.html#idm2354536">Berkeley DB is not a network database</a> + <a href="intro_dbisnot.html#idm2742952">Berkeley DB is not a network database</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_dbisnot.html#idm2301256">Berkeley DB is not a database server</a> + <a href="intro_dbisnot.html#idm940752">Berkeley DB is not a database server</a> </span> </dt> </dl> @@ -223,7 +223,8 @@ </dt> <dt> <span class="sect1"> - <a href="intro_distrib.html">What does the Berkeley DB distribution include?</a> + <a href="intro_distrib.html">What does the Berkeley DB + distribution include?</a> </span> </dt> <dt> @@ -240,22 +241,22 @@ <dl> <dt> <span class="sect2"> - <a href="intro_products.html#idm2240216">Berkeley DB Data Store</a> + <a href="intro_products.html#idm1332168">Berkeley DB Data Store</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_products.html#idm1817232">Berkeley DB Concurrent Data Store</a> + <a href="intro_products.html#idm1927528">Berkeley DB Concurrent Data Store</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_products.html#idm1869736">Berkeley DB Transactional Data Store</a> + <a href="intro_products.html#idm1455680">Berkeley DB Transactional Data Store</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_products.html#idm1577368">Berkeley DB High Availability</a> + <a href="intro_products.html#idm851864">Berkeley DB High Availability</a> </span> </dt> </dl> @@ -264,45 +265,42 @@ </dd> <dt> <span class="chapter"> - <a href="am_conf.html">2. - Access Method Configuration - </a> + <a href="am_conf.html">2. Access Method Configuration </a> </span> </dt> <dd> <dl> <dt> <span class="sect1"> - <a href="am_conf.html#am_conf_intro"> - What are the available access methods? - </a> + <a href="am_conf.html#am_conf_intro"> What are the available access + methods? </a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="am_conf.html#idm2161896">Btree</a> + <a href="am_conf.html#idm2727912">Btree</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_conf.html#idp32168">Hash</a> + <a href="am_conf.html#idm1674312">Hash</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_conf.html#idm2680320">Heap</a> + <a href="am_conf.html#idm1985432">Heap</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_conf.html#idm2335248">Queue</a> + <a href="am_conf.html#idm1872368">Queue</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_conf.html#idm1801904">Recno</a> + <a href="am_conf.html#idm1924872">Recno</a> </span> </dt> </dl> @@ -316,17 +314,17 @@ <dl> <dt> <span class="sect2"> - <a href="am_conf_select.html#idm1772384">Btree or Heap?</a> + <a href="am_conf_select.html#idm2274400">Btree or Heap?</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_conf_select.html#idm2622384">Hash or Btree?</a> + <a href="am_conf_select.html#idm2915784">Hash or Btree?</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_conf_select.html#idm1789184">Queue or Recno?</a> + <a href="am_conf_select.html#idm2732704">Queue or Recno?</a> </span> </dt> </dl> @@ -365,7 +363,8 @@ </dt> <dt> <span class="sect2"> - <a href="general_am_conf.html#am_conf_malloc">Non-local memory allocation</a> + <a href="general_am_conf.html#am_conf_malloc">Non-local memory + allocation</a> </span> </dt> </dl> @@ -384,7 +383,8 @@ </dt> <dt> <span class="sect2"> - <a href="bt_conf.html#am_conf_bt_prefix">Btree prefix comparison</a> + <a href="bt_conf.html#am_conf_bt_prefix">Btree prefix + comparison</a> </span> </dt> <dt> @@ -406,7 +406,8 @@ </dd> <dt> <span class="sect1"> - <a href="hash_conf.html">Hash access method specific configuration</a> + <a href="hash_conf.html">Hash access method specific + configuration</a> </span> </dt> <dd> @@ -430,7 +431,8 @@ </dd> <dt> <span class="sect1"> - <a href="heap_conf.html">Heap access method specific configuration</a> + <a href="heap_conf.html">Heap access method specific + configuration</a> </span> </dt> <dt> @@ -442,22 +444,26 @@ <dl> <dt> <span class="sect2"> - <a href="rq_conf.html#am_conf_recno">Managing record-based databases</a> + <a href="rq_conf.html#am_conf_recno">Managing record-based + databases</a> </span> </dt> <dt> <span class="sect2"> - <a href="rq_conf.html#am_conf_extentsize">Selecting a Queue extent size</a> + <a href="rq_conf.html#am_conf_extentsize">Selecting a Queue extent + size</a> </span> </dt> <dt> <span class="sect2"> - <a href="rq_conf.html#am_conf_re_source">Flat-text backing files</a> + <a href="rq_conf.html#am_conf_re_source">Flat-text backing + files</a> </span> </dt> <dt> <span class="sect2"> - <a href="rq_conf.html#am_conf_renumber">Logically renumbering records</a> + <a href="rq_conf.html#am_conf_renumber">Logically renumbering + records</a> </span> </dt> </dl> @@ -466,9 +472,7 @@ </dd> <dt> <span class="chapter"> - <a href="am.html">3. - Access Method Operations - </a> + <a href="am.html">3. Access Method Operations </a> </span> </dt> <dd> @@ -480,24 +484,25 @@ </dt> <dt> <span class="sect1"> - <a href="am_opensub.html">Opening multiple databases in a single file</a> + <a href="am_opensub.html">Opening multiple databases in a + single file</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="am_opensub.html#idp724392">Configuring databases sharing a file</a> + <a href="am_opensub.html#idm1925008">Configuring databases sharing a file</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_opensub.html#idp768720">Caching databases sharing a file</a> + <a href="am_opensub.html#idm148416">Caching databases sharing a file</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_opensub.html#idp769416">Locking in databases based on sharing a file</a> + <a href="am_opensub.html#idm548184">Locking in databases based on sharing a file</a> </span> </dt> </dl> @@ -511,17 +516,20 @@ <dl> <dt> <span class="sect2"> - <a href="am_partition.html#am_partition_keys">Specifying partition keys</a> + <a href="am_partition.html#am_partition_keys">Specifying partition + keys</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_partition.html#am_partition_function">Partitioning callback</a> + <a href="am_partition.html#am_partition_function">Partitioning + callback</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_partition.html#partition_file_placement">Placing partition files</a> + <a href="am_partition.html#partition_file_placement">Placing partition + files</a> </span> </dt> </dl> @@ -558,7 +566,8 @@ </dt> <dt> <span class="sect1"> - <a href="am_verify.html">Database verification and salvage</a> + <a href="am_verify.html">Database verification and + salvage</a> </span> </dt> <dt> @@ -580,7 +589,7 @@ <dl> <dt> <span class="sect2"> - <a href="am_second.html#idp863384">Error Handling With Secondary Indexes</a> + <a href="am_second.html#idp382496">Error Handling With Secondary Indexes</a> </span> </dt> </dl> @@ -599,7 +608,8 @@ <dl> <dt> <span class="sect2"> - <a href="am_cursor.html#am_curget">Retrieving records with a cursor</a> + <a href="am_cursor.html#am_curget">Retrieving records with a + cursor</a> </span> </dt> <dt> @@ -638,9 +648,7 @@ </dd> <dt> <span class="chapter"> - <a href="am_misc.html">4. - Access Method Wrapup - </a> + <a href="am_misc.html">4. Access Method Wrapup </a> </span> </dt> <dd> @@ -676,12 +684,14 @@ </dd> <dt> <span class="sect1"> - <a href="am_misc_partial.html">Partial record storage and retrieval</a> + <a href="am_misc_partial.html">Partial record storage and + retrieval</a> </span> </dt> <dt> <span class="sect1"> - <a href="am_misc_struct.html">Storing C/C++ structures/objects</a> + <a href="am_misc_struct.html">Storing C/C++ + structures/objects</a> </span> </dt> <dt> @@ -706,26 +716,62 @@ </dt> <dt> <span class="sect1"> - <a href="am_misc_diskspace.html">Disk space requirements</a> + <a href="am_misc_diskspace.html">Disk space + requirements</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="am_misc_diskspace.html#idp1074008">Btree</a> + <a href="am_misc_diskspace.html#idp595712">Btree</a> </span> </dt> <dt> <span class="sect2"> - <a href="am_misc_diskspace.html#idp1074072">Hash</a> + <a href="am_misc_diskspace.html#idp595776">Hash</a> </span> </dt> </dl> </dd> <dt> <span class="sect1"> - <a href="am_misc_db_sql.html">Specifying a Berkeley DB schema using SQL DDL</a> + <a href="blobs.html">BLOB support</a> + </span> + </dt> + <dd> + <dl> + <dt> + <span class="sect2"> + <a href="blobs.html#blob_threshold">The BLOB threshold</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="blobs.html#blob_create">Creating BLOBs</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="blobs.html#blob_access">BLOB access</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="blobs.html#blob_storage">BLOB storage</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="blobs.html#blob_replication">BLOBs and Replication</a> + </span> + </dt> + </dl> + </dd> + <dt> + <span class="sect1"> + <a href="am_misc_db_sql.html">Specifying a Berkeley DB schema + using SQL DDL</a> </span> </dt> <dt> @@ -742,9 +788,7 @@ </dd> <dt> <span class="chapter"> - <a href="java.html">5. - Java API - </a> + <a href="java.html">5. Java API </a> </span> </dt> <dd> @@ -806,7 +850,8 @@ </dt> <dt> <span class="sect2"> - <a href="stl.html#stl_intro_performance">Performance overhead</a> + <a href="stl.html#stl_intro_performance">Performance + overhead</a> </span> </dt> <dt> @@ -835,51 +880,52 @@ <dl> <dt> <span class="sect2"> - <a href="stl_db_usage.html#idp1202384">Registering database and environment handles</a> + <a href="stl_db_usage.html#idp756272">Registering database and environment handles</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_db_usage.html#idp1225928">Truncate requirements</a> + <a href="stl_db_usage.html#idp772680">Truncate requirements</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_db_usage.html#idp1236168">Auto commit support</a> + <a href="stl_db_usage.html#idp773680">Auto commit support</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_db_usage.html#idp1239792">Database and environment identity checks</a> + <a href="stl_db_usage.html#idp792728">Database and environment identity checks</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_db_usage.html#idp1236512">Products, constructors and configurations</a> + <a href="stl_db_usage.html#idp795408">Products, constructors and configurations</a> </span> </dt> </dl> </dd> <dt> <span class="sect1"> - <a href="stl_db_advanced_usage.html">Using advanced Berkeley DB features with dbstl</a> + <a href="stl_db_advanced_usage.html">Using advanced Berkeley DB + features with dbstl</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="stl_db_advanced_usage.html#idp1232296">Using bulk retrieval iterators</a> + <a href="stl_db_advanced_usage.html#idp672128">Using bulk retrieval iterators</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_db_advanced_usage.html#idp1232520">Using the DB_RMW flag</a> + <a href="stl_db_advanced_usage.html#idp794736">Using the DB_RMW flag</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_db_advanced_usage.html#idp1199288">Using secondary index database and secondary containers</a> + <a href="stl_db_advanced_usage.html#idp773392">Using secondary index database and secondary containers</a> </span> </dt> </dl> @@ -891,7 +937,8 @@ </dt> <dt> <span class="sect1"> - <a href="stl_mt_usage.html">Using dbstl in multithreaded applications</a> + <a href="stl_mt_usage.html">Using dbstl in multithreaded + applications</a> </span> </dt> <dt> @@ -903,31 +950,32 @@ <dl> <dt> <span class="sect2"> - <a href="stl_primitive_rw.html#idp1288424">Storing strings</a> + <a href="stl_primitive_rw.html#idp849056">Storing strings</a> </span> </dt> </dl> </dd> <dt> <span class="sect1"> - <a href="stl_complex_rw.html">Store and Retrieve data or objects of complex types </a> + <a href="stl_complex_rw.html">Store and Retrieve data or + objects of complex types </a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="stl_complex_rw.html#idp1279008">Storing varying length objects</a> + <a href="stl_complex_rw.html#idp808056">Storing varying length objects</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_complex_rw.html#idp1278616">Storing arbitrary sequences</a> + <a href="stl_complex_rw.html#idp859120">Storing arbitrary sequences</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_complex_rw.html#idp1344912">Notes</a> + <a href="stl_complex_rw.html#idp909008">Notes</a> </span> </dt> </dl> @@ -958,38 +1006,40 @@ </dd> <dt> <span class="sect1"> - <a href="stl_container_specific.html">Dbstl container specific notes</a> + <a href="stl_container_specific.html">Dbstl container specific + notes</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="stl_container_specific.html#idp1313840">db_vector specific notes</a> + <a href="stl_container_specific.html#idp873704">db_vector specific notes</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_container_specific.html#idp1381768">Associative container specific notes</a> + <a href="stl_container_specific.html#idp948648">Associative container specific notes</a> </span> </dt> </dl> </dd> <dt> <span class="sect1"> - <a href="stl_efficienct_use.html">Using dbstl efficiently</a> + <a href="stl_efficienct_use.html">Using dbstl + efficiently</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="stl_efficienct_use.html#idp1350664">Using iterators efficiently</a> + <a href="stl_efficienct_use.html#idp915608">Using iterators efficiently</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_efficienct_use.html#idp1350448">Using containers efficiently</a> + <a href="stl_efficienct_use.html#idp957584">Using containers efficiently</a> </span> </dt> </dl> @@ -1003,12 +1053,12 @@ <dl> <dt> <span class="sect2"> - <a href="stl_memory_mgmt.html#idp1384984">Freeing memory</a> + <a href="stl_memory_mgmt.html#idp952296">Freeing memory</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_memory_mgmt.html#idp1389512">Type specific notes</a> + <a href="stl_memory_mgmt.html#idp994336">Type specific notes</a> </span> </dt> </dl> @@ -1022,12 +1072,13 @@ <dl> <dt> <span class="sect2"> - <a href="stl_misc.html#idp1407848">Special notes about trivial methods</a> + <a href="stl_misc.html#idp953048">Special notes about trivial methods</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_misc.html#idp1421568">Using correct container and iterator public types</a> + <a href="stl_misc.html#idp993208">Using correct container and iterator public + types</a> </span> </dt> </dl> @@ -1041,9 +1092,7 @@ </dd> <dt> <span class="chapter"> - <a href="arch.html">8. - Berkeley DB Architecture - </a> + <a href="arch.html">8. Berkeley DB Architecture </a> </span> </dt> <dd> @@ -1067,27 +1116,27 @@ <dl> <dt> <span class="sect2"> - <a href="arch_apis.html#idp1156848">C</a> + <a href="arch_apis.html#idp1016624">C</a> </span> </dt> <dt> <span class="sect2"> - <a href="arch_apis.html#idp1467704">C++</a> + <a href="arch_apis.html#idp1020280">C++</a> </span> </dt> <dt> <span class="sect2"> - <a href="arch_apis.html#idp1468224">STL</a> + <a href="arch_apis.html#idp1041496">STL</a> </span> </dt> <dt> <span class="sect2"> - <a href="arch_apis.html#idp1467768">Java</a> + <a href="arch_apis.html#idp1042312">Java</a> </span> </dt> <dt> <span class="sect2"> - <a href="arch_apis.html#idp1485576">Dbm/Ndbm, Hsearch</a> + <a href="arch_apis.html#idp1061592">Dbm/Ndbm, Hsearch</a> </span> </dt> </dl> @@ -1101,17 +1150,17 @@ <dl> <dt> <span class="sect2"> - <a href="arch_script.html#idp1461432">Perl</a> + <a href="arch_script.html#idp1034968">Perl</a> </span> </dt> <dt> <span class="sect2"> - <a href="arch_script.html#idp1460536">PHP</a> + <a href="arch_script.html#idp1033112">PHP</a> </span> </dt> <dt> <span class="sect2"> - <a href="arch_script.html#idp1477280">Tcl</a> + <a href="arch_script.html#idp1052216">Tcl</a> </span> </dt> </dl> @@ -1125,9 +1174,7 @@ </dd> <dt> <span class="chapter"> - <a href="env.html">9. - The Berkeley DB Environment - </a> + <a href="env.html">9. The Berkeley DB Environment </a> </span> </dt> <dd> @@ -1139,7 +1186,8 @@ </dt> <dt> <span class="sect1"> - <a href="env_create.html">Creating a database environment</a> + <a href="env_create.html">Creating a database + environment</a> </span> </dt> <dt> @@ -1149,7 +1197,8 @@ </dt> <dt> <span class="sect1"> - <a href="env_open.html">Opening databases within the environment</a> + <a href="env_open.html">Opening databases within the + environment</a> </span> </dt> <dt> @@ -1159,7 +1208,8 @@ </dt> <dt> <span class="sect1"> - <a href="env_db_config.html">DB_CONFIG configuration file</a> + <a href="env_db_config.html">DB_CONFIG configuration + file</a> </span> </dt> <dt> @@ -1171,17 +1221,17 @@ <dl> <dt> <span class="sect2"> - <a href="env_naming.html#idp1570112">Specifying file naming to Berkeley DB</a> + <a href="env_naming.html#idp1149160">Specifying file naming to Berkeley DB</a> </span> </dt> <dt> <span class="sect2"> - <a href="env_naming.html#idp1584200">Filename resolution in Berkeley DB</a> + <a href="env_naming.html#idp1160456">Filename resolution in Berkeley DB</a> </span> </dt> <dt> <span class="sect2"> - <a href="env_naming.html#idp1605872">Examples</a> + <a href="env_naming.html#idp1182752">Examples</a> </span> </dt> </dl> @@ -1215,16 +1265,16 @@ </dd> <dt> <span class="chapter"> - <a href="cam.html">10. - Berkeley DB Concurrent Data Store Applications - </a> + <a href="cam.html">10. Berkeley DB Concurrent Data Store + Applications </a> </span> </dt> <dd> <dl> <dt> <span class="sect1"> - <a href="cam.html#cam_intro">Concurrent Data Store introduction</a> + <a href="cam.html#cam_intro">Concurrent Data Store + introduction</a> </span> </dt> <dt> @@ -1241,9 +1291,7 @@ </dd> <dt> <span class="chapter"> - <a href="transapp.html">11. - Berkeley DB Transactional Data Store Applications - </a> + <a href="transapp.html">11. Berkeley DB Transactional Data Store Applications </a> </span> </dt> <dd> @@ -1270,7 +1318,8 @@ </dt> <dt> <span class="sect1"> - <a href="transapp_app.html">Architecting Transactional Data Store applications</a> + <a href="transapp_app.html">Architecting Transactional Data + Store applications</a> </span> </dt> <dt> @@ -1324,7 +1373,8 @@ </dt> <dt> <span class="sect1"> - <a href="transapp_admin.html">Environment infrastructure</a> + <a href="transapp_admin.html">Environment + infrastructure</a> </span> </dt> <dt> @@ -1339,7 +1389,8 @@ </dt> <dt> <span class="sect1"> - <a href="transapp_archival.html">Database and log file archival</a> + <a href="transapp_archival.html">Database and log file + archival</a> </span> </dt> <dt> @@ -1379,7 +1430,8 @@ </dt> <dt> <span class="sect1"> - <a href="transapp_throughput.html">Transaction throughput</a> + <a href="transapp_throughput.html">Transaction + throughput</a> </span> </dt> <dt> @@ -1391,9 +1443,7 @@ </dd> <dt> <span class="chapter"> - <a href="rep.html">12. - Berkeley DB Replication - </a> + <a href="rep.html">12. Berkeley DB Replication </a> </span> </dt> <dd> @@ -1425,7 +1475,7 @@ </dt> <dt> <span class="sect1"> - <a href="rep_base_meth.html">Base API Methods</a> + <a href="rep_base_meth.html">Base API methods</a> </span> </dt> <dt> @@ -1440,104 +1490,132 @@ </dt> <dt> <span class="sect1"> - <a href="group_membership.html">Managing Replication Manager Group Membership</a> + <a href="group_membership.html">Managing Replication Manager + group membership</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="group_membership.html#group_mem_add">Adding Sites to a Replication Group</a> + <a href="group_membership.html#group_mem_add">Adding sites to a replication + group</a> </span> </dt> <dt> <span class="sect2"> - <a href="group_membership.html#group_mem_remove">Removing Sites from a Replication Group</a> + <a href="group_membership.html#group_mem_remove">Removing sites from a + replication group</a> </span> </dt> <dt> <span class="sect2"> - <a href="group_membership.html#group_mem_primordialstartup">Primordial Startups</a> + <a href="group_membership.html#group_mem_primordialstartup">Primordial startups</a> </span> </dt> <dt> <span class="sect2"> - <a href="group_membership.html#group_mem_upgrade">Upgrading Groups</a> + <a href="group_membership.html#group_mem_upgrade">Upgrading groups</a> </span> </dt> </dl> </dd> <dt> <span class="sect1"> - <a href="rep_filename.html">Managing Replication Files</a> + <a href="rep_partview.html">Replication views</a> + </span> + </dt> + <dt> + <span class="sect1"> + <a href="rep_filename.html">Managing replication directories and files</a> </span> </dt> + <dd> + <dl> + <dt> + <span class="sect2"> + <a href="rep_filename.html#rep_dir">Replication database directory + considerations</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="rep_filename.html#rep_files">Managing replication internal + files</a> + </span> + </dt> + </dl> + </dd> <dt> <span class="sect1"> - <a href="rep_mgrmulti.html">Running Replication Manager in multiple processes</a> + <a href="rep_mgrmulti.html">Running Replication Manager in + multiple processes</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="rep_mgrmulti.html#idp2239216">One replication process and multiple subordinate processes</a> + <a href="rep_mgrmulti.html#idp1913552">One replication process and multiple subordinate + processes</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_mgrmulti.html#idp2202400">Persistence of local site network address configuration</a> + <a href="rep_mgrmulti.html#idp1910720">Persistence of local site network address + configuration</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_mgrmulti.html#idp2221464">Programming considerations</a> + <a href="rep_mgrmulti.html#idp1905672">Programming considerations</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_mgrmulti.html#idp2233184">Handling failure</a> + <a href="rep_mgrmulti.html#idp1902040">Handling failure</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_mgrmulti.html#idp2233360">Other miscellaneous rules</a> + <a href="rep_mgrmulti.html#idp1877896">Other miscellaneous rules</a> </span> </dt> </dl> </dd> <dt> <span class="sect1"> - <a href="rep_replicate.html">Running Replication using the db_replicate Utility</a> + <a href="rep_replicate.html">Running replication using the + db_replicate utility</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="rep_replicate.html#idp2251336">One Replication Process and Multiple Subordinate Processes</a> + <a href="rep_replicate.html#idp1926936">One replication process and multiple subordinate processes</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_replicate.html#idp2268704">Common Use Case</a> + <a href="rep_replicate.html#idp1906152">Common use case</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_replicate.html#idp2278848">Avoiding Rollback</a> + <a href="rep_replicate.html#idp1939624">Avoiding rollback</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_replicate.html#idp2283896">When to Consider an Integrated HA Application</a> + <a href="rep_replicate.html#idp1937856">When to consider an integrated replication application</a> </span> </dt> </dl> </dd> <dt> <span class="sect1"> - <a href="rep_mgr_ack.html">Choosing a Replication Manager Ack Policy</a> + <a href="rep_mgr_ack.html">Choosing a Replication Manager acknowledgement policy</a> </span> </dt> <dt> @@ -1547,7 +1625,8 @@ </dt> <dt> <span class="sect1"> - <a href="rep_mastersync.html">Synchronizing with a master</a> + <a href="rep_mastersync.html">Synchronizing with a + master</a> </span> </dt> <dd> @@ -1564,12 +1643,12 @@ </dt> <dt> <span class="sect2"> - <a href="rep_mastersync.html#idp2309616">Blocked client operations</a> + <a href="rep_mastersync.html#idp1985136">Blocked client operations</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_mastersync.html#idp2331664">Clients too far out-of-date to synchronize</a> + <a href="rep_mastersync.html#idp2008240">Clients too far out-of-date to synchronize</a> </span> </dt> </dl> @@ -1591,14 +1670,15 @@ </dt> <dt> <span class="sect1"> - <a href="rep_lease.html">Master Leases</a> + <a href="rep_lease.html">Master leases</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="rep_lease.html#masterlease_change_groupsize">Changing Group Size</a> + <a href="rep_lease.html#masterlease_change_groupsize">Changing group + size</a> </span> </dt> </dl> @@ -1629,7 +1709,7 @@ </dd> <dt> <span class="sect1"> - <a href="rep_clock_skew.html">Clock Skew</a> + <a href="rep_clock_skew.html">Clock skew</a> </span> </dt> <dt> @@ -1661,6 +1741,25 @@ <a href="rep_twosite.html">Special considerations for two-site replication groups</a> </span> </dt> + <dd> + <dl> + <dt> + <span class="sect2"> + <a href="rep_twosite.html#twosite_strict">Two-site strict configuration</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="rep_twosite.html#twosite_prefmas">Preferred master mode</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="rep_twosite.html#twosite_other">Other alternatives</a> + </span> + </dt> + </dl> + </dd> <dt> <span class="sect1"> <a href="rep_partition.html">Network partitions</a> @@ -1683,22 +1782,20 @@ </dt> <dt> <span class="sect1"> - <a href="rep_ex_rq.html">Ex_rep_base: putting it all together</a> + <a href="rep_ex_rq.html">Ex_rep_base: putting it all + together</a> </span> </dt> <dt> <span class="sect1"> - <a href="rep_ex_chan.html">Ex_rep_chan: a Replication Manager -channel example</a> + <a href="rep_ex_chan.html">Ex_rep_chan: a Replication Manager channel example</a> </span> </dt> </dl> </dd> <dt> <span class="chapter"> - <a href="xa.html">13. - Distributed Transactions - </a> + <a href="xa.html">13. Distributed Transactions </a> </span> </dt> <dd> @@ -1722,32 +1819,27 @@ channel example</a> <dl> <dt> <span class="sect2"> - <a href="xa_build.html#idp2599152">Communicating with multiple Berkeley DB environments</a> - </span> - </dt> - <dt> - <span class="sect2"> - <a href="xa_build.html#idp2600096">Recovering from GTM failure</a> + <a href="xa_build.html#idp2310216">Communicating with multiple Berkeley DB environments</a> </span> </dt> <dt> <span class="sect2"> - <a href="xa_build.html#idp2584512">Managing the Global Transaction ID (GID) name space</a> + <a href="xa_build.html#idp2311288">Managing the Global Transaction ID (GID) name space</a> </span> </dt> <dt> <span class="sect2"> - <a href="xa_build.html#idp2523640">Maintaining state for each distributed transaction.</a> + <a href="xa_build.html#idp2293376">Maintaining state for each distributed transaction.</a> </span> </dt> <dt> <span class="sect2"> - <a href="xa_build.html#idp2597672">Recovering from the failure of a single environment</a> + <a href="xa_build.html#idp2226880">Recovering from the failure of a single environment</a> </span> </dt> <dt> <span class="sect2"> - <a href="xa_build.html#idp2600560">Recovering from GTM failure</a> + <a href="xa_build.html#idp2287824">Recovering from GTM failure</a> </span> </dt> </dl> @@ -1766,17 +1858,17 @@ channel example</a> <dl> <dt> <span class="sect2"> - <a href="xa_xa_config.html#idp2607440">Update the Resource Manager File in Tuxedo</a> + <a href="xa_xa_config.html#idp2312920">Update the Resource Manager File in Tuxedo</a> </span> </dt> <dt> <span class="sect2"> - <a href="xa_xa_config.html#idp2633056">Build the Transaction Manager Server</a> + <a href="xa_xa_config.html#idp2341840">Build the Transaction Manager Server</a> </span> </dt> <dt> <span class="sect2"> - <a href="xa_xa_config.html#idp2580168">Update the UBBCONFIG File</a> + <a href="xa_xa_config.html#idp1702176">Update the UBBCONFIG File</a> </span> </dt> </dl> @@ -1795,26 +1887,28 @@ channel example</a> </dd> <dt> <span class="chapter"> - <a href="apprec.html">14. - Application Specific Logging and Recovery - </a> + <a href="apprec.html">14. Application Specific Logging and + Recovery </a> </span> </dt> <dd> <dl> <dt> <span class="sect1"> - <a href="apprec.html#apprec_intro">Introduction to application specific logging and recovery</a> + <a href="apprec.html#apprec_intro">Introduction to application + specific logging and recovery</a> </span> </dt> <dt> <span class="sect1"> - <a href="apprec_def.html">Defining application-specific log records</a> + <a href="apprec_def.html">Defining application-specific log + records</a> </span> </dt> <dt> <span class="sect1"> - <a href="apprec_auto.html">Automatically generated functions</a> + <a href="apprec_auto.html">Automatically generated + functions</a> </span> </dt> <dt> @@ -1826,9 +1920,7 @@ channel example</a> </dd> <dt> <span class="chapter"> - <a href="program.html">15. - Programmer Notes - </a> + <a href="program.html">15. Programmer Notes </a> </span> </dt> <dd> @@ -1840,11 +1932,36 @@ channel example</a> </dt> <dt> <span class="sect1"> - <a href="program_errorret.html">Error returns to applications</a> + <a href="program_errorret.html">Error returns to + applications</a> </span> </dt> <dt> <span class="sect1"> + <a href="program_i18n.html">Globalization Support</a> + </span> + </dt> + <dd> + <dl> + <dt> + <span class="sect2"> + <a href="program_i18n.html#idp2473976">Message Format</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="program_i18n.html#idp2493888">Enable Globalization Support</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="program_i18n.html#localization_example">Localization Example</a> + </span> + </dt> + </dl> + </dd> + <dt> + <span class="sect1"> <a href="program_environ.html">Environment variables</a> </span> </dt> @@ -1867,19 +1984,21 @@ channel example</a> <dl> <dt> <span class="sect2"> - <a href="program_namespace.html#idp2805992">C Language Name Space</a> + <a href="program_namespace.html#idp2522952">C Language Name + Space</a> </span> </dt> <dt> <span class="sect2"> - <a href="program_namespace.html#idp2778712">Filesystem Name Space</a> + <a href="program_namespace.html#idp2540280">Filesystem Name Space</a> </span> </dt> </dl> </dd> <dt> <span class="sect1"> - <a href="program_ram.html">Memory-only or Flash configurations</a> + <a href="program_ram.html">Memory-only or Flash + configurations</a> </span> </dt> <dt> @@ -1904,7 +2023,8 @@ channel example</a> </dt> <dt> <span class="sect1"> - <a href="program_perfmon.html">Performance Event Monitoring</a> + <a href="program_perfmon.html">Performance Event + Monitoring</a> </span> </dt> <dd> @@ -1921,7 +2041,8 @@ channel example</a> </dt> <dt> <span class="sect2"> - <a href="program_perfmon.html#program_perfmon_examples">Example Scripts</a> + <a href="program_perfmon.html#program_perfmon_examples">Example + Scripts</a> </span> </dt> <dt> @@ -1940,16 +2061,15 @@ channel example</a> </dd> <dt> <span class="chapter"> - <a href="lock.html">16. - The Locking Subsystem - </a> + <a href="lock.html">16. The Locking Subsystem </a> </span> </dt> <dd> <dl> <dt> <span class="sect1"> - <a href="lock.html#lock_intro">Introduction to the locking subsystem</a> + <a href="lock.html#lock_intro">Introduction to the locking + subsystem</a> </span> </dt> <dt> @@ -1959,7 +2079,8 @@ channel example</a> </dt> <dt> <span class="sect1"> - <a href="lock_max.html">Configuring locking: sizing the system</a> + <a href="lock_max.html">Configuring locking: sizing the + system</a> </span> </dt> <dt> @@ -1974,7 +2095,8 @@ channel example</a> </dt> <dt> <span class="sect1"> - <a href="lock_timeout.html">Deadlock detection using timers</a> + <a href="lock_timeout.html">Deadlock detection using + timers</a> </span> </dt> <dt> @@ -1994,7 +2116,8 @@ channel example</a> </dt> <dt> <span class="sect1"> - <a href="lock_twopl.html">Locking with transactions: two-phase locking</a> + <a href="lock_twopl.html">Locking with transactions: two-phase + locking</a> </span> </dt> <dt> @@ -2004,28 +2127,29 @@ channel example</a> </dt> <dt> <span class="sect1"> - <a href="lock_am_conv.html">Berkeley DB Transactional Data Store locking conventions</a> + <a href="lock_am_conv.html">Berkeley DB Transactional Data + Store locking conventions</a> </span> </dt> <dt> <span class="sect1"> - <a href="lock_nondb.html">Locking and non-Berkeley DB applications</a> + <a href="lock_nondb.html">Locking and non-Berkeley DB + applications</a> </span> </dt> </dl> </dd> <dt> <span class="chapter"> - <a href="log.html">17. - The Logging Subsystem - </a> + <a href="log.html">17. The Logging Subsystem </a> </span> </dt> <dd> <dl> <dt> <span class="sect1"> - <a href="log.html#log_intro">Introduction to the logging subsystem</a> + <a href="log.html#log_intro">Introduction to the logging + subsystem</a> </span> </dt> <dt> @@ -2042,9 +2166,7 @@ channel example</a> </dd> <dt> <span class="chapter"> - <a href="mp.html">18. - The Memory Pool Subsystem - </a> + <a href="mp.html">18. The Memory Pool Subsystem </a> </span> </dt> <dd> @@ -2077,9 +2199,7 @@ channel example</a> </dd> <dt> <span class="chapter"> - <a href="txn.html">19. - The Transaction Subsystem - </a> + <a href="txn.html">19. The Transaction Subsystem </a> </span> </dt> <dd> @@ -2103,17 +2223,17 @@ channel example</a> <dl> <dt> <span class="sect2"> - <a href="txn_limits.html#idp3173248">Transaction IDs</a> + <a href="txn_limits.html#idp2936928">Transaction IDs</a> </span> </dt> <dt> <span class="sect2"> - <a href="txn_limits.html#idp3049928">Cursors</a> + <a href="txn_limits.html#idp2856976">Cursors</a> </span> </dt> <dt> <span class="sect2"> - <a href="txn_limits.html#idp3082656">Multiple Threads of Control</a> + <a href="txn_limits.html#idp2907576">Multiple Threads of Control</a> </span> </dt> </dl> @@ -2122,16 +2242,12 @@ channel example</a> </dd> <dt> <span class="chapter"> - <a href="sequence.html">20. - Sequences - </a> + <a href="sequence.html">20. Sequences </a> </span> </dt> <dt> <span class="chapter"> - <a href="tcl.html">21. - Berkeley DB Extensions: Tcl - </a> + <a href="tcl.html">21. Berkeley DB Extensions: Tcl </a> </span> </dt> <dd> @@ -2145,12 +2261,12 @@ channel example</a> <dl> <dt> <span class="sect2"> - <a href="tcl.html#idp3187200">Installing as a Tcl Package</a> + <a href="tcl.html#idp2951744">Installing as a Tcl Package</a> </span> </dt> <dt> <span class="sect2"> - <a href="tcl.html#idp3177840">Loading Berkeley DB with Tcl</a> + <a href="tcl.html#idp2941624">Loading Berkeley DB with Tcl</a> </span> </dt> </dl> @@ -2179,9 +2295,7 @@ channel example</a> </dd> <dt> <span class="chapter"> - <a href="ext.html">22. - Berkeley DB Extensions - </a> + <a href="ext.html">22. Berkeley DB Extensions </a> </span> </dt> <dd> @@ -2205,9 +2319,8 @@ channel example</a> </dd> <dt> <span class="chapter"> - <a href="dumpload.html">23. - Dumping and Reloading Databases - </a> + <a href="dumpload.html">23. Dumping and Reloading Databases + </a> </span> </dt> <dd> @@ -2231,9 +2344,7 @@ channel example</a> </dd> <dt> <span class="chapter"> - <a href="refs.html">24. - Additional References - </a> + <a href="refs.html">24. Additional References</a> </span> </dt> <dd> @@ -2247,17 +2358,17 @@ channel example</a> <dl> <dt> <span class="sect2"> - <a href="refs.html#idp3221064">Technical Papers on Berkeley DB</a> + <a href="refs.html#idp2980680">Technical Papers on Berkeley DB</a> </span> </dt> <dt> <span class="sect2"> - <a href="refs.html#idp3270728">Background on Berkeley DB Features</a> + <a href="refs.html#idp3039152">Background on Berkeley DB Features</a> </span> </dt> <dt> <span class="sect2"> - <a href="refs.html#idp3264824">Database Systems Theory</a> + <a href="refs.html#idp3033336">Database Systems Theory</a> </span> </dt> </dl> diff --git a/docs/programmer_reference/intro.html b/docs/programmer_reference/intro.html index f1e5670d..81824769 100644 --- a/docs/programmer_reference/intro.html +++ b/docs/programmer_reference/intro.html @@ -14,13 +14,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 1. - Introduction - </th> + <th colspan="3" align="center">Chapter 1. Introduction </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="moreinfo.html">Prev</a> </td> @@ -34,9 +32,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="intro"></a>Chapter 1. - Introduction - </h2> + <h2 class="title"><a id="intro"></a>Chapter 1. Introduction </h2> </div> </div> </div> @@ -47,39 +43,41 @@ <dl> <dt> <span class="sect1"> - <a href="intro.html#intro_data">An introduction to data management</a> + <a href="intro.html#intro_data">An introduction to data + management</a> </span> </dt> <dt> <span class="sect1"> - <a href="intro_terrain.html">Mapping the terrain: theory and practice</a> + <a href="intro_terrain.html">Mapping the terrain: theory and + practice</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="intro_terrain.html#idm1895840">Data access and data management</a> + <a href="intro_terrain.html#idm3285464">Data access and data management</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_terrain.html#idm2229408">Relational databases</a> + <a href="intro_terrain.html#idm2046400">Relational databases</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_terrain.html#idm2389408">Object-oriented databases</a> + <a href="intro_terrain.html#idm3331136">Object-oriented databases</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_terrain.html#idm2511776">Network databases</a> + <a href="intro_terrain.html#idm2845328">Network databases</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_terrain.html#idm1916248">Clients and servers</a> + <a href="intro_terrain.html#idm1493864">Clients and servers</a> </span> </dt> </dl> @@ -93,17 +91,17 @@ <dl> <dt> <span class="sect2"> - <a href="intro_dbis.html#idm1665072">Data Access Services</a> + <a href="intro_dbis.html#idm2881808">Data Access Services</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_dbis.html#idm1554168">Data management services</a> + <a href="intro_dbis.html#idm2939256">Data management services</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_dbis.html#idm157888">Design</a> + <a href="intro_dbis.html#idm2483664">Design</a> </span> </dt> </dl> @@ -117,22 +115,22 @@ <dl> <dt> <span class="sect2"> - <a href="intro_dbisnot.html#idm1802280">Berkeley DB is not a relational database</a> + <a href="intro_dbisnot.html#idm1825416">Berkeley DB is not a relational database</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_dbisnot.html#idm2288920">Berkeley DB is not an object-oriented database</a> + <a href="intro_dbisnot.html#idm1180680">Berkeley DB is not an object-oriented database</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_dbisnot.html#idm2354536">Berkeley DB is not a network database</a> + <a href="intro_dbisnot.html#idm2742952">Berkeley DB is not a network database</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_dbisnot.html#idm2301256">Berkeley DB is not a database server</a> + <a href="intro_dbisnot.html#idm940752">Berkeley DB is not a database server</a> </span> </dt> </dl> @@ -149,7 +147,8 @@ </dt> <dt> <span class="sect1"> - <a href="intro_distrib.html">What does the Berkeley DB distribution include?</a> + <a href="intro_distrib.html">What does the Berkeley DB + distribution include?</a> </span> </dt> <dt> @@ -166,22 +165,22 @@ <dl> <dt> <span class="sect2"> - <a href="intro_products.html#idm2240216">Berkeley DB Data Store</a> + <a href="intro_products.html#idm1332168">Berkeley DB Data Store</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_products.html#idm1817232">Berkeley DB Concurrent Data Store</a> + <a href="intro_products.html#idm1927528">Berkeley DB Concurrent Data Store</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_products.html#idm1869736">Berkeley DB Transactional Data Store</a> + <a href="intro_products.html#idm1455680">Berkeley DB Transactional Data Store</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_products.html#idm1577368">Berkeley DB High Availability</a> + <a href="intro_products.html#idm851864">Berkeley DB High Availability</a> </span> </dt> </dl> @@ -192,43 +191,60 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="intro_data"></a>An introduction to data management</h2> + <h2 class="title" style="clear: both"><a id="intro_data"></a>An introduction to data + management</h2> </div> </div> </div> - <p>Cheap, powerful computing and networking have created countless new -applications that could not have existed a decade ago. The advent of the -World-Wide Web, and its influence in driving the Internet into homes and -businesses, is one obvious example. Equally important, though, is the -shift from large, general-purpose desktop and server computers toward -smaller, special-purpose devices with built-in processing and -communications services.</p> - <p>As computer hardware has spread into virtually every corner of our -lives, of course, software has followed. Software developers today are -building applications not just for conventional desktop and server -environments, but also for handheld computers, home appliances, -networking hardware, cars and trucks, factory floor automation systems, -cellphones, and more.</p> - <p>While these operating environments are diverse, the problems that -software engineers must solve in them are often strikingly similar. Most -systems must deal with the outside world, whether that means -communicating with users or controlling machinery. As a result, most -need some sort of I/O system. Even a simple, single-function system -generally needs to handle multiple tasks, and so needs some kind of -operating system to schedule and manage control threads. Also, many -computer systems must store and retrieve data to track history, record -configuration settings, or manage access.</p> - <p>Data management can be very simple. In some cases, just recording -configuration in a flat text file is enough. More often, though, -programs need to store and search a large amount of data, or -structurally complex data. Database management systems are tools that -programmers can use to do this work quickly and efficiently using -off-the-shelf software.</p> - <p>Of course, database management systems have been around for a long time. -Data storage is a problem dating back to the earliest days of computing. -Software developers can choose from hundreds of good, -commercially-available database systems. The problem is selecting the -one that best solves the problems that their applications face.</p> + <p> + Cheap, powerful computing and networking have created + countless new applications that could not have existed a + decade ago. The advent of the World-Wide Web, and its + influence in driving the Internet into homes and businesses, + is one obvious example. Equally important, though, is the + shift from large, general-purpose desktop and server computers + toward smaller, special-purpose devices with built-in + processing and communications services. + </p> + <p> + As computer hardware has spread into virtually every corner + of our lives, of course, software has followed. Software + developers today are building applications not just for + conventional desktop and server environments, but also for + handheld computers, home appliances, networking hardware, cars + and trucks, factory floor automation systems, cellphones, and + more. + </p> + <p> + While these operating environments are diverse, the problems + that software engineers must solve in them are often + strikingly similar. Most systems must deal with the outside + world, whether that means communicating with users or + controlling machinery. As a result, most need some sort of I/O + system. Even a simple, single-function system generally needs + to handle multiple tasks, and so needs some kind of operating + system to schedule and manage control threads. Also, many + computer systems must store and retrieve data to track + history, record configuration settings, or manage + access. + </p> + <p> + Data management can be very simple. In some cases, just + recording configuration in a flat text file is enough. More + often, though, programs need to store and search a large + amount of data, or structurally complex data. Database + management systems are tools that programmers can use to do + this work quickly and efficiently using off-the-shelf + software. + </p> + <p> + Of course, database management systems have been around for + a long time. Data storage is a problem dating back to the + earliest days of computing. Software developers can choose + from hundreds of good, commercially-available database + systems. The problem is selecting the one that best solves the + problems that their applications face. + </p> </div> </div> <div class="navfooter"> @@ -244,7 +260,8 @@ one that best solves the problems that their applications face.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Mapping the terrain: theory and practice</td> + <td width="40%" align="right" valign="top"> Mapping the terrain: theory and + practice</td> </tr> </table> </div> diff --git a/docs/programmer_reference/intro_dbis.html b/docs/programmer_reference/intro_dbis.html index cae1f92c..a64c1416 100644 --- a/docs/programmer_reference/intro_dbis.html +++ b/docs/programmer_reference/intro_dbis.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="intro_terrain.html">Prev</a> </td> - <th width="60%" align="center">Chapter 1. - Introduction - </th> + <th width="60%" align="center">Chapter 1. Introduction </th> <td width="20%" align="right"> <a accesskey="n" href="intro_dbisnot.html">Next</a></td> </tr> </table> @@ -42,252 +40,266 @@ <dl> <dt> <span class="sect2"> - <a href="intro_dbis.html#idm1665072">Data Access Services</a> + <a href="intro_dbis.html#idm2881808">Data Access Services</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_dbis.html#idm1554168">Data management services</a> + <a href="intro_dbis.html#idm2939256">Data management services</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_dbis.html#idm157888">Design</a> + <a href="intro_dbis.html#idm2483664">Design</a> </span> </dt> </dl> </div> <p> - So far, we have discussed database systems in general terms. It is - time now to consider Berkeley DB in particular and see how it fits - into the framework we have introduced. The key question is, what - kinds of applications should use Berkeley DB? - </p> + So far, we have discussed database systems in general + terms. It is time now to consider Berkeley DB in particular + and see how it fits into the framework we have introduced. The + key question is, what kinds of applications should use + Berkeley DB? </p> <p> - Berkeley DB is an Open Source embedded database library that - provides scalable, high-performance, transaction-protected data - management services to applications. Berkeley DB provides a simple - function-call API for data access and management. + Berkeley DB is an Open Source embedded database library + that provides scalable, high-performance, + transaction-protected data management services to + applications. Berkeley DB provides a simple function-call API + for data access and management. </p> - <p> - By "Open Source," we mean Berkeley DB is distributed under a - license that conforms to the - <a class="ulink" href="http://www.opensource.org/osd.html" target="_top"> Open Source Definition</a>. - This license guarantees Berkeley DB is freely available for use and - redistribution in other Open Source applications. Oracle - Corporation sells commercial licenses allowing the redistribution - of Berkeley DB in proprietary applications. In all cases the - complete source code for Berkeley DB is freely available for - download and use. + <p> + By "Open Source," we mean Berkeley DB is distributed under + a license that conforms to the <a class="ulink" href="http://opensource.org/osd.html" target="_top"> Open Source + Definition</a>. This license guarantees Berkeley DB is + freely available for use and redistribution in other Open + Source applications. Oracle Corporation sells commercial + licenses allowing the redistribution of Berkeley DB in + proprietary applications. In all cases the complete source + code for Berkeley DB is freely available for download and use. </p> <p> - Berkeley DB is "embedded" because it links directly into the - application. It runs in the same address space as the application. - As a result, no inter-process communication, either over the - network or between processes on the same machine, is required for - database operations. Berkeley DB provides a simple function-call - API for a number of programming languages, including C, C++, Java, - Perl, Tcl, Python, and PHP. All database operations happen inside - the library. Multiple processes, or multiple threads in a single - process, can all use the database at the same time as each uses the - Berkeley DB library. Low-level services like locking, transaction - logging, shared buffer management, memory management, and so on are - all handled transparently by the library. + Berkeley DB is "embedded" because it links directly into + the application. It runs in the same address space as the + application. As a result, no inter-process communication, + either over the network or between processes on the same + machine, is required for database operations. Berkeley DB + provides a simple function-call API for a number of + programming languages, including C, C++, Java, Perl, Tcl, + Python, and PHP. All database operations happen inside the + library. Multiple processes, or multiple threads in a single + process, can all use the database at the same time as each + uses the Berkeley DB library. Low-level services like locking, + transaction logging, shared buffer management, memory + management, and so on are all handled transparently by the + library. </p> - <p> - The Berkeley DB library is extremely portable. It runs under almost - all UNIX and Linux variants, Windows, and a number of embedded - real-time operating systems. It runs on both 32-bit and 64-bit - systems. It has been deployed on high-end Internet servers, - desktop machines, and on palmtop computers, set-top boxes, in - network switches, and elsewhere. Once Berkeley DB is linked into - the application, the end user generally does not know that there is - a database present at all. + <p> + The Berkeley DB library is extremely portable. It runs + under almost all UNIX and Linux variants, Windows, and a + number of embedded real-time operating systems. It runs on + both 32-bit and 64-bit systems. It has been deployed on + high-end Internet servers, desktop machines, and on palmtop + computers, set-top boxes, in network switches, and elsewhere. + Once Berkeley DB is linked into the application, the end user + generally does not know that there is a database present at + all. </p> <p> - Berkeley DB is scalable in a number of respects. The database library - itself is quite compact (under 300 kilobytes of text space on common - architectures), which means it is small enough to run in tightly - constrained embedded systems, but yet it can take advantage of - gigabytes of memory and terabytes of disk if you are using hardware that - has those resources. + Berkeley DB is scalable in a number of respects. The + database library itself is quite compact (under 300 kilobytes + of text space on common architectures), which means it is + small enough to run in tightly constrained embedded systems, + but yet it can take advantage of gigabytes of memory and + terabytes of disk if you are using hardware that has those + resources. </p> <p> Each of Berkeley DB's database files can contain up to 256 - terabytes of data, assuming the underlying filesystem is capable of supporting - files of that size. Note that Berkeley DB applications often use - multiple database files. This means that the amount of data your - Berkeley DB application can manage is really limited only by the - constraints imposed by your operating system, filesystem, and physical - hardware. + terabytes of data, assuming the underlying filesystem is + capable of supporting files of that size. Note that Berkeley + DB applications often use multiple database files. This means + that the amount of data your Berkeley DB application can + manage is really limited only by the constraints imposed by + your operating system, filesystem, and physical hardware. </p> - <p> - Berkeley DB also supports high concurrency, allowing thousands of users - to operate on the same database files at the same time. + <p> + Berkeley DB also supports high concurrency, allowing + thousands of users to operate on the same database files at + the same time. </p> - <p> - Berkeley DB generally outperforms relational and object-oriented - database systems in embedded applications for a couple of reasons. - First, because the library runs in the same address space, no - inter-process communication is required for database operations. - The cost of communicating between processes on a single machine, or - among machines on a network, is much higher than the cost of making - a function call. Second, because Berkeley DB uses a simple - function-call interface for all operations, there is no query - language to parse, and no execution plan to produce. + <p> + Berkeley DB generally outperforms relational and + object-oriented database systems in embedded applications for + a couple of reasons. First, because the library runs in the + same address space, no inter-process communication is required + for database operations. The cost of communicating between + processes on a single machine, or among machines on a network, + is much higher than the cost of making a function call. + Second, because Berkeley DB uses a simple function-call + interface for all operations, there is no query language to + parse, and no execution plan to produce. </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm1665072"></a>Data Access Services</h3> + <h3 class="title"><a id="idm2881808"></a>Data Access Services</h3> </div> </div> </div> <p> - Berkeley DB applications can choose the storage structure that - best suits the application. Berkeley DB supports hash tables, - Btrees, simple record-number-based storage, and persistent - queues. Programmers can create tables using any of these - storage structures, and can mix operations on different kinds - of tables in a single application. + Berkeley DB applications can choose the storage + structure that best suits the application. Berkeley DB + supports hash tables, Btrees, simple record-number-based + storage, and persistent queues. Programmers can create + tables using any of these storage structures, and can mix + operations on different kinds of tables in a single + application. </p> <p> - Hash tables are generally good for very large databases that - need predictable search and update times for random-access - records. Hash tables allow users to ask, "Does this key - exist?" or to fetch a record with a known key. Hash tables do - not allow users to ask for records with keys that are close to - a known key. + Hash tables are generally good for very large databases + that need predictable search and update times for + random-access records. Hash tables allow users to ask, + "Does this key exist?" or to fetch a record with a known + key. Hash tables do not allow users to ask for records + with keys that are close to a known key. </p> - <p> + <p> Btrees are better for range-based searches, as when the - application needs to find all records with keys between some - starting and ending value. Btrees also do a better job of - exploiting <span class="emphasis"><em>locality of reference</em></span>. If the - application is likely to touch keys near each other at the same - time, the Btrees work well. The tree structure keeps keys that - are close together near one another in storage, so fetching - nearby values usually does not require a disk access. + application needs to find all records with keys between + some starting and ending value. Btrees also do a better + job of exploiting <span class="emphasis"><em>locality of + reference</em></span>. If the application is likely to + touch keys near each other at the same time, the Btrees + work well. The tree structure keeps keys that are close + together near one another in storage, so fetching nearby + values usually does not require a disk access. </p> <p> - Record-number-based storage is natural for applications that - need to store and fetch records, but that do not have a simple - way to generate keys of their own. In a record number table, - the record number is the key for the record. Berkeley DB will - generate these record numbers automatically. + Record-number-based storage is natural for applications + that need to store and fetch records, but that do not have + a simple way to generate keys of their own. In a record + number table, the record number is the key for the record. + Berkeley DB will generate these record numbers + automatically. </p> - <p> - Queues are well-suited for applications that create records, - and then must deal with those records in creation order. A good - example is on-line purchasing systems. Orders can enter the - system at any time, but should generally be filled in the order - in which they were placed. + <p> + Queues are well-suited for applications that create + records, and then must deal with those records in creation + order. A good example is on-line purchasing systems. + Orders can enter the system at any time, but should + generally be filled in the order in which they were + placed. </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm1554168"></a>Data management services</h3> + <h3 class="title"><a id="idm2939256"></a>Data management services</h3> </div> </div> </div> - <p> + <p> Berkeley DB offers important data management services, - including concurrency, transactions, and recovery. All of these - services work on all of the storage structures. + including concurrency, transactions, and recovery. All of + these services work on all of the storage structures. </p> - <p> - Many users can work on the same database concurrently. Berkeley - DB handles locking transparently, ensuring that two users - working on the same record do not interfere with one - another. + <p> + Many users can work on the same database concurrently. + Berkeley DB handles locking transparently, ensuring that + two users working on the same record do not interfere with + one another. </p> <p> - The library provides strict ACID transaction semantics, by - default. However, applications are allowed to relax the + The library provides strict ACID transaction semantics, + by default. However, applications are allowed to relax the isolation guarantees the database system makes. </p> - <p> - Multiple operations can be grouped into a single transaction, - and can be committed or rolled back atomically. Berkeley DB - uses a technique called <span class="emphasis"><em>two-phase locking</em></span> - to be sure that concurrent transactions are isolated from one - another, and a technique called <span class="emphasis"><em>write-ahead - logging</em></span> to guarantee that committed changes + <p> + Multiple operations can be grouped into a single + transaction, and can be committed or rolled back + atomically. Berkeley DB uses a technique called + <span class="emphasis"><em>two-phase locking</em></span> to be sure that + concurrent transactions are isolated from one another, and + a technique called <span class="emphasis"><em>write-ahead + logging</em></span> to guarantee that committed changes survive application, system, or hardware failures. </p> - <p> - When an application starts up, it can ask Berkeley DB to run - recovery. Recovery restores the database to a clean state, - with all committed changes present, even after a crash. The - database is guaranteed to be consistent and all committed - changes are guaranteed to be present when recovery - completes. + <p> + When an application starts up, it can ask Berkeley DB + to run recovery. Recovery restores the database to a clean + state, with all committed changes present, even after a + crash. The database is guaranteed to be consistent and all + committed changes are guaranteed to be present when + recovery completes. </p> <p> - An application can specify, when it starts up, which data - management services it will use. Some applications need fast, - single-user, non-transactional Btree data storage. In that - case, the application can disable the locking and transaction - systems, and will not incur the overhead of locking or logging. - If an application needs to support multiple concurrent users, - but does not need transactions, it can turn on locking without - transactions. Applications that need concurrent, - transaction-protected database access can enable all of the - subsystems. + An application can specify, when it starts up, which + data management services it will use. Some applications + need fast, single-user, non-transactional Btree data + storage. In that case, the application can disable the + locking and transaction systems, and will not incur the + overhead of locking or logging. If an application needs to + support multiple concurrent users, but does not need + transactions, it can turn on locking without transactions. + Applications that need concurrent, transaction-protected + database access can enable all of the subsystems. </p> - <p> - In all these cases, the application uses the same function-call - API to fetch and update records. + <p> + In all these cases, the application uses the same + function-call API to fetch and update records. </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm157888"></a>Design</h3> + <h3 class="title"><a id="idm2483664"></a>Design</h3> </div> </div> </div> - <p> + <p> Berkeley DB was designed to provide industrial-strength - database services to application developers, without requiring - them to become database experts. It is a classic C-library - style <span class="emphasis"><em>toolkit</em></span>, providing a broad base of - functionality to application writers. Berkeley DB was designed - by programmers, for programmers: its modular design surfaces - simple, orthogonal interfaces to core services, and it provides - mechanism (for example, good thread support) without imposing - policy (for example, the use of threads is not required). Just - as importantly, Berkeley DB allows developers to balance - performance against the need for crash recovery and concurrent - use. An application can use the storage structure that - provides the fastest access to its data and can request only - the degree of logging and locking that it needs. + database services to application developers, without + requiring them to become database experts. It is a classic + C-library style <span class="emphasis"><em>toolkit</em></span>, providing a + broad base of functionality to application writers. + Berkeley DB was designed by programmers, for programmers: + its modular design surfaces simple, orthogonal interfaces + to core services, and it provides mechanism (for example, + good thread support) without imposing policy (for example, + the use of threads is not required). Just as importantly, + Berkeley DB allows developers to balance performance + against the need for crash recovery and concurrent use. An + application can use the storage structure that provides + the fastest access to its data and can request only the + degree of logging and locking that it needs. </p> <p> - Because of the tool-based approach and separate interfaces for - each Berkeley DB subsystem, you can support a complete - transaction environment for other system operations. Berkeley - DB even allows you to wrap transactions around the standard - UNIX file read and write operations! Further, Berkeley DB was - designed to interact correctly with the native system's - toolset, a feature no other database package offers. For - example, on UNIX systems Berkeley DB supports hot backups - (database backups while the database is in use), using standard - UNIX system utilities, for example, dump, tar, cpio, pax or - even cp. On other systems which do not support filesystems - with read isolation, Berkeley DB provides a tool for safely - copying files. + Because of the tool-based approach and separate + interfaces for each Berkeley DB subsystem, you can support + a complete transaction environment for other system + operations. Berkeley DB even allows you to wrap + transactions around the standard UNIX file read and write + operations! Further, Berkeley DB was designed to interact + correctly with the native system's toolset, a feature no + other database package offers. For example, on UNIX + systems Berkeley DB supports hot backups (database backups + while the database is in use), using standard UNIX system + utilities, for example, dump, tar, cpio, pax or even cp. + On other systems which do not support filesystems with + read isolation, Berkeley DB provides a tool for safely + copying files. </p> - <p> - Finally, because scripting language interfaces are available - for Berkeley DB (notably Tcl and Perl), application writers can - build incredibly powerful database engines with little effort. - You can build transaction-protected database applications using - your favorite scripting languages, an increasingly important + <p> + Finally, because scripting language interfaces are + available for Berkeley DB (notably Tcl and Perl), + application writers can build incredibly powerful database + engines with little effort. You can build + transaction-protected database applications using your + favorite scripting languages, an increasingly important feature in a world using CGI scripts to deliver HTML. </p> </div> @@ -303,7 +315,8 @@ <td width="40%" align="right"> <a accesskey="n" href="intro_dbisnot.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Mapping the terrain: theory and practice </td> + <td width="40%" align="left" valign="top">Mapping the terrain: theory and + practice </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/intro_dbisnot.html b/docs/programmer_reference/intro_dbisnot.html index 3b1a68d9..4e5a93a9 100644 --- a/docs/programmer_reference/intro_dbisnot.html +++ b/docs/programmer_reference/intro_dbisnot.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="intro_dbis.html">Prev</a> </td> - <th width="60%" align="center">Chapter 1. - Introduction - </th> + <th width="60%" align="center">Chapter 1. Introduction </th> <td width="20%" align="right"> <a accesskey="n" href="intro_need.html">Next</a></td> </tr> </table> @@ -42,226 +40,256 @@ <dl> <dt> <span class="sect2"> - <a href="intro_dbisnot.html#idm1802280">Berkeley DB is not a relational database</a> + <a href="intro_dbisnot.html#idm1825416">Berkeley DB is not a relational database</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_dbisnot.html#idm2288920">Berkeley DB is not an object-oriented database</a> + <a href="intro_dbisnot.html#idm1180680">Berkeley DB is not an object-oriented database</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_dbisnot.html#idm2354536">Berkeley DB is not a network database</a> + <a href="intro_dbisnot.html#idm2742952">Berkeley DB is not a network database</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_dbisnot.html#idm2301256">Berkeley DB is not a database server</a> + <a href="intro_dbisnot.html#idm940752">Berkeley DB is not a database server</a> </span> </dt> </dl> </div> - <p> - In contrast to most other database systems, Berkeley DB provides relatively - simple data access services. + <p> + In contrast to most other database systems, Berkeley DB + provides relatively simple data access services. </p> <p> Records in Berkeley DB are (<span class="emphasis"><em>key</em></span>, - <span class="emphasis"><em>value</em></span>) pairs. Berkeley DB supports only a few - logical operations on records. They are: + <span class="emphasis"><em>value</em></span>) pairs. Berkeley DB supports + only a few logical operations on records. They are: </p> <div class="itemizedlist"> <ul type="disc"> - <li>Insert a record in a table.</li> - <li>Delete a record from a table.</li> - <li>Find a record in a table by looking up its key.</li> - <li>Update a record that has already been found.</li> + <li> + Insert a record in a table. + </li> + <li> + Delete a record from a table. + </li> + <li> + Find a record in a table by looking up its + key. + </li> + <li> + Update a record that has already been + found. + </li> </ul> </div> <p> - Notice that Berkeley DB never operates on the value part of a - record. Values are simply payload, to be stored with keys and - reliably delivered back to the application on demand. + Notice that Berkeley DB never operates on the value part of + a record. Values are simply payload, to be stored with keys + and reliably delivered back to the application on demand. </p> <p> Both keys and values can be arbitrary byte strings, either - fixed-length or variable-length. As a result, programmers can put - native programming language data structures into the database - without converting them to a foreign record format first. Storage - and retrieval are very simple, but the application needs to know - what the structure of a key and a value is in advance. It cannot - ask Berkeley DB, because Berkeley DB doesn't know. + fixed-length or variable-length. As a result, programmers can + put native programming language data structures into the + database without converting them to a foreign record format + first. Storage and retrieval are very simple, but the + application needs to know what the structure of a key and a + value is in advance. It cannot ask Berkeley DB, because + Berkeley DB doesn't know. </p> <p> This is an important feature of Berkeley DB, and one worth - considering more carefully. On the one hand, Berkeley DB cannot - provide the programmer with any information on the contents or - structure of the values that it stores. The application must - understand the keys and values that it uses. On the other hand, - there is literally no limit to the data types that can be store in - a Berkeley DB database. The application never needs to convert its - own program data into the data types that Berkeley DB supports. - Berkeley DB is able to operate on any data type the application - uses, no matter how complex. + considering more carefully. On the one hand, Berkeley DB + cannot provide the programmer with any information on the + contents or structure of the values that it stores. The + application must understand the keys and values that it uses. + On the other hand, there is literally no limit to the data + types that can be store in a Berkeley DB database. The + application never needs to convert its own program data into + the data types that Berkeley DB supports. Berkeley DB is able + to operate on any data type the application uses, no matter + how complex. </p> <p> - Because both keys and values can be up to four gigabytes in length, - a single record can store images, audio streams, or other large - data values. Large values are not treated specially in Berkeley - DB. They are simply broken into page-sized chunks, and reassembled - on demand when the application needs them. Unlike some other - database systems, Berkeley DB offers no special support for binary - large objects (BLOBs). + Because both keys and values can be up to four gigabytes in + length, a single record can store images, audio streams, or + other large data values. By default large values are not + treated specially in Berkeley DB. They are simply broken into + page-sized chunks, and reassembled on demand when the + application needs them. However, you can configure Berkeley DB + to treat large objects in a special way, so that they are + accessed in a more efficient manner. Note that these + specially-treated large objects are not confined to the four + gigabyte limit used for other database objects. See <a class="xref" href="blobs.html" title="BLOB support">BLOB support</a> for more + information. </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm1802280"></a>Berkeley DB is not a relational database</h3> + <h3 class="title"><a id="idm1825416"></a>Berkeley DB is not a relational database</h3> </div> </div> </div> <p> - While Berkeley DB does provide a set of optional SQL APIs, - usually all access to data stored in Berkeley DB is performed - using the traditional Berkeley DB APIs. + While Berkeley DB does provide a set of optional SQL + APIs, usually all access to data stored in Berkeley DB is + performed using the traditional Berkeley DB APIs. </p> - <p> - The traditional Berkeley DB APIs are the way that most Berkeley DB - users will use Berkeley DB. Although the - interfaces are fairly simple, they are non-standard in that they - do not support SQL statements. + <p> + The traditional Berkeley DB APIs are the way that most + Berkeley DB users will use Berkeley DB. Although the + interfaces are fairly simple, they are non-standard in + that they do not support SQL statements. </p> <p> - That said, Berkeley DB does provide a set of SQL APIs that behave - nearly identically to SQLite. By using these APIs you can interface - with Berkeley DB using SQL statements. For Unix systems, these APIs - are not available by default, while for Windows systems they are - available by default. For more information, see the - <em class="citetitle">Berkeley DB Getting Started with the SQL APIs</em> guide. + That said, Berkeley DB does provide a set of SQL APIs + that behave nearly identically to SQLite. By using these + APIs you can interface with Berkeley DB using SQL + statements. For Unix systems, these APIs are not available + by default, while for Windows systems they are available + by default. For more information, see the + <em class="citetitle">Berkeley DB Getting Started with the SQL APIs</em> guide. </p> - <p> - Be aware that SQL support is a double-edged sword. One big advantage of - relational databases is that they allow users to write simple - declarative queries in a high-level language. The database system - knows everything about the data and can carry out the command. This - means that it's simple to search for data in new ways, and to ask - new questions of the database. No programming is required. + <p> + Be aware that SQL support is a double-edged sword. One + big advantage of relational databases is that they allow + users to write simple declarative queries in a high-level + language. The database system knows everything about the + data and can carry out the command. This means that it's + simple to search for data in new ways, and to ask new + questions of the database. No programming is required. </p> - <p> - On the other hand, if a programmer can predict in advance how an - application will access data, then writing a low-level program to - get and store records can be faster. It eliminates the overhead of - query parsing, optimization, and execution. The programmer must - understand the data representation, and must write the code to do - the work, but once that's done, the application can be very fast. + <p> + On the other hand, if a programmer can predict in + advance how an application will access data, then writing + a low-level program to get and store records can be + faster. It eliminates the overhead of query parsing, + optimization, and execution. The programmer must + understand the data representation, and must write the + code to do the work, but once that's done, the application + can be very fast. </p> - <p> - Unless Berkeley DB is used with its SQL APIs, it has no notion of - <span class="emphasis"><em>schema</em></span> and data types in the way that - relational systems do. Schema is the structure of records in - tables, and the relationships among the tables in the database. For - example, in a relational system the programmer can create a record - from a fixed menu of data types. Because the record types are - declared to the system, the relational engine can reach inside - records and examine individual values in them. In addition, - programmers can use SQL to declare relationships among tables, and - to create indices on tables. Relational engines usually maintain - these relationships and indices automatically. + <p> + Unless Berkeley DB is used with its SQL APIs, it has no + notion of <span class="emphasis"><em>schema</em></span> and data types in + the way that relational systems do. Schema is the + structure of records in tables, and the relationships + among the tables in the database. For example, in a + relational system the programmer can create a record from + a fixed menu of data types. Because the record types are + declared to the system, the relational engine can reach + inside records and examine individual values in them. In + addition, programmers can use SQL to declare relationships + among tables, and to create indices on tables. Relational + engines usually maintain these relationships and indices + automatically. </p> <p> - In Berkeley DB, the key and value in a record are opaque to - Berkeley DB. They may have a rich internal structure, but the - library is unaware of it. As a result, Berkeley DB cannot decompose - the value part of a record into its constituent parts, and cannot - use those parts to find values of interest. Only the application, - which knows the data structure, can do that. Berkeley DB does - support indices on tables and automatically maintain those indices + In Berkeley DB, the key and value in a record are + opaque to Berkeley DB. They may have a rich internal + structure, but the library is unaware of it. As a result, + Berkeley DB cannot decompose the value part of a record + into its constituent parts, and cannot use those parts to + find values of interest. Only the application, which knows + the data structure, can do that. Berkeley DB does support + indices on tables and automatically maintain those indices as their associated tables are modified. </p> - <p> - Berkeley DB is not a relational system. Relational database systems - are semantically rich and offer high-level database access. - Compared to such systems, Berkeley DB is a high-performance, - transactional library for record storage. It is possible to build a - relational system on top of Berkeley DB (indeed, this is what the + <p> + Berkeley DB is not a relational system. Relational + database systems are semantically rich and offer + high-level database access. Compared to such systems, + Berkeley DB is a high-performance, transactional library + for record storage. It is possible to build a relational + system on top of Berkeley DB (indeed, this is what the Berkeley DB SQL API really is). In fact, the popular MySQL - relational system uses Berkeley DB for transaction-protected table - management, and takes care of all the SQL parsing and execution. It - uses Berkeley DB for the storage level, and provides the semantics - and access tools. + relational system uses Berkeley DB for + transaction-protected table management, and takes care of + all the SQL parsing and execution. It uses Berkeley DB for + the storage level, and provides the semantics and access + tools. </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm2288920"></a>Berkeley DB is not an object-oriented database</h3> + <h3 class="title"><a id="idm1180680"></a>Berkeley DB is not an object-oriented database</h3> </div> </div> </div> - <p> - Object-oriented databases are designed for very tight integration - with object-oriented programming languages. Berkeley DB is written - entirely in the C programming language. It includes language - bindings for C++, Java, and other languages, but the library has no - information about the objects created in any object-oriented - application. Berkeley DB never makes method calls on any - application object. It has no idea what methods are defined on user - objects, and cannot see the public or private members of any - instance. The key and value part of all records are opaque to - Berkeley DB. + <p> + Object-oriented databases are designed for very tight + integration with object-oriented programming languages. + Berkeley DB is written entirely in the C programming + language. It includes language bindings for C++, Java, and + other languages, but the library has no information about + the objects created in any object-oriented application. + Berkeley DB never makes method calls on any application + object. It has no idea what methods are defined on user + objects, and cannot see the public or private members of + any instance. The key and value part of all records are + opaque to Berkeley DB. </p> - <p> - Berkeley DB cannot automatically page in objects as they are - accessed, as some object-oriented databases do. The object-oriented - application programmer must decide what records are required, and - must fetch them by making method calls on Berkeley DB objects. + <p> + Berkeley DB cannot automatically page in objects as + they are accessed, as some object-oriented databases do. + The object-oriented application programmer must decide + what records are required, and must fetch them by making + method calls on Berkeley DB objects. </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm2354536"></a>Berkeley DB is not a network database</h3> + <h3 class="title"><a id="idm2742952"></a>Berkeley DB is not a network database</h3> </div> </div> </div> - <p> - Berkeley DB does not support network-style navigation among - records, as network databases do. Records in a Berkeley DB table - may move around over time, as new records are added to the table - and old ones are deleted. Berkeley DB is able to do fast searches - for records based on keys, but there is no way to create a - persistent physical pointer to a record. Applications can only - refer to records by key, not by address. + <p> + Berkeley DB does not support network-style navigation + among records, as network databases do. Records in a + Berkeley DB table may move around over time, as new + records are added to the table and old ones are deleted. + Berkeley DB is able to do fast searches for records based + on keys, but there is no way to create a persistent + physical pointer to a record. Applications can only refer + to records by key, not by address. </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm2301256"></a>Berkeley DB is not a database server</h3> + <h3 class="title"><a id="idm940752"></a>Berkeley DB is not a database server</h3> </div> </div> </div> - <p> - Berkeley DB is not a standalone database server. It is a library, - and runs in the address space of the application that uses it. If - more than one application links in Berkeley DB, then all can use - the same database at the same time; the library handles - coordination among the applications, and guarantees that they do - not interfere with one another. + <p> + Berkeley DB is not a standalone database server. It is + a library, and runs in the address space of the + application that uses it. If more than one application + links in Berkeley DB, then all can use the same database + at the same time; the library handles coordination among + the applications, and guarantees that they do not + interfere with one another. </p> - <p> - It is possible to build a server application that uses Berkeley DB - for data management. For example, many commercial and open source - Lightweight Directory Access Protocol (LDAP) servers use Berkeley - DB for record storage. LDAP clients connect to these servers over - the network. Individual servers make calls through the Berkeley DB - API to find records and return them to clients. On its own, - however, Berkeley DB is not a server. + <p> + It is possible to build a server application that uses + Berkeley DB for data management. For example, many + commercial and open source Lightweight Directory Access + Protocol (LDAP) servers use Berkeley DB for record + storage. LDAP clients connect to these servers over the + network. Individual servers make calls through the + Berkeley DB API to find records and return them to + clients. On its own, however, Berkeley DB is not a server. </p> </div> </div> diff --git a/docs/programmer_reference/intro_distrib.html b/docs/programmer_reference/intro_distrib.html index cd56d6eb..d7f27581 100644 --- a/docs/programmer_reference/intro_distrib.html +++ b/docs/programmer_reference/intro_distrib.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">What does the Berkeley DB distribution include?</th> + <th colspan="3" align="center">What does the Berkeley DB + distribution include?</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="intro_what.html">Prev</a> </td> - <th width="60%" align="center">Chapter 1. - Introduction - </th> + <th width="60%" align="center">Chapter 1. Introduction </th> <td width="20%" align="right"> <a accesskey="n" href="intro_where.html">Next</a></td> </tr> </table> @@ -34,16 +33,20 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="intro_distrib"></a>What does the Berkeley DB distribution include?</h2> + <h2 class="title" style="clear: both"><a id="intro_distrib"></a>What does the Berkeley DB + distribution include?</h2> </div> </div> </div> - <p>The Berkeley DB distribution includes complete source code for the Berkeley DB -library, including all three Berkeley DB products and their supporting -utilities, as well as complete documentation in HTML format. The -distribution includes prebuilt binaries and libraries for a small -number of platforms. The distribution does not include hard-copy -documentation.</p> + <p> + The Berkeley DB distribution includes complete source code + for the Berkeley DB library, including all three Berkeley DB + products and their supporting utilities, as well as complete + documentation in HTML format. The distribution includes + prebuilt binaries and libraries for a small number of + platforms. The distribution does not include hard-copy + documentation. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/intro_need.html b/docs/programmer_reference/intro_need.html index ee00e244..80d2fd8a 100644 --- a/docs/programmer_reference/intro_need.html +++ b/docs/programmer_reference/intro_need.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="intro_dbisnot.html">Prev</a> </td> - <th width="60%" align="center">Chapter 1. - Introduction - </th> + <th width="60%" align="center">Chapter 1. Introduction </th> <td width="20%" align="right"> <a accesskey="n" href="intro_what.html">Next</a></td> </tr> </table> @@ -38,45 +36,65 @@ </div> </div> </div> - <p>Berkeley DB is an ideal database system for applications that need fast, -scalable, and reliable embedded database management. For applications -that need different services, however, it can be a poor choice.</p> - <p>First, do you need the ability to access your data in ways you cannot -predict in advance? If your users want to be able to enter SQL -queries to perform -complicated searches that you cannot program into your application to -begin with, then you should consider a relational engine instead. Berkeley DB -requires a programmer to write code in order to run a new kind of query.</p> - <p>On the other hand, if you can predict your data access patterns up front -— and in particular if you need fairly simple key/value lookups — then -Berkeley DB is a good choice. The queries can be coded up once, and will then -run very quickly because there is no SQL to parse and execute.</p> - <p>Second, are there political arguments for or against a standalone -relational server? If you're building an application for your own use -and have a relational system installed with administrative support -already, it may be simpler to use that than to build and learn Berkeley DB. -On the other hand, if you'll be shipping many copies of your application -to customers, and don't want your customers to have to buy, install, -and manage a separate database system, then Berkeley DB may be a better -choice.</p> - <p>Third, are there any technical advantages to an embedded database? If -you're building an application that will run unattended for long periods -of time, or for end users who are not sophisticated administrators, then -a separate server process may be too big a burden. It will require -separate installation and management, and if it creates new ways for -the application to fail, or new complexities to master in the field, -then Berkeley DB may be a better choice.</p> - <p>The fundamental question is, how closely do your requirements match the -Berkeley DB design? Berkeley DB was conceived and built to provide fast, reliable, -transaction-protected record storage. The library itself was never -intended to provide interactive query support, graphical reporting -tools, or similar services that some other database systems provide. We -have tried always to err on the side of minimalism and simplicity. By -keeping the library small and simple, we create fewer opportunities for -bugs to creep in, and we guarantee that the database system stays fast, -because there is very little code to execute. If your application needs -that set of features, then Berkeley DB is almost certainly the best choice -for you.</p> + <p> + Berkeley DB is an ideal database system for applications + that need fast, scalable, and reliable embedded database + management. For applications that need different services, + however, it can be a poor choice. + </p> + <p> + First, do you need the ability to access your data in ways + you cannot predict in advance? If your users want to be able + to enter SQL queries to perform complicated searches that you + cannot program into your application to begin with, then you + should consider a relational engine instead. Berkeley DB + requires a programmer to write code in order to run a new kind + of query. + </p> + <p> + On the other hand, if you can predict your data access + patterns up front — and in particular if you need fairly + simple key/value lookups — then Berkeley DB is a good + choice. The queries can be coded up once, and will then run + very quickly because there is no SQL to parse and + execute. + </p> + <p> + Second, are there political arguments for or against a + standalone relational server? If you're building an + application for your own use and have a relational system + installed with administrative support already, it may be + simpler to use that than to build and learn Berkeley DB. On + the other hand, if you'll be shipping many copies of your + application to customers, and don't want your customers to + have to buy, install, and manage a separate database system, + then Berkeley DB may be a better choice. + </p> + <p> + Third, are there any technical advantages to an embedded + database? If you're building an application that will run + unattended for long periods of time, or for end users who are + not sophisticated administrators, then a separate server + process may be too big a burden. It will require separate + installation and management, and if it creates new ways for + the application to fail, or new complexities to master in the + field, then Berkeley DB may be a better choice. + </p> + <p> + The fundamental question is, how closely do your + requirements match the Berkeley DB design? Berkeley DB was + conceived and built to provide fast, reliable, + transaction-protected record storage. The library itself was + never intended to provide interactive query support, graphical + reporting tools, or similar services that some other database + systems provide. We have tried always to err on the side of + minimalism and simplicity. By keeping the library small and + simple, we create fewer opportunities for bugs to creep in, + and we guarantee that the database system stays fast, because + there is very little code to execute. If your application + needs that set of features, then Berkeley DB is almost + certainly the best choice for you. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/intro_products.html b/docs/programmer_reference/intro_products.html index 47406366..04422347 100644 --- a/docs/programmer_reference/intro_products.html +++ b/docs/programmer_reference/intro_products.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="intro_where.html">Prev</a> </td> - <th width="60%" align="center">Chapter 1. - Introduction - </th> + <th width="60%" align="center">Chapter 1. Introduction </th> <td width="20%" align="right"> <a accesskey="n" href="am_conf.html">Next</a></td> </tr> </table> @@ -42,29 +40,29 @@ <dl> <dt> <span class="sect2"> - <a href="intro_products.html#idm2240216">Berkeley DB Data Store</a> + <a href="intro_products.html#idm1332168">Berkeley DB Data Store</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_products.html#idm1817232">Berkeley DB Concurrent Data Store</a> + <a href="intro_products.html#idm1927528">Berkeley DB Concurrent Data Store</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_products.html#idm1869736">Berkeley DB Transactional Data Store</a> + <a href="intro_products.html#idm1455680">Berkeley DB Transactional Data Store</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_products.html#idm1577368">Berkeley DB High Availability</a> + <a href="intro_products.html#idm851864">Berkeley DB High Availability</a> </span> </dt> </dl> </div> <p> - Oracle provides four Berkeley DB products, each differing by the - level of database support that they offer. + Oracle provides four Berkeley DB products, each differing + by the level of database support that they offer. </p> <div class="itemizedlist"> <ul type="disc"> @@ -77,34 +75,34 @@ <li> Berkeley DB Transactional Data Store </li> - <li> + <li> Berkeley DB High Availability </li> </ul> </div> <p> - Each product provides additional functionality to the product that - precedes it in the list. As a result, you can download Berkeley DB - and build an application that provides read-only database access - for a single-user, and later add support for more complex database - access patterns for multiple users. - </p> - <p> - The single Open Source distribution of Berkeley DB from Oracle - includes the four products and building the distribution - automatically builds all four products. However, you must use the - same Berkeley DB product throughout an application or group of - applications. + Each product provides additional functionality to the + product that precedes it in the list. As a result, you can + download Berkeley DB and build an application that provides + read-only database access for a single-user, and later add + support for more complex database access patterns for multiple + users. + </p> + <p> + The single Open Source distribution of Berkeley DB from + Oracle includes the four products and building the + distribution automatically builds all four products. However, + you must use the same Berkeley DB product throughout an + application or group of applications. </p> <p> - To redistribute Berkeley DB software, you must have a license for - the Berkeley DB product you use. For further details, refer to the - licensing informaion at: - <a class="ulink" href="http://www.oracle.com/technetwork/database/berkeleydb/downloads/index.html" target="_top">http://www.oracle.com/technetwork/database/berkeleydb/downloads/index.html</a> + To redistribute Berkeley DB software, you must have a + license for the Berkeley DB product you use. For further + details, refer to the licensing informaion at: <a class="ulink" href="http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/downloads/index.html" target="_top">http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/downloads/index.html</a> </p> <p> - A comparison of the four Berkeley DB product features is provided - in the following table. + A comparison of the four Berkeley DB product features is + provided in the following table. </p> <div class="informaltable"> <table border="1" width="80%"> @@ -118,200 +116,141 @@ <thead> <tr> <th align="center"> </th> - <th align="center">Berkeley DB Data Store</th> - <th align="center">Berkeley DB Concurrent Data Store</th> - <th align="center">Berkeley DB Transactional Data Store</th> - <th align="center">Berkeley DB High Availability</th> + <th align="center">Berkeley DB Data + Store</th> + <th align="center">Berkeley DB Concurrent Data + Store</th> + <th align="center">Berkeley DB Transactional + Data Store</th> + <th align="center">Berkeley DB High + Availability</th> </tr> </thead> <tbody> <tr> <td align="left"> - <p> - What is this product? - </p> + <p> What is this product? </p> </td> <td align="left"> - <p> - Provides indexed, single-reader/single-writer embedded data storage - </p> + <p> Provides indexed, + single-reader/single-writer embedded data + storage </p> </td> <td align="left"> - <p> - Adds simple locking with multiple-reader/single-writer capabilities + <p> Adds simple locking with + multiple-reader/single-writer capabilities </p> </td> <td align="left"> - <p> - Adds complete ACID transaction support, as well as recovery - </p> + <p> Adds complete ACID transaction support, + as well as recovery </p> </td> <td align="left"> - <p> - Adds single-master data replication across multiple physical machines - </p> + <p> Adds single-master data replication + across multiple physical machines </p> </td> </tr> <tr> <td align="left"> - <p> - Ensures recovery operation - </p> + <p> Ensures recovery operation </p> </td> <td align="left"> - <p> - No - </p> + <p> No </p> </td> <td align="left"> - <p> - No - </p> + <p> No </p> </td> <td align="left"> - <p> - Yes - </p> + <p> Yes </p> </td> <td align="left"> - <p> - Yes - </p> + <p> Yes </p> </td> </tr> <tr> <td align="left"> - <p> - Provides Locking feature - </p> + <p> Provides Locking feature </p> </td> <td align="left"> - <p> - No - </p> + <p> No </p> </td> <td align="left"> - <p> - Yes - </p> + <p> Yes </p> </td> <td align="left"> - <p> - Yes - </p> + <p> Yes </p> </td> <td align="left"> - <p> - Yes - </p> + <p> Yes </p> </td> </tr> <tr> <td align="left"> - <p> - Provides concurrent read-write access + <p> Provides concurrent read-write access </p> </td> <td align="left"> - <p> - No - </p> + <p> No </p> </td> <td align="left"> - <p> - Yes - </p> + <p> Yes </p> </td> <td align="left"> - <p> - Yes - </p> + <p> Yes </p> </td> <td align="left"> - <p> - Yes - </p> + <p> Yes </p> </td> </tr> <tr> <td align="left"> - <p> - Provides transactional support - </p> + <p> Provides transactional support </p> </td> <td align="left"> - <p> - No - </p> + <p> No </p> </td> <td align="left"> - <p> - No - </p> + <p> No </p> </td> <td align="left"> - <p> - Yes - </p> + <p> Yes </p> </td> <td align="left"> - <p> - Yes - </p> + <p> Yes </p> </td> </tr> <tr> <td align="left"> - <p> - Supports SQL access - </p> + <p> Supports the SQL API </p> </td> <td align="left"> - <p> - No - </p> + <p> No </p> </td> <td align="left"> - <p> - No - </p> + <p> No </p> </td> <td align="left"> - <p> - Yes - </p> + <p> Yes </p> </td> <td align="left"> - <p> - No - </p> + <p> No </p> </td> </tr> <tr> <td align="left"> - <p> - Provides replication support - </p> + <p> Provides replication support </p> </td> <td align="left"> - <p> - No - </p> + <p> No </p> </td> <td align="left"> - <p> - No - </p> + <p> No </p> </td> <td align="left"> - <p> - No - </p> + <p> No </p> </td> <td align="left"> - <p> - Yes - </p> + <p> Yes </p> </td> </tr> </tbody> @@ -321,64 +260,82 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm2240216"></a>Berkeley DB Data Store</h3> + <h3 class="title"><a id="idm1332168"></a>Berkeley DB Data Store</h3> </div> </div> </div> - <p>The Berkeley DB Data Store product is an embeddable, high-performance data store. This product - supports multiple concurrent threads of control, including multiple - processes and multiple threads of control within a process. - However, Berkeley DB Data Store does not support locking, - and hence does not guarantee correct behavior if more than one - thread of control is updating the database at a time. The Berkeley DB Data Store is - intended for use in read-only applications or applications which can - guarantee no more than one thread of control updates the - database at a time.</p> + <p> + The Berkeley DB Data Store product is an embeddable, + high-performance data store. This product supports + multiple concurrent threads of control, including multiple + processes and multiple threads of control within a + process. However, Berkeley DB Data Store does not support + locking, and hence does not guarantee correct behavior if + more than one thread of control is updating the database + at a time. The Berkeley DB Data Store is intended for use + in read-only applications or applications which can + guarantee no more than one thread of control updates the + database at a time. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm1817232"></a>Berkeley DB Concurrent Data Store</h3> + <h3 class="title"><a id="idm1927528"></a>Berkeley DB Concurrent Data Store</h3> </div> </div> </div> - <p>The Berkeley DB Concurrent Data Store product adds multiple-reader, single writer capabilities to - the Berkeley DB Data Store product. This product provides built-in concurrency and locking feature. - Berkeley DB Concurrent Data Store is - intended for applications that need support for concurrent updates to a - database that is largely used for reading.</p> + <p> + The Berkeley DB Concurrent Data Store product adds + multiple-reader, single writer capabilities to the + Berkeley DB Data Store product. This product provides + built-in concurrency and locking feature. Berkeley DB + Concurrent Data Store is intended for applications that + need support for concurrent updates to a database that is + largely used for reading. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm1869736"></a>Berkeley DB Transactional Data Store</h3> + <h3 class="title"><a id="idm1455680"></a>Berkeley DB Transactional Data Store</h3> </div> </div> </div> - <p>The Berkeley DB Transactional Data Store product adds support for transactions and database recovery. - Berkeley DB Transactional Data Store is intended for applications that require - industrial-strength database services, including excellent performance - under high-concurrency workloads of read and write operations, - the ability to commit or roll back multiple changes to the database at - a single instant, and the guarantee that in the event of a - catastrophic system or hardware failure, all committed database changes - are preserved.</p> + <p> + The Berkeley DB Transactional Data Store product adds + support for transactions and database recovery. Berkeley + DB Transactional Data Store is intended for applications + that require industrial-strength database services, + including excellent performance under high-concurrency + workloads of read and write operations, the ability to + commit or roll back multiple changes to the database at a + single instant, and the guarantee that in the event of a + catastrophic system or hardware failure, all committed + database changes are preserved. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm1577368"></a>Berkeley DB High Availability</h3> + <h3 class="title"><a id="idm851864"></a>Berkeley DB High Availability</h3> </div> </div> </div> - <p>The Berkeley DB High Availability product adds support for data replication. A single master system - handles all updates, and distributes these updates to multiple replicas. The number of replicas depends on the application requirements. All replicas can handle read requests during - normal processing. If the master system fails for any reason, one of - the replicas takes over as the new master system, and distributes - updates to the remaining replicas.</p> + <p> + The Berkeley DB High Availability product adds support + for data replication. A single master system handles all + updates, and distributes these updates to multiple + replicas. The number of replicas depends on the + application requirements. All replicas can handle read + requests during normal processing. If the master system + fails for any reason, one of the replicas takes over as + the new master system, and distributes updates to the + remaining replicas. + </p> </div> </div> <div class="navfooter"> @@ -396,9 +353,7 @@ <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 2. - Access Method Configuration - </td> + <td width="40%" align="right" valign="top"> Chapter 2. Access Method Configuration </td> </tr> </table> </div> diff --git a/docs/programmer_reference/intro_terrain.html b/docs/programmer_reference/intro_terrain.html index 4e5ce206..68b7e4cb 100644 --- a/docs/programmer_reference/intro_terrain.html +++ b/docs/programmer_reference/intro_terrain.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Mapping the terrain: theory and practice</th> + <th colspan="3" align="center">Mapping the terrain: theory and + practice</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="intro.html">Prev</a> </td> - <th width="60%" align="center">Chapter 1. - Introduction - </th> + <th width="60%" align="center">Chapter 1. Introduction </th> <td width="20%" align="right"> <a accesskey="n" href="intro_dbis.html">Next</a></td> </tr> </table> @@ -34,7 +33,8 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="intro_terrain"></a>Mapping the terrain: theory and practice</h2> + <h2 class="title" style="clear: both"><a id="intro_terrain"></a>Mapping the terrain: theory and + practice</h2> </div> </div> </div> @@ -42,299 +42,431 @@ <dl> <dt> <span class="sect2"> - <a href="intro_terrain.html#idm1895840">Data access and data management</a> + <a href="intro_terrain.html#idm3285464">Data access and data management</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_terrain.html#idm2229408">Relational databases</a> + <a href="intro_terrain.html#idm2046400">Relational databases</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_terrain.html#idm2389408">Object-oriented databases</a> + <a href="intro_terrain.html#idm3331136">Object-oriented databases</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_terrain.html#idm2511776">Network databases</a> + <a href="intro_terrain.html#idm2845328">Network databases</a> </span> </dt> <dt> <span class="sect2"> - <a href="intro_terrain.html#idm1916248">Clients and servers</a> + <a href="intro_terrain.html#idm1493864">Clients and servers</a> </span> </dt> </dl> </div> - <p>The first step in selecting a database system is figuring out what the -choices are. Decades of research and real-world deployment have produced -countless systems. We need to organize them somehow to reduce the number -of options.</p> - <p>One obvious way to group systems is to use the common labels that -vendors apply to them. The buzzwords here include "network," -"relational," "object-oriented," and "embedded," with some -cross-fertilization like "object-relational" and "embedded network". -Understanding the buzzwords is important. Each has some grounding in -theory, but has also evolved into a practical label for categorizing -systems that work in a certain way.</p> - <p>All database systems, regardless of the buzzwords that apply to them, -provide a few common services. All of them store data, for example. -We'll begin by exploring the common services that all systems provide, -and then examine the differences among the different kinds of systems.</p> + <p> + The first step in selecting a database system is figuring + out what the choices are. Decades of research and real-world + deployment have produced countless systems. We need to + organize them somehow to reduce the number of options. + </p> + <p> + One obvious way to group systems is to use the common labels + that vendors apply to them. The buzzwords here include + "network," "relational," "object-oriented," and "embedded," + with some cross-fertilization like "object-relational" and + "embedded network". Understanding the buzzwords is important. + Each has some grounding in theory, but has also evolved into a + practical label for categorizing systems that work in a + certain way. + </p> + <p> + All database systems, regardless of the buzzwords that apply + to them, provide a few common services. All of them store + data, for example. We'll begin by exploring the common + services that all systems provide, and then examine the + differences among the different kinds of systems. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm1895840"></a>Data access and data management</h3> + <h3 class="title"><a id="idm3285464"></a>Data access and data management</h3> </div> </div> </div> - <p>Fundamentally, database systems provide two services.</p> - <p>The first service is <span class="emphasis"><em>data access</em></span>. Data access means adding -new data to the database (inserting), finding data of interest -(searching), changing data already stored (updating), and removing data -from the database (deleting). All databases provide these services. How -they work varies from category to category, and depends on the record -structure that the database supports.</p> - <p>Each record in a database is a collection of values. For example, the -record for a Web site customer might include a name, email address, -shipping address, and payment information. Records are usually stored -in tables. Each table holds records of the same kind. For example, the -<span class="bold"><strong>customer</strong></span> table at an e-commerce Web site might store the -customer records for every person who shopped at the site. Often, -database records have a different structure from the structures or -instances supported by the programming language in which an application -is written. As a result, working with records can mean:</p> + <p> + Fundamentally, database systems provide two + services. + </p> + <p> + The first service is <span class="emphasis"><em>data access</em></span>. + Data access means adding new data to the database + (inserting), finding data of interest (searching), + changing data already stored (updating), and removing data + from the database (deleting). All databases provide these + services. How they work varies from category to category, + and depends on the record structure that the database + supports. + </p> + <p> + Each record in a database is a collection of values. For + example, the record for a Web site customer might include + a name, email address, shipping address, and payment + information. Records are usually stored in tables. Each + table holds records of the same kind. For example, the + <span class="bold"><strong>customer</strong></span> table at an + e-commerce Web site might store the customer records for + every person who shopped at the site. Often, database + records have a different structure from the structures or + instances supported by the programming language in which + an application is written. As a result, working with + records can mean: + </p> <div class="itemizedlist"> <ul type="disc"> - <li>using database operations like searches and updates on records; and</li> - <li>converting between programming language structures and database record -types in the application.</li> + <li> + using database operations like searches and + updates on records; and + </li> + <li> + converting between programming language + structures and database record types in the + application. + </li> </ul> </div> - <p>The second service is <span class="emphasis"><em>data management</em></span>. Data management is -more complicated than data access. Providing good data management -services is the hard part of building a database system. When you -choose a database system to use in an application you build, making sure -it supports the data management services you need is critical.</p> - <p>Data management services include allowing multiple users to work on the -database simultaneously (concurrency), allowing multiple records to be -changed instantaneously (transactions), and surviving application and -system crashes (recovery). Different database systems offer different -data management services. Data management services are entirely -independent of the data access services listed above. For example, -nothing about relational database theory requires that the system -support transactions, but most commercial relational systems do.</p> - <p>Concurrency means that multiple users can operate on the database at -the same time. Support for concurrency ranges from none (single-user -access only) to complete (many readers and writers working -simultaneously).</p> - <p>Transactions permit users to make multiple changes appear at once. For -example, a transfer of funds between bank accounts needs to be a -transaction because the balance in one account is reduced and the -balance in the other increases. If the reduction happened before the -increase, than a poorly-timed system crash could leave the customer -poorer; if the bank used the opposite order, then the same system crash -could make the customer richer. Obviously, both the customer and the -bank are best served if both operations happen at the same instant.</p> - <p>Transactions have well-defined properties in database systems. They are -<span class="emphasis"><em>atomic</em></span>, so that the changes happen all at once or not at all. -They are <span class="emphasis"><em>consistent</em></span>, so that the database is in a legal state -when the transaction begins and when it ends. They are typically -<span class="emphasis"><em>isolated</em></span>, which means that any other users in the database -cannot interfere with them while they are in progress. And they are -<span class="emphasis"><em>durable</em></span>, so that if the system or application crashes after -a transaction finishes, the changes are not lost. Together, the -properties of <span class="emphasis"><em>atomicity</em></span>, <span class="emphasis"><em>consistency</em></span>, -<span class="emphasis"><em>isolation</em></span>, and <span class="emphasis"><em>durability</em></span> are known as the ACID -properties.</p> - <p>As is the case for concurrency, support for transactions varies among -databases. Some offer atomicity without making guarantees about -durability. Some ignore isolatability, especially in single-user -systems; there's no need to isolate other users from the effects of -changes when there are no other users.</p> - <p>Another important data management service is recovery. Strictly -speaking, recovery is a procedure that the system carries out when it -starts up. The purpose of recovery is to guarantee that the database is -complete and usable. This is most important after a system or -application crash, when the database may have been damaged. The recovery -process guarantees that the internal structure of the database is good. -Recovery usually means that any completed transactions are checked, and -any lost changes are reapplied to the database. At the end of the -recovery process, applications can use the database as if there had been -no interruption in service.</p> - <p>Finally, there are a number of data management services that permit -copying of data. For example, most database systems are able to import -data from other sources, and to export it for use elsewhere. Also, most -systems provide some way to back up databases and to restore in the -event of a system failure that damages the database. Many commercial -systems allow <span class="emphasis"><em>hot backups</em></span>, so that users can back up -databases while they are in use. Many applications must run without -interruption, and cannot be shut down for backups.</p> - <p>A particular database system may provide other data management services. -Some provide browsers that show database structure and contents. Some -include tools that enforce data integrity rules, such as the rule that -no employee can have a negative salary. These data management services -are not common to all systems, however. Concurrency, recovery, and -transactions are the data management services that most database vendors -support.</p> - <p>Deciding what kind of database to use means understanding the data -access and data management services that your application needs. Berkeley DB -is an embedded database that supports fairly simple data access with a -rich set of data management services. To highlight its strengths and -weaknesses, we can compare it to other database system categories.</p> + <p> + The second service is <span class="emphasis"><em>data + management</em></span>. Data management is more + complicated than data access. Providing good data + management services is the hard part of building a + database system. When you choose a database system to use + in an application you build, making sure it supports the + data management services you need is critical. + </p> + <p> + Data management services include allowing multiple users + to work on the database simultaneously (concurrency), + allowing multiple records to be changed instantaneously + (transactions), and surviving application and system + crashes (recovery). Different database systems offer + different data management services. Data management + services are entirely independent of the data access + services listed above. For example, nothing about + relational database theory requires that the system + support transactions, but most commercial relational + systems do. + </p> + <p> + Concurrency means that multiple users can operate on the + database at the same time. Support for concurrency ranges + from none (single-user access only) to complete (many + readers and writers working simultaneously). + </p> + <p> + Transactions permit users to make multiple changes + appear at once. For example, a transfer of funds between + bank accounts needs to be a transaction because the + balance in one account is reduced and the balance in the + other increases. If the reduction happened before the + increase, than a poorly-timed system crash could leave the + customer poorer; if the bank used the opposite order, then + the same system crash could make the customer richer. + Obviously, both the customer and the bank are best served + if both operations happen at the same instant. + </p> + <p> + Transactions have well-defined properties in database + systems. They are <span class="emphasis"><em>atomic</em></span>, so that the + changes happen all at once or not at all. They are + <span class="emphasis"><em>consistent</em></span>, so that the database + is in a legal state when the transaction begins and when + it ends. They are typically <span class="emphasis"><em>isolated</em></span>, + which means that any other users in the database cannot + interfere with them while they are in progress. And they + are <span class="emphasis"><em>durable</em></span>, so that if the system or + application crashes after a transaction finishes, the + changes are not lost. Together, the properties of + <span class="emphasis"><em>atomicity</em></span>, + <span class="emphasis"><em>consistency</em></span>, + <span class="emphasis"><em>isolation</em></span>, and + <span class="emphasis"><em>durability</em></span> are known as the ACID + properties. + </p> + <p> + As is the case for concurrency, support for transactions + varies among databases. Some offer atomicity without + making guarantees about durability. Some ignore + isolatability, especially in single-user systems; there's + no need to isolate other users from the effects of changes + when there are no other users. + </p> + <p> + Another important data management service is recovery. + Strictly speaking, recovery is a procedure that the system + carries out when it starts up. The purpose of recovery is + to guarantee that the database is complete and usable. + This is most important after a system or application + crash, when the database may have been damaged. The + recovery process guarantees that the internal structure of + the database is good. Recovery usually means that any + completed transactions are checked, and any lost changes + are reapplied to the database. At the end of the recovery + process, applications can use the database as if there had + been no interruption in service. + </p> + <p> + Finally, there are a number of data management services + that permit copying of data. For example, most database + systems are able to import data from other sources, and to + export it for use elsewhere. Also, most systems provide + some way to back up databases and to restore in the event + of a system failure that damages the database. Many + commercial systems allow <span class="emphasis"><em>hot backups</em></span>, + so that users can back up databases while they are in use. + Many applications must run without interruption, and + cannot be shut down for backups. + </p> + <p> + A particular database system may provide other data + management services. Some provide browsers that show + database structure and contents. Some include tools that + enforce data integrity rules, such as the rule that no + employee can have a negative salary. These data management + services are not common to all systems, however. + Concurrency, recovery, and transactions are the data + management services that most database vendors + support. + </p> + <p> + Deciding what kind of database to use means + understanding the data access and data management services + that your application needs. Berkeley DB is an embedded + database that supports fairly simple data access with a + rich set of data management services. To highlight its + strengths and weaknesses, we can compare it to other + database system categories. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm2229408"></a>Relational databases</h3> + <h3 class="title"><a id="idm2046400"></a>Relational databases</h3> </div> </div> </div> - <p>Relational databases are probably the best-known database variant, -because of the success of companies like Oracle. Relational databases -are based on the mathematical field of set theory. The term "relation" -is really just a synonym for "set" -- a relation is just a set of -records or, in our terminology, a table. One of the main innovations in -early relational systems was to insulate the programmer from the -physical organization of the database. Rather than walking through -arrays of records or traversing pointers, programmers make statements -about tables in a high-level language, and the system executes those -statements.</p> - <p>Relational databases operate on <span class="emphasis"><em>tuples</em></span>, or records, composed -of values of several different data types, including integers, character -strings, and others. Operations include searching for records whose -values satisfy some criteria, updating records, and so on.</p> - <p>Virtually all relational databases use the Structured Query Language, -or SQL. This language permits people and computer programs to work with -the database by writing simple statements. The database engine reads -those statements and determines how to satisfy them on the tables in -the database.</p> - <p>SQL is the main practical advantage of relational database systems. -Rather than writing a computer program to find records of interest, the -relational system user can just type a query in a simple syntax, and -let the engine do the work. This gives users enormous flexibility; they -do not need to decide in advance what kind of searches they want to do, -and they do not need expensive programmers to find the data they need. -Learning SQL requires some effort, but it's much simpler than a -full-blown high-level programming language for most purposes. And there -are a lot of programmers who have already learned SQL.</p> + <p> + Relational databases are probably the best-known + database variant, because of the success of companies like + Oracle. Relational databases are based on the mathematical + field of set theory. The term "relation" is really just a + synonym for "set" -- a relation is just a set of records + or, in our terminology, a table. One of the main + innovations in early relational systems was to insulate + the programmer from the physical organization of the + database. Rather than walking through arrays of records or + traversing pointers, programmers make statements about + tables in a high-level language, and the system executes + those statements. + </p> + <p> + Relational databases operate on + <span class="emphasis"><em>tuples</em></span>, or records, composed of + values of several different data types, including + integers, character strings, and others. Operations + include searching for records whose values satisfy some + criteria, updating records, and so on. + </p> + <p> + Virtually all relational databases use the Structured + Query Language, or SQL. This language permits people and + computer programs to work with the database by writing + simple statements. The database engine reads those + statements and determines how to satisfy them on the + tables in the database. + </p> + <p> + SQL is the main practical advantage of relational + database systems. Rather than writing a computer program + to find records of interest, the relational system user + can just type a query in a simple syntax, and let the + engine do the work. This gives users enormous flexibility; + they do not need to decide in advance what kind of + searches they want to do, and they do not need expensive + programmers to find the data they need. Learning SQL + requires some effort, but it's much simpler than a + full-blown high-level programming language for most + purposes. And there are a lot of programmers who have + already learned SQL. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm2389408"></a>Object-oriented databases</h3> + <h3 class="title"><a id="idm3331136"></a>Object-oriented databases</h3> </div> </div> </div> - <p>Object-oriented databases are less common than relational systems, but -are still fairly widespread. Most object-oriented databases were -originally conceived as persistent storage systems closely wedded to -particular high-level programming languages like C++. With the spread -of Java, most now support more than one programming language, but -object-oriented database systems fundamentally provide the same class -and method abstractions as do object-oriented programming languages.</p> - <p>Many object-oriented systems allow applications to operate on objects -uniformly, whether they are in memory or on disk. These systems create -the illusion that all objects are in memory all the time. The advantage -to object-oriented programmers who simply want object storage and -retrieval is clear. They need never be aware of whether an object is in -memory or not. The application simply uses objects, and the database -system moves them between disk and memory transparently. All of the -operations on an object, and all its behavior, are determined by the -programming language.</p> - <p>Object-oriented databases aren't nearly as widely deployed as relational -systems. In order to attract developers who understand relational -systems, many of the object-oriented systems have added support for -query languages very much like SQL. In practice, though, object-oriented -databases are mostly used for persistent storage of objects in C++ and -Java programs.</p> + <p> + Object-oriented databases are less common than + relational systems, but are still fairly widespread. Most + object-oriented databases were originally conceived as + persistent storage systems closely wedded to particular + high-level programming languages like C++. With the spread + of Java, most now support more than one programming + language, but object-oriented database systems + fundamentally provide the same class and method + abstractions as do object-oriented programming + languages. + </p> + <p> + Many object-oriented systems allow applications to + operate on objects uniformly, whether they are in memory + or on disk. These systems create the illusion that all + objects are in memory all the time. The advantage to + object-oriented programmers who simply want object storage + and retrieval is clear. They need never be aware of + whether an object is in memory or not. The application + simply uses objects, and the database system moves them + between disk and memory transparently. All of the + operations on an object, and all its behavior, are + determined by the programming language. + </p> + <p> + Object-oriented databases aren't nearly as widely + deployed as relational systems. In order to attract + developers who understand relational systems, many of the + object-oriented systems have added support for query + languages very much like SQL. In practice, though, + object-oriented databases are mostly used for persistent + storage of objects in C++ and Java programs. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm2511776"></a>Network databases</h3> + <h3 class="title"><a id="idm2845328"></a>Network databases</h3> </div> </div> </div> - <p>The "network model" is a fairly old technique for managing and -navigating application data. Network databases are designed to make -pointer traversal very fast. Every record stored in a network database -is allowed to contain pointers to other records. These pointers are -generally physical addresses, so fetching the record to which it refers -just means reading it from disk by its disk address.</p> - <p>Network database systems generally permit records to contain integers, -floating point numbers, and character strings, as well as references to -other records. An application can search for records of interest. After -retrieving a record, the application can fetch any record to which it -refers, quickly.</p> - <p>Pointer traversal is fast because most network systems use physical disk -addresses as pointers. When the application wants to fetch a record, -the database system uses the address to fetch exactly the right string -of bytes from the disk. This requires only a single disk access in all -cases. Other systems, by contrast, often must do more than one disk read -to find a particular record.</p> - <p>The key advantage of the network model is also its main drawback. The -fact that pointer traversal is so fast means that applications that do -it will run well. On the other hand, storing pointers all over the -database makes it very hard to reorganize the database. In effect, once -you store a pointer to a record, it is difficult to move that record -elsewhere. Some network databases handle this by leaving forwarding -pointers behind, but this defeats the speed advantage of doing a single -disk access in the first place. Other network databases find, and fix, -all the pointers to a record when it moves, but this makes -reorganization very expensive. Reorganization is often necessary in -databases, since adding and deleting records over time will consume -space that cannot be reclaimed without reorganizing. Without periodic -reorganization to compact network databases, they can end up with a -considerable amount of wasted space.</p> + <p> + The "network model" is a fairly old technique for + managing and navigating application data. Network + databases are designed to make pointer traversal very + fast. Every record stored in a network database is allowed + to contain pointers to other records. These pointers are + generally physical addresses, so fetching the record to + which it refers just means reading it from disk by its + disk address. + </p> + <p> + Network database systems generally permit records to + contain integers, floating point numbers, and character + strings, as well as references to other records. An + application can search for records of interest. After + retrieving a record, the application can fetch any record + to which it refers, quickly. + </p> + <p> + Pointer traversal is fast because most network systems + use physical disk addresses as pointers. When the + application wants to fetch a record, the database system + uses the address to fetch exactly the right string of + bytes from the disk. This requires only a single disk + access in all cases. Other systems, by contrast, often + must do more than one disk read to find a particular + record. + </p> + <p> + The key advantage of the network model is also its main + drawback. The fact that pointer traversal is so fast means + that applications that do it will run well. On the other + hand, storing pointers all over the database makes it very + hard to reorganize the database. In effect, once you store + a pointer to a record, it is difficult to move that record + elsewhere. Some network databases handle this by leaving + forwarding pointers behind, but this defeats the speed + advantage of doing a single disk access in the first + place. Other network databases find, and fix, all the + pointers to a record when it moves, but this makes + reorganization very expensive. Reorganization is often + necessary in databases, since adding and deleting records + over time will consume space that cannot be reclaimed + without reorganizing. Without periodic reorganization to + compact network databases, they can end up with a + considerable amount of wasted space. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idm1916248"></a>Clients and servers</h3> + <h3 class="title"><a id="idm1493864"></a>Clients and servers</h3> </div> </div> </div> - <p>Database vendors have two choices for system architecture. They can -build a server to which remote clients connect, and do all the database -management inside the server. Alternatively, they can provide a module -that links directly into the application, and does all database -management locally. In either case, the application developer needs -some way of communicating with the database (generally, an Application -Programming Interface (API) that does work in the process or that -communicates with a server to get work done).</p> - <p>Almost all commercial database products are implemented as servers, and -applications connect to them as clients. Servers have several features -that make them attractive.</p> - <p>First, because all of the data is managed by a separate process, and -possibly on a separate machine, it's easy to isolate the database server -from bugs and crashes in the application.</p> - <p>Second, because some database products (particularly relational engines) -are quite large, splitting them off as separate server processes keeps -applications small, which uses less disk space and memory. Relational -engines include code to parse SQL statements, to analyze them and -produce plans for execution, to optimize the plans, and to execute -them.</p> - <p>Finally, by storing all the data in one place and managing it with a -single server, it's easier for organizations to back up, protect, and -set policies on their databases. The enterprise databases for large -companies often have several full-time administrators caring for them, -making certain that applications run quickly, granting and denying -access to users, and making backups.</p> - <p>However, centralized administration can be a disadvantage in some cases. -In particular, if a programmer wants to build an application that uses -a database for storage of important information, then shipping and -supporting the application is much harder. The end user needs to install -and administer a separate database server, and the programmer must -support not just one product, but two. Adding a server process to the -application creates new opportunity for installation mistakes and -run-time problems.</p> + <p> + Database vendors have two choices for system + architecture. They can build a server to which remote + clients connect, and do all the database management inside + the server. Alternatively, they can provide a module that + links directly into the application, and does all database + management locally. In either case, the application + developer needs some way of communicating with the + database (generally, an Application Programming Interface + (API) that does work in the process or that communicates + with a server to get work done). + </p> + <p> + Almost all commercial database products are implemented + as servers, and applications connect to them as clients. + Servers have several features that make them + attractive. + </p> + <p> + First, because all of the data is managed by a separate + process, and possibly on a separate machine, it's easy to + isolate the database server from bugs and crashes in the + application. + </p> + <p> + Second, because some database products (particularly + relational engines) are quite large, splitting them off as + separate server processes keeps applications small, which + uses less disk space and memory. Relational engines + include code to parse SQL statements, to analyze them and + produce plans for execution, to optimize the plans, and to + execute them. + </p> + <p> + Finally, by storing all the data in one place and + managing it with a single server, it's easier for + organizations to back up, protect, and set policies on + their databases. The enterprise databases for large + companies often have several full-time administrators + caring for them, making certain that applications run + quickly, granting and denying access to users, and making + backups. + </p> + <p> + However, centralized administration can be a + disadvantage in some cases. In particular, if a programmer + wants to build an application that uses a database for + storage of important information, then shipping and + supporting the application is much harder. The end user + needs to install and administer a separate database + server, and the programmer must support not just one + product, but two. Adding a server process to the + application creates new opportunity for installation + mistakes and run-time problems. + </p> </div> </div> <div class="navfooter"> @@ -348,9 +480,7 @@ run-time problems.</p> <td width="40%" align="right"> <a accesskey="n" href="intro_dbis.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 1. - Introduction - </td> + <td width="40%" align="left" valign="top">Chapter 1. Introduction </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/intro_what.html b/docs/programmer_reference/intro_what.html index 84da80b6..e9cd540f 100644 --- a/docs/programmer_reference/intro_what.html +++ b/docs/programmer_reference/intro_what.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="intro_need.html">Prev</a> </td> - <th width="60%" align="center">Chapter 1. - Introduction - </th> + <th width="60%" align="center">Chapter 1. Introduction </th> <td width="20%" align="right"> <a accesskey="n" href="intro_distrib.html">Next</a></td> </tr> </table> @@ -38,49 +36,72 @@ </div> </div> </div> - <p>Berkeley DB also provides core database services to developers. These -services include:</p> + <p> + Berkeley DB also provides core database services to + developers. These services include: + </p> <div class="variablelist"> <dl> <dt> <span class="term">Page cache management:</span> </dt> - <dd>The page cache provides fast access to a cache of database pages, -handling the I/O associated with the cache to ensure that dirty pages -are written back to the file system and that new pages are allocated on -demand. Applications may use the Berkeley DB shared memory buffer manager to -serve their own files and pages.</dd> + <dd> + The page cache provides fast access to a cache + of database pages, handling the I/O associated with + the cache to ensure that dirty pages are written back + to the file system and that new pages are allocated on + demand. Applications may use the Berkeley DB shared + memory buffer manager to serve their own files and + pages. + </dd> <dt> <span class="term">Transactions and logging:</span> </dt> - <dd>The transaction and logging systems provide recoverability and atomicity -for multiple database operations. The transaction system uses two-phase -locking and write-ahead logging protocols to ensure that database -operations may be undone or redone in the case of application or system -failure. Applications may use Berkeley DB transaction and logging subsystems -to protect their own data structures and operations from application or -system failure.</dd> + <dd> + The transaction and logging systems provide + recoverability and atomicity for multiple database + operations. The transaction system uses two-phase + locking and write-ahead logging protocols to ensure + that database operations may be undone or redone in + the case of application or system failure. + Applications may use Berkeley DB transaction and + logging subsystems to protect their own data + structures and operations from application or system + failure. + </dd> <dt> <span class="term">Locking:</span> </dt> - <dd>The locking system provides multiple reader or single writer access to -objects. The Berkeley DB access methods use the locking system to acquire -the right to read or write database pages. Applications may use the -Berkeley DB locking subsystem to support their own locking needs.</dd> + <dd> + The locking system provides multiple reader or + single writer access to objects. The Berkeley DB + access methods use the locking system to acquire the + right to read or write database pages. Applications + may use the Berkeley DB locking subsystem to support + their own locking needs. + </dd> </dl> </div> - <p>By combining the page cache, transaction, locking, and logging systems, -Berkeley DB provides the same services found in much larger, more complex and -more expensive database systems. Berkeley DB supports multiple simultaneous -readers and writers and guarantees that all changes are recoverable, even -in the case of a catastrophic hardware failure during a database update.</p> - <p>Developers may select some or all of the core database services for any -access method or database. Therefore, it is possible to choose the -appropriate storage structure and the right degrees of concurrency and -recoverability for any application. In addition, some of the subsystems -(for example, the Locking subsystem) can be called separately from the -Berkeley DB access method. As a result, developers can integrate non-database -objects into their transactional applications using Berkeley DB.</p> + <p> + By combining the page cache, transaction, locking, and + logging systems, Berkeley DB provides the same services found + in much larger, more complex and more expensive database + systems. Berkeley DB supports multiple simultaneous readers + and writers and guarantees that all changes are recoverable, + even in the case of a catastrophic hardware failure during a + database update. + </p> + <p> + Developers may select some or all of the core database + services for any access method or database. Therefore, it is + possible to choose the appropriate storage structure and the + right degrees of concurrency and recoverability for any + application. In addition, some of the subsystems (for example, + the Locking subsystem) can be called separately from the + Berkeley DB access method. As a result, developers can + integrate non-database objects into their transactional + applications using Berkeley DB. + </p> </div> <div class="navfooter"> <hr /> @@ -97,7 +118,8 @@ objects into their transactional applications using Berkeley DB.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> What does the Berkeley DB distribution include?</td> + <td width="40%" align="right" valign="top"> What does the Berkeley DB + distribution include?</td> </tr> </table> </div> diff --git a/docs/programmer_reference/intro_where.html b/docs/programmer_reference/intro_where.html index 93d287a2..302ef4ae 100644 --- a/docs/programmer_reference/intro_where.html +++ b/docs/programmer_reference/intro_where.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="intro_distrib.html">Prev</a> </td> - <th width="60%" align="center">Chapter 1. - Introduction - </th> + <th width="60%" align="center">Chapter 1. Introduction </th> <td width="20%" align="right"> <a accesskey="n" href="intro_products.html">Next</a></td> </tr> </table> @@ -38,32 +36,35 @@ </div> </div> </div> + <p> + Berkeley DB requires only underlying IEEE/ANSI Std 1003.1 + (POSIX) system calls and can be ported easily to new + architectures by adding stub routines to connect the native + system interfaces to the Berkeley DB POSIX-style system calls. + See the Berkeley DB Porting Guide for more information. + </p> + <p> + Berkeley DB will autoconfigure and run on almost any modern + UNIX, POSIX or Linux systems, and on most historical UNIX + platforms. Berkeley DB will autoconfigure and run on almost + any GNU gcc toolchain-based embedded platform, including + Cygwin, OpenLinux and others. See the Berkeley DB Installation and Build Guide for + more information. + </p> + <p> + The Berkeley DB distribution includes support for QNX + Neutrino. See the Berkeley DB Installation and Build Guide for more information. + </p> + <p> + The Berkeley DB distribution includes support for VxWorks. + See the Berkeley DB Installation and Build Guide for more information. + </p> <p> - Berkeley DB requires only underlying IEEE/ANSI Std 1003.1 (POSIX) - system calls and can be ported easily to new architectures by adding - stub routines to connect the native system interfaces to the Berkeley - DB POSIX-style system calls. See the Berkeley DB Porting Guide for more information. -</p> - <p> - Berkeley DB will autoconfigure and run on almost any modern UNIX, POSIX - or Linux systems, and on most historical UNIX platforms. Berkeley DB - will autoconfigure and run on almost any GNU gcc toolchain-based - embedded platform, including Cygwin, OpenLinux and others. See - the Berkeley DB Installation and Build Guide for more information. -</p> - <p> - The Berkeley DB distribution includes support for QNX Neutrino. See - the Berkeley DB Installation and Build Guide for more information. -</p> - <p> - The Berkeley DB distribution includes support for VxWorks. See - the Berkeley DB Installation and Build Guide for more information. -</p> - <p> - The Berkeley DB distribution includes support for Windows/NT, - Windows/2000 and Windows/XP, via the Microsoft Visual C++ 6.0 and .NET - development environments. See the Berkeley DB Installation and Build Guide for more information. -</p> + The Berkeley DB distribution includes support for + Windows/NT, Windows/2000 and Windows/XP, via the Microsoft + Visual C++ 6.0 and .NET development environments. See the + Berkeley DB Installation and Build Guide for more information. + </p> </div> <div class="navfooter"> <hr /> @@ -76,7 +77,8 @@ <td width="40%" align="right"> <a accesskey="n" href="intro_products.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">What does the Berkeley DB distribution include? </td> + <td width="40%" align="left" valign="top">What does the Berkeley DB + distribution include? </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/java.html b/docs/programmer_reference/java.html index b14e40db..b154d4c9 100644 --- a/docs/programmer_reference/java.html +++ b/docs/programmer_reference/java.html @@ -14,13 +14,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 5. - Java API - </th> + <th colspan="3" align="center">Chapter 5. Java API </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="am_misc_faq.html">Prev</a> </td> @@ -34,9 +32,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="java"></a>Chapter 5. - Java API - </h2> + <h2 class="title"><a id="java"></a>Chapter 5. Java API </h2> </div> </div> </div> @@ -75,18 +71,24 @@ </div> </div> </div> - <p>Building the Berkeley DB java classes, the examples and the native support -library is integrated into the normal build process. See -<a href="../installation/build_unix_conf.html" class="olink">Configuring Berkeley DB</a> and <a href="../installation/build_win_java.html" class="olink">Building the Java API</a> in the Berkeley DB Installation and Build Guide -for more information.</p> - <p>We expect that you already installed the Java JDK or equivalent on your -system. For the sake of discussion, we assume that it is in a directory -called db-VERSION; for example, you downloaded a Berkeley DB archive, and you -did not change the top-level directory name. The files related to Java -are in three subdirectories of db-VERSION: java (the java source files), -libdb_java (the C++ files that provide the "glue" between java and -Berkeley DB) and examples_java (containing all examples code). The directory -tree looks like this:</p> + <p> + Building the Berkeley DB java classes, the examples and the + native support library is integrated into the normal build + process. See <a href="../installation/build_unix_conf.html" class="olink">Configuring Berkeley DB</a> and <a href="../installation/build_win_java.html" class="olink">Building the Java API</a> in the + Berkeley DB Installation and Build Guide for more information. + </p> + <p> + We expect that you already installed the Java JDK or + equivalent on your system. For the sake of discussion, we + assume that it is in a directory called db-VERSION; for + example, you downloaded a Berkeley DB archive, and you did not + change the top-level directory name. The files related to Java + are in three subdirectories of db-VERSION: java (the java + source files), libdb_java (the C++ files that provide the + "glue" between java and Berkeley DB) and examples/java + (containing all examples code). The directory tree looks like + this: + </p> <pre class="programlisting">db-VERSION |-- java | `-- src @@ -103,55 +105,83 @@ tree looks like this:</p> `-- libdb_java `-- ... </pre> - <p>This naming conforms to the de facto standard for naming java packages. -When the java code is built, it is placed into two jar files: -<code class="filename">db.jar</code>, containing the db package, -and <code class="filename">dbexamples.jar</code>, containing the examples.</p> - <p>For your application to use Berkeley DB successfully, you must set your -<code class="literal">CLASSPATH</code> environment variable to include the full pathname of -the db jar files as well as the classes in your java distribution. -On UNIX, <code class="literal">CLASSPATH</code> is a colon-separated -list of directories and jar files; -on Windows, it is separated by semicolons. -On UNIX, the jar files are put in your build directory, and when -you do the make install step, they are copied to the lib directory -of your installation tree. On Windows, the jar files are placed -in the Release or Debug subdirectory with your other objects.</p> - <p>The Berkeley DB Java classes are mostly implemented in native -methods. Before you can use them, you need to make sure that the -DLL or shared library containing the native methods can be found -by your Java runtime. On Windows, you should set your PATH variable -to include:</p> + <p> + This naming conforms to the de facto standard for naming + java packages. When the java code is built, it is placed into + two jar files: <code class="filename">db.jar</code>, containing the db + package, and <code class="filename">dbexamples.jar</code>, containing + the examples. + </p> + <p> + For your application to use Berkeley DB successfully, you + must set your <code class="literal">CLASSPATH</code> environment + variable to include the full pathname of the db jar files as + well as the classes in your java distribution. On UNIX, + <code class="literal">CLASSPATH</code> is a colon-separated list of + directories and jar files; on Windows, it is separated by + semicolons. On UNIX, the jar files are put in your build + directory, and when you do the make install step, they are + copied to the lib directory of your installation tree. On + Windows, the jar files are placed in the Release or Debug + subdirectory with your other objects. + </p> + <p> + The Berkeley DB Java classes are mostly implemented in + native methods. Before you can use them, you need to make sure + that the DLL or shared library containing the native methods + can be found by your Java runtime. On Windows, you should set + your PATH variable to include: + </p> <pre class="programlisting"> <code class="filename">db-VERSION\build_windows\Release</code> </pre> - <p>On UNIX, you should set the -<code class="literal">LD_LIBRARY_PATH</code> environment variable or local equivalent -to include the Berkeley DB library installation directory. Of course, the -standard install directory may have been changed for your site; see your -system administrator for details.</p> - <p>On other platforms, the path can be set on the command line as follows -(assuming the shared library is in <code class="filename">/usr/local/BerkeleyDB/lib</code>:)</p> + <p> + On UNIX, you should set the + <code class="literal">LD_LIBRARY_PATH</code> environment variable or + local equivalent to include the Berkeley DB library + installation directory. Of course, the standard install + directory may have been changed for your site; see your system + administrator for details. + </p> + <p> + On other platforms, the path can be set on the command line + as follows (assuming the shared library is in + <code class="filename">/usr/local/BerkeleyDB/lib</code>:) + </p> <pre class="programlisting">% java -Djava.library.path=/usr/local/BerkeleyDB/lib ...</pre> - <p>Regardless, if you get the following exception when you run, you -probably do not have the library search path configured correctly:</p> + <p> + Regardless, if you get the following exception when you run, + you probably do not have the library search path configured + correctly: + </p> <pre class="programlisting">java.lang.UnsatisfiedLinkError</pre> - <p>Different Java interpreters provide different error messages if the -<code class="literal">CLASSPATH</code> value is incorrect, a typical error is the following:</p> + <p> + Different Java interpreters provide different error messages + if the <code class="literal">CLASSPATH</code> value is incorrect, a + typical error is the following: + </p> <pre class="programlisting">java.lang.NoClassDefFoundError</pre> - <p>To ensure that everything is running correctly, you may want to try a -simple test from the example programs in</p> + <p> + To ensure that everything is running correctly, you may want + to try a simple test from the example programs in + </p> <pre class="programlisting"> - <code class="filename">db-VERSION/examples_java/src/db</code> + <code class="filename">db-VERSION/examples/java/src/db</code> </pre> - <p>For example, the following sample program will prompt for text input -lines, which are then stored in a Btree database named <code class="filename">access.db</code> in -your current directory:</p> + <p> + For example, the following sample program will prompt for + text input lines, which are then stored in a Btree database + named <code class="filename">access.db</code> in your current + directory: + </p> <pre class="programlisting">% java db.AccessExample</pre> - <p>Try giving it a few lines of input text and then end-of-file. Before -it exits, you should see a list of the lines you entered display with -data items. This is a simple check to make sure the fundamental -configuration is working correctly.</p> + <p> + Try giving it a few lines of input text and then + end-of-file. Before it exits, you should see a list of the + lines you entered display with data items. This is a simple + check to make sure the fundamental configuration is working + correctly. + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/java_compat.html b/docs/programmer_reference/java_compat.html index e67c0154..46ead15d 100644 --- a/docs/programmer_reference/java_compat.html +++ b/docs/programmer_reference/java_compat.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="java.html">Prev</a> </td> - <th width="60%" align="center">Chapter 5. - Java API - </th> + <th width="60%" align="center">Chapter 5. Java API </th> <td width="20%" align="right"> <a accesskey="n" href="java_program.html">Next</a></td> </tr> </table> @@ -38,9 +36,12 @@ </div> </div> </div> - <p>The Berkeley DB Java API has been tested with the Sun Microsystem's JDK 1.5 -(Java 5) on Linux, Windows and OS X. It should work with any JDK -1.5- compatible environment.</p> + <p> + The Berkeley DB Java API has been tested and shown to work + with Java 5, 6, 7 on Linux and Windows. + The Berkeley DB Java API has been tested and shown to work + with Java 6 and 7 on Mac OS X. + </p> </div> <div class="navfooter"> <hr /> @@ -53,9 +54,7 @@ <td width="40%" align="right"> <a accesskey="n" href="java_program.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 5. - Java API - </td> + <td width="40%" align="left" valign="top">Chapter 5. Java API </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/java_faq.html b/docs/programmer_reference/java_faq.html index 6b3f98f7..ea8e78a5 100644 --- a/docs/programmer_reference/java_faq.html +++ b/docs/programmer_reference/java_faq.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="java_program.html">Prev</a> </td> - <th width="60%" align="center">Chapter 5. - Java API - </th> + <th width="60%" align="center">Chapter 5. Java API </th> <td width="20%" align="right"> <a accesskey="n" href="csharp.html">Next</a></td> </tr> </table> @@ -42,78 +40,110 @@ <ol type="1"> <li> <span class="bold"> - <strong>On what platforms is the Berkeley DB Java API supported?</strong> + <strong>On what platforms is the Berkeley DB + Java API supported?</strong> </span> - <p>All platforms supported by Berkeley DB that have a JVM compatible with J2SE -1.4 or above.</p> + <p>All platforms supported by Berkeley DB that have a + JVM compatible with J2SE 1.4 or above.</p> </li> <li> <span class="bold"> - <strong>How does the Berkeley DB Java API relate to the J2EE standard?</strong> + <strong>How does the Berkeley DB Java API + relate to the J2EE standard?</strong> </span> - <p>The Berkeley DB Java API does not currently implement any part of the J2EE -standard. That said, it does implement the implicit standard for Java -<a class="ulink" href="http://download.oracle.com/javase/1.5.0/docs/guide/collections/" target="_top">Java Collections</a>. The concept of a transaction exists in several -Java packages (J2EE, XA, JINI to name a few). Support for these APIs -will be added based on demand in future versions of Berkeley DB.</p> + <p> + The Berkeley DB Java API does not currently + implement any part of the J2EE standard. That said, it + does implement the implicit standard for Java <a class="ulink" href="http://download.oracle.com/javase/1.5.0/docs/guide/collections/" target="_top"> + Java Collections</a>. The concept of a + transaction exists in several Java packages (J2EE, XA, + JINI to name a few). Support for these APIs will be + added based on demand in future versions of Berkeley + DB. + </p> </li> <li> <span class="bold"> - <strong>How should I incorporate db.jar and the db native library into -a Tomcat or other J2EE application servers?</strong> + <strong>How should I incorporate db.jar and + the db native library into a Tomcat or other J2EE + application servers?</strong> </span> - <p>Tomcat and other J2EE application servers have the ability to rebuild -and reload code automatically. When using Tomcat this is the case when -"reloadable" is set to "true". If your WAR file includes the db.jar it -too will be reloaded each time your code is reloaded. This causes -exceptions as the native library can't be loaded more than once and -there is no way to unload native code. The solution is to place the -db.jar in $TOMCAT_HOME/common/lib and let Tomcat load that library once -at start time rather than putting it into the WAR that gets reloaded -over and over.</p> + <p> + Tomcat and other J2EE application servers have the + ability to rebuild and reload code automatically. When + using Tomcat this is the case when "reloadable" is set + to "true". If your WAR file includes the db.jar it too + will be reloaded each time your code is reloaded. This + causes exceptions as the native library can't be + loaded more than once and there is no way to unload + native code. The solution is to place the db.jar in + $TOMCAT_HOME/common/lib and let Tomcat load that + library once at start time rather than putting it into + the WAR that gets reloaded over and over. + </p> </li> <li> <span class="bold"> - <strong>Can I use the Berkeley DB Java API from within a EJB, a Servlet or a -JSP page?</strong> + <strong>Can I use the Berkeley DB Java API + from within a EJB, a Servlet or a JSP page?</strong> </span> - <p>Yes. The Berkeley DB Java API can be used from within all the popular J2EE -application servers in many different ways.</p> + <p> + Yes. The Berkeley DB Java API can be used from + within all the popular J2EE application servers in + many different ways. + </p> </li> <li> <span class="bold"> - <strong>During one of the first calls to the Berkeley DB Java API, a -DbException is thrown with a "Bad file number" or "Bad file descriptor" -message.</strong> + <strong>During one of the first calls to the + Berkeley DB Java API, a DbException is thrown with a + "Bad file number" or "Bad file descriptor" + message.</strong> </span> - <p>There are known large-file support bugs under JNI in various releases -of the JDK. Please upgrade to the latest release of the JDK, and, if -that does not solve the problem, disable big file support using the ---disable-largefile configuration option.</p> + <p> + There are known large-file support bugs under JNI in + various releases of the JDK. Please upgrade to the + latest release of the JDK, and, if that does not solve + the problem, disable big file support using the + --disable-largefile configuration option. + </p> </li> <li> <span class="bold"> - <strong>How can I use native methods from a debug build of the -Java library?</strong> + <strong>How can I use native methods from a + debug build of the Java library?</strong> </span> - <p>Set Java's library path so that the debug version of Berkeley DB's Java -library appears, but the release version does not. Berkeley DB tries to load -the release library first, and if that fails tries the debug library.</p> + <p> + Set Java's library path so that the debug version of + Berkeley DB's Java library appears, but the release + version does not. Berkeley DB tries to load the + release library first, and if that fails tries the + debug library. + </p> </li> <li> <span class="bold"> - <strong>Why is ClassNotFoundException thrown when adding a record to -the database, when a SerialBinding is used?</strong> + <strong>Why is ClassNotFoundException thrown + when adding a record to the database, when a + SerialBinding is used?</strong> </span> - <p>This problem occurs if you copy the db.jar file into the Java extensions -(ext) directory. This will cause the database code to run under the -System class loader, and it won't be able to find your application -classes.</p> - <p>You'll have to actually remove db.jar from the Java extension directory. -If you have more than one installation of Java, be sure to remove it -from all of them. This is necessary even if db.jar is specified in the -classpath.</p> - <p>An example of the exception is:</p> + <p> + This problem occurs if you copy the db.jar file into + the Java extensions (ext) directory. This will cause + the database code to run under the System class + loader, and it won't be able to find your application + classes. + </p> + <p> + You'll have to actually remove db.jar from the Java + extension directory. If you have more than one + installation of Java, be sure to remove it from all of + them. This is necessary even if db.jar is specified in + the classpath. + </p> + <p> + An example of the exception is: + </p> <pre class="programlisting">collections.ship.basic.SupplierKey at java.net.URLClassLoader$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) @@ -129,19 +159,26 @@ getClassInfo(StoredClassCatalog.java:211) </li> <li> <span class="bold"> - <strong>I'm upgrading my Java application to Berkeley DB 4.3. Can I use the -com.sleepycat.db.internal package rather than porting my code to the new -API?</strong> + <strong>I'm upgrading my Java application to + Berkeley DB 4.3. Can I use the + com.sleepycat.db.internal package rather than porting + my code to the new API?</strong> </span> - <p>While it is possible to use the low-level API from applications, there -are some caveats that should be considered when upgrading. The first is -that the internal API depends on some classes in the public API such as -DatabaseEntry.</p> - <p>In addition, the internal API is closer to the C API and doesn't have -some of the default settings that were part of the earlier Java API. -For example, applications will need to set the DB_THREAD flag explicitly -if handles are to be used from multiple threads, or subtle errors may -occur.</p> + <p> + While it is possible to use the low-level API from + applications, there are some caveats that should be + considered when upgrading. The first is that the + internal API depends on some classes in the public API + such as DatabaseEntry. + </p> + <p> + In addition, the internal API is closer to the C API + and doesn't have some of the default settings that + were part of the earlier Java API. For example, + applications will need to set the DB_THREAD flag + explicitly if handles are to be used from multiple + threads, or subtle errors may occur. + </p> </li> </ol> </div> diff --git a/docs/programmer_reference/java_program.html b/docs/programmer_reference/java_program.html index 3a191702..8a4a4308 100644 --- a/docs/programmer_reference/java_program.html +++ b/docs/programmer_reference/java_program.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="java_compat.html">Prev</a> </td> - <th width="60%" align="center">Chapter 5. - Java API - </th> + <th width="60%" align="center">Chapter 5. Java API </th> <td width="20%" align="right"> <a accesskey="n" href="java_faq.html">Next</a></td> </tr> </table> @@ -38,54 +36,89 @@ </div> </div> </div> - <p>Although the Java API parallels the Berkeley DB C++/C interface in many ways, -it differs where the Java language requires. For example, the handle -method names are camel-cased and conform to Java naming patterns. (The -C++/C method names are currently provided, but are deprecated.)</p> + <p> + Although the Java API parallels the Berkeley DB C++/C + interface in many ways, it differs where the Java language + requires. For example, the handle method names are camel-cased + and conform to Java naming patterns. (The C++/C method names + are currently provided, but are deprecated.) + </p> <div class="orderedlist"> <ol type="1"> - <li>The Java runtime does not automatically close Berkeley DB objects on -finalization. There are several reasons for this. One is that -finalization is generally run only when garbage collection occurs, and -there is no guarantee that this occurs at all, even on exit. Allowing -specific Berkeley DB actions to occur in ways that cannot be replicated seems -wrong. Second, finalization of objects may happen in an arbitrary -order, so we would have to do extra bookkeeping to make sure that -everything was closed in the proper order. The best word of advice is -to always do a close() for any matching open() call. Specifically, the -Berkeley DB package requires that you explicitly call close on each individual -<a class="ulink" href="../java/com/sleepycat/db/Database.html" target="_top">Database</a> and -<a class="ulink" href="../java/com/sleepycat/db/Cursor.html" target="_top">Cursor</a> object that you opened. Your database -activity may not be synchronized to disk unless you do so.</li> - <li>Some methods in the Java API have no return type, and throw a -<a class="ulink" href="../java/com/sleepycat/db/DatabaseException.html" target="_top">DatabaseException</a> when an severe error -arises. There are some notable methods that do have a return value, and -can also throw an exception. The "get" methods in -<a class="ulink" href="../java/com/sleepycat/db/Database.html" target="_top">Database</a> and -<a class="ulink" href="../java/com/sleepycat/db/Cursor.html" target="_top">Cursor</a> both return 0 when a get -succeeds, <a class="link" href="program_errorret.html#program_errorret.DB_NOTFOUND">DB_NOTFOUND</a> when the key is not found, and throw an -error when there is a severe error. This approach allows the programmer -to check for typical data-driven errors by watching return values -without special casing exceptions. -<p>An object of type <a class="ulink" href="../java/com/sleepycat/db/MemoryException.html" target="_top">MemoryException</a> is -thrown when a Dbt is too small to hold the corresponding key or data item.</p><p>An object of type <a class="ulink" href="../java/com/sleepycat/db/DeadlockException.html" target="_top">DeadlockException</a> is -thrown when a deadlock would occur.</p><p>An object of type <a class="ulink" href="../java/com/sleepycat/db/RunRecoveryException.html" target="_top">RunRecoveryException</a>, a -subclass of <a class="ulink" href="../java/com/sleepycat/db/DatabaseException.html" target="_top">DatabaseException</a>, is thrown when -there is an error that requires a recovery of the database using <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility.</p><p>An object of type <a class="ulink" href="http://download.oracle.com/javase/1.5.0/docs/api/java/lang/IllegalArgumentException.html" target="_top">IllegalArgumentException</a> -a standard Java Language exception, is thrown when there is an error in -method arguments.</p><p>An object of type <a class="ulink" href="http://download.oracle.com/javase/1.5.0/docs/api/java/lang/OutOfMemoryError.html" target="_top">OutOfMemoryError</a> is thrown -when the system cannot provide enough memory to complete the operation -(the ENOMEM system error on UNIX).</p></li> - <li>If there are embedded nulls in the <span class="bold"><strong>curslist</strong></span> argument for -<a class="ulink" href="../java/com/sleepycat/db/Database.html#join(com.sleepycat.db.Cursor[], com.sleepycat.db.JoinConfig)" target="_top">Database.join(com.sleepycat.db.Cursor[], com.sleepycat.db.JoinConfig)</a>, -they will be treated as the end of the list of cursors, even if you -may have allocated a longer array. Fill in all the cursors in your -array unless you intend to cut it short.</li> - <li>If you are using custom class loaders in your application, make sure -that the Berkeley DB classes are loaded by the system class loader, not a -custom class loader. This is due to a JVM bug that can cause an access -violation during finalization (see the bug 4238486 in Sun Microsystem's -Java Bug Database).</li> + <li> + The Java runtime does not automatically close + Berkeley DB objects on finalization. There are several + reasons for this. One is that finalization is generally + run only when garbage collection occurs, and there is no + guarantee that this occurs at all, even on exit. Allowing + specific Berkeley DB actions to occur in ways that cannot + be replicated seems wrong. Second, finalization of objects + may happen in an arbitrary order, so we would have to do + extra bookkeeping to make sure that everything was closed + in the proper order. The best word of advice is to always + do a close() for any matching open() call. Specifically, + the Berkeley DB package requires that you explicitly call + close on each individual <a class="ulink" href="../java/com/sleepycat/db/Database.html" target="_top">Database</a> and <a class="ulink" href="../java/com/sleepycat/db/Cursor.html" target="_top">Cursor</a> object + that you opened. Your database + activity may not be synchronized to disk unless you do + so. + </li> + <li> + Some methods in the Java API have no return type, + and throw a <a class="ulink" href="../java/com/sleepycat/db/DatabaseException.html" target="_top">DatabaseException</a> + when an severe error + arises. There are some notable methods that do have a + return value, and can also throw an exception. The "get" + methods in <a class="ulink" href="../java/com/sleepycat/db/Database.html" target="_top">Database</a> and <a class="ulink" href="../java/com/sleepycat/db/Cursor.html" target="_top">Cursor</a> both + return 0 when a get succeeds, + <a class="link" href="program_errorret.html#program_errorret.DB_NOTFOUND">DB_NOTFOUND</a> + when the key is not found, and throw an error when there is a severe + error. This approach allows the programmer to check for typical data-driven + errors by watching return values without special casing + exceptions. + <p> + An object of type <a class="ulink" href="../java/com/sleepycat/db/MemoryException.html" target="_top"> + MemoryException</a> is thrown when a Dbt is + too small to hold the corresponding key or data + item. + </p><p> + An object of type <a class="ulink" href="../java/com/sleepycat/db/DeadlockException.html" target="_top"> + DeadlockException</a> is thrown when a + deadlock would occur. + </p><p> + An object of type <a class="ulink" href="../java/com/sleepycat/db/RunRecoveryException.html" target="_top"> + RunRecoveryException</a>, a subclass of + <a class="ulink" href="../java/com/sleepycat/db/DatabaseException.html" target="_top"> + DatabaseException</a>, is thrown when there + is an error that requires a recovery of the database + using <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility. + </p><p> + An object of type <a class="ulink" href="http://download.oracle.com/javase/1.5.0/docs/api/java/lang/IllegalArgumentException.html" target="_top"> + IllegalArgumentException</a> a standard Java + Language exception, is thrown when there is an error + in method arguments.</p><p> + An object of type <a class="ulink" href="http://download.oracle.com/javase/1.5.0/docs/api/java/lang/OutOfMemoryError.html" target="_top"> + OutOfMemoryError</a> is thrown when the + system cannot provide enough memory to complete the + operation (the ENOMEM system error on + UNIX). + </p></li> + <li> + If there are embedded nulls in the <span class="bold"><strong>curslist</strong></span> argument for <a class="ulink" href="../java/com/sleepycat/db/Database.html#join(com.sleepycat.db.Cursor[], com.sleepycat.db.JoinConfig)" target="_top"> + Database.join(com.sleepycat.db.Cursor[], + com.sleepycat.db.JoinConfig)</a>, they will be + treated as the end of the list of cursors, even if you may + have allocated a longer array. Fill in all the cursors in + your array unless you intend to cut it short. + </li> + <li> + If you are using custom class loaders in your + application, make sure that the Berkeley DB classes are + loaded by the system class loader, not a custom class + loader. This is due to a JVM bug that can cause an access + violation during finalization (see the bug 4238486 in Sun + Microsystem's Java Bug Database). + </li> </ol> </div> </div> diff --git a/docs/programmer_reference/lock.html b/docs/programmer_reference/lock.html index a8e1b422..692326e6 100644 --- a/docs/programmer_reference/lock.html +++ b/docs/programmer_reference/lock.html @@ -14,13 +14,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 16. - The Locking Subsystem - </th> + <th colspan="3" align="center">Chapter 16. The Locking Subsystem </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="program_faq.html">Prev</a> </td> @@ -34,9 +32,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="lock"></a>Chapter 16. - The Locking Subsystem - </h2> + <h2 class="title"><a id="lock"></a>Chapter 16. The Locking Subsystem </h2> </div> </div> </div> @@ -47,7 +43,8 @@ <dl> <dt> <span class="sect1"> - <a href="lock.html#lock_intro">Introduction to the locking subsystem</a> + <a href="lock.html#lock_intro">Introduction to the locking + subsystem</a> </span> </dt> <dt> @@ -57,7 +54,8 @@ </dt> <dt> <span class="sect1"> - <a href="lock_max.html">Configuring locking: sizing the system</a> + <a href="lock_max.html">Configuring locking: sizing the + system</a> </span> </dt> <dt> @@ -72,7 +70,8 @@ </dt> <dt> <span class="sect1"> - <a href="lock_timeout.html">Deadlock detection using timers</a> + <a href="lock_timeout.html">Deadlock detection using + timers</a> </span> </dt> <dt> @@ -92,7 +91,8 @@ </dt> <dt> <span class="sect1"> - <a href="lock_twopl.html">Locking with transactions: two-phase locking</a> + <a href="lock_twopl.html">Locking with transactions: two-phase + locking</a> </span> </dt> <dt> @@ -102,12 +102,14 @@ </dt> <dt> <span class="sect1"> - <a href="lock_am_conv.html">Berkeley DB Transactional Data Store locking conventions</a> + <a href="lock_am_conv.html">Berkeley DB Transactional Data + Store locking conventions</a> </span> </dt> <dt> <span class="sect1"> - <a href="lock_nondb.html">Locking and non-Berkeley DB applications</a> + <a href="lock_nondb.html">Locking and non-Berkeley DB + applications</a> </span> </dt> </dl> @@ -116,77 +118,113 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="lock_intro"></a>Introduction to the locking subsystem</h2> + <h2 class="title" style="clear: both"><a id="lock_intro"></a>Introduction to the locking + subsystem</h2> </div> </div> </div> - <p>The locking subsystem provides interprocess and intraprocess concurrency -control mechanisms. Although the lock system is used extensively by -the Berkeley DB access methods and transaction system, it may also be used as -a standalone subsystem to provide concurrency control to any set of -designated resources.</p> - <p>The Lock subsystem is created, initialized, and opened by calls to -<a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> with the <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOCK" class="olink">DB_INIT_LOCK</a> or <a href="../api_reference/C/envopen.html#envopen_DB_INIT_CDB" class="olink">DB_INIT_CDB</a> -flags specified.</p> - <p>The <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> method is used to acquire and release locks. The -<a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> method performs any number of lock operations atomically. It -also provides the capability to release all locks held by a particular -locker and release all the locks on a particular object. (Performing -multiple lock operations atomically is useful in performing Btree -traversals -- you want to acquire a lock on a child page and once -acquired, immediately release the lock on its parent. This is -traditionally referred to as <span class="emphasis"><em>lock-coupling</em></span>). Two additional -methods, <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a> and <a href="../api_reference/C/lockput.html" class="olink">DB_ENV->lock_put()</a>, are provided. These -methods are simpler front-ends to the <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> functionality, -where <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a> acquires a lock, and <a href="../api_reference/C/lockput.html" class="olink">DB_ENV->lock_put()</a> releases a -lock that was acquired using <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a> or <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a>. All -locks explicitly requested by an application should be released via -calls to <a href="../api_reference/C/lockput.html" class="olink">DB_ENV->lock_put()</a> or <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a>. Using <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> -instead of separate calls to <a href="../api_reference/C/lockput.html" class="olink">DB_ENV->lock_put()</a> and <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a> also -reduces the synchronization overhead between multiple threads or -processes. The three methods are fully compatible, and may be used -interchangeably.</p> - <p>Applications must specify lockers and lock objects appropriately. When -used with the Berkeley DB access methods, lockers and objects are handled -completely internally, but an application using the lock manager -directly must either use the same conventions as the access methods or -define its own convention to which it adheres. If an application is -using the access methods with locking at the same time that it is -calling the lock manager directly, the application must follow a -convention that is compatible with the access methods' use of the -locking subsystem. See <a class="xref" href="lock_am_conv.html" title="Berkeley DB Transactional Data Store locking conventions">Berkeley DB Transactional Data Store locking conventions</a> for more information.</p> - <p>The <a href="../api_reference/C/lockid.html" class="olink">DB_ENV->lock_id()</a> function returns a unique ID that may safely be used -as the locker parameter to the <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> method. The access methods -use <a href="../api_reference/C/lockid.html" class="olink">DB_ENV->lock_id()</a> to generate unique lockers for the cursors -associated with a database.</p> - <p>The <a href="../api_reference/C/lockdetect.html" class="olink">DB_ENV->lock_detect()</a> function provides the programmatic interface to -the Berkeley DB deadlock detector. Whenever two threads of control issue lock -requests concurrently, the possibility for deadlock arises. A deadlock -occurs when two or more threads of control are blocked, waiting for -actions that another one of the blocked threads must take. For example, -assume that threads A and B have each obtained read locks on object X. -Now suppose that both threads want to obtain write locks on object X. -Neither thread can be granted its write lock (because of the other -thread's read lock). Both threads block and will never unblock because -the event for which they are waiting can never happen.</p> - <p>The deadlock detector examines all the locks held in the environment, -and identifies situations where no thread can make forward progress. -It then selects one of the participants in the deadlock (according to -the argument that was specified to <a href="../api_reference/C/envset_lk_detect.html" class="olink">DB_ENV->set_lk_detect()</a>), and -forces it to return the value <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a>, which indicates -that a deadlock occurred. The thread receiving such an error must -release all of its locks and undo any incomplete modifications to the -locked resource. Locks are typically released, and modifications -undone, by closing any cursors involved in the operation and aborting -any transaction enclosing the operation. The operation may optionally -be retried.</p> - <p>The <a href="../api_reference/C/lockstat.html" class="olink">DB_ENV->lock_stat()</a> function returns information about the status of -the lock subsystem. It is the programmatic interface used by the -<a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility.</p> - <p>The locking subsystem is closed by the call to <a href="../api_reference/C/envclose.html" class="olink">DB_ENV->close()</a>.</p> - <p>Finally, the entire locking subsystem may be discarded using the -<a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> method.</p> - <p>For more information on the locking subsystem methods, see the <a href="../api_reference/C/lock.html#locklist" class="olink">Locking Subsystem and Related Methods</a> section in the <em class="citetitle">Berkeley DB C API Reference Guide.</em> </p> + <p> + The locking subsystem provides interprocess and intraprocess + concurrency control mechanisms. Although the lock system is + used extensively by the Berkeley DB access methods and + transaction system, it may also be used as a standalone + subsystem to provide concurrency control to any set of + designated resources. + </p> + <p> + The Lock subsystem is created, initialized, and opened by + calls to <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> with the <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOCK" class="olink">DB_INIT_LOCK</a> or <a href="../api_reference/C/envopen.html#envopen_DB_INIT_CDB" class="olink">DB_INIT_CDB</a> + flags specified. + </p> + <p> + The <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> method is used to acquire and release locks. + The <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> method performs any number of lock operations + atomically. It also provides the capability to release all + locks held by a particular locker and release all the locks on + a particular object. (Performing multiple lock operations + atomically is useful in performing Btree traversals -- you + want to acquire a lock on a child page and once acquired, + immediately release the lock on its parent. This is + traditionally referred to as + <span class="emphasis"><em>lock-coupling</em></span>). Two additional + methods, <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a> and <a href="../api_reference/C/lockput.html" class="olink">DB_ENV->lock_put()</a>, are provided. These methods + are simpler front-ends to the <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> functionality, where + <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a> acquires a lock, and <a href="../api_reference/C/lockput.html" class="olink">DB_ENV->lock_put()</a> releases a lock that + was acquired using <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a> or <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a>. All locks + explicitly requested by an application should be released via + calls to <a href="../api_reference/C/lockput.html" class="olink">DB_ENV->lock_put()</a> or <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a>. Using <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> instead of + separate calls to <a href="../api_reference/C/lockput.html" class="olink">DB_ENV->lock_put()</a> and <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a> also reduces the + synchronization overhead between multiple threads or + processes. The three methods are fully compatible, and may be + used interchangeably. + </p> + <p> + Applications must specify lockers and lock objects + appropriately. When used with the Berkeley DB access methods, + lockers and objects are handled completely internally, but an + application using the lock manager directly must either use + the same conventions as the access methods or define its own + convention to which it adheres. If an application is using the + access methods with locking at the same time that it is + calling the lock manager directly, the application must follow + a convention that is compatible with the access methods' use + of the locking subsystem. See <a class="xref" href="lock_am_conv.html" title="Berkeley DB Transactional Data Store locking conventions">Berkeley DB Transactional Data + Store locking conventions</a> for more + information. + </p> + <p> + The <a href="../api_reference/C/lockid.html" class="olink">DB_ENV->lock_id()</a> function returns a unique ID that may safely be + used as the locker parameter to the <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> method. The + access methods use <a href="../api_reference/C/lockid.html" class="olink">DB_ENV->lock_id()</a> to generate unique lockers for the + cursors associated with a database. + </p> + <p> + The <a href="../api_reference/C/lockdetect.html" class="olink">DB_ENV->lock_detect()</a> function provides the programmatic + interface to the Berkeley DB deadlock detector. Whenever two + threads of control issue lock requests concurrently, the + possibility for deadlock arises. A deadlock occurs when two or + more threads of control are blocked, waiting for actions that + another one of the blocked threads must take. For example, + assume that threads A and B have each obtained read locks on + object X. Now suppose that both threads want to obtain write + locks on object X. Neither thread can be granted its write + lock (because of the other thread's read lock). Both threads + block and will never unblock because the event for which they + are waiting can never happen. + </p> + <p> + The deadlock detector examines all the locks held in the + environment, and identifies situations where no thread can + make forward progress. It then selects one of the participants + in the deadlock (according to the argument that was specified + to <a href="../api_reference/C/envset_lk_detect.html" class="olink">DB_ENV->set_lk_detect()</a>), and forces it to return the value + <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a>, + which indicates that a deadlock occurred. The thread receiving such an + error must release all of its locks and undo any incomplete modifications to the + locked resource. Locks are typically released, and + modifications undone, by closing any cursors involved in the + operation and aborting any transaction enclosing the + operation. The operation may optionally be retried. + </p> + <p> + The <a href="../api_reference/C/lockstat.html" class="olink">DB_ENV->lock_stat()</a> function returns information about the status + of the lock subsystem. It is the programmatic interface used + by the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility. + </p> + <p> + The locking subsystem is closed by the call to + <a href="../api_reference/C/envclose.html" class="olink">DB_ENV->close()</a>. + </p> + <p> + Finally, the entire locking subsystem may be discarded using + the <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> method. + </p> + <p> + For more information on the locking subsystem methods, see + the <a href="../api_reference/C/lock.html#locklist" class="olink">Locking + Subsystem and Related Methods</a> section in the + <em class="citetitle">Berkeley DB C API Reference Guide.</em> + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/lock_am_conv.html b/docs/programmer_reference/lock_am_conv.html index f61e4f78..69de8c99 100644 --- a/docs/programmer_reference/lock_am_conv.html +++ b/docs/programmer_reference/lock_am_conv.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Berkeley DB Transactional Data Store locking conventions</th> + <th colspan="3" align="center">Berkeley DB Transactional Data + Store locking conventions</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="lock_cam_conv.html">Prev</a> </td> - <th width="60%" align="center">Chapter 16. - The Locking Subsystem - </th> + <th width="60%" align="center">Chapter 16. The Locking Subsystem </th> <td width="20%" align="right"> <a accesskey="n" href="lock_nondb.html">Next</a></td> </tr> </table> @@ -34,111 +33,154 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="lock_am_conv"></a>Berkeley DB Transactional Data Store locking conventions</h2> + <h2 class="title" style="clear: both"><a id="lock_am_conv"></a>Berkeley DB Transactional Data + Store locking conventions</h2> </div> </div> </div> - <p>All Berkeley DB access methods follow the same conventions for locking -database objects. Applications that do their own locking and also do -locking via the access methods must be careful to adhere to these -conventions.</p> - <p>Whenever a Berkeley DB database is opened, the <a href="../api_reference/C/db.html" class="olink">DB</a> handle is assigned -a unique locker ID. Unless transactions are specified, that ID is used -as the locker for all calls that the Berkeley DB methods make to the lock -subsystem. In order to lock a file, pages in the file, or records in -the file, we must create a unique ID that can be used as the object to -be locked in calls to the lock manager. Under normal operation, that -object is a 28-byte value created by the concatenation of a unique file -identifier, a page or record number, and an object type (page or record).</p> - <p>In a transaction-protected environment, database create and delete -operations are recoverable and single-threaded. This single-threading -is achieved using a single lock for the entire environment that must be -acquired before beginning a create or delete operation. In this case, -the object on which Berkeley DB will lock is a 4-byte unsigned integer with -a value of 0.</p> - <p>If applications are using the lock subsystem directly while they are -also using locking via the access methods, they must take care not to -inadvertently lock objects that happen to be equal to the unique file -IDs used to lock files. This is most easily accomplished by using a -lock object with a length different from the values used by Berkeley DB.</p> - <p>All the access methods other than Queue use standard read/write locks -in a simple multiple-reader/single writer page-locking scheme. An -operation that returns data (for example, <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or -<a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a>) obtains a read lock on all the pages accessed while -locating the requested record. When an update operation is requested -(for example, <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> or <a href="../api_reference/C/dbcdel.html" class="olink">DBC->del()</a>), the page containing -the updated (or new) data is write-locked. As read-modify-write cycles -are quite common and are deadlock-prone under normal circumstances, the -Berkeley DB interfaces allow the application to specify the <a href="../api_reference/C/dbcget.html#dbcget_DB_RMW" class="olink">DB_RMW</a> -flag, which causes operations to immediately obtain a write lock, even -though they are only reading the data. Although this may reduce -concurrency somewhat, it reduces the probability of deadlock. In the -presence of transactions, page locks are held until transaction commit.</p> - <p>The Queue access method does not hold long-term page locks. Instead, -page locks are held only long enough to locate records or to change -metadata on a page, and record locks are held for the appropriate -duration. In the presence of transactions, record locks are held until -transaction commit. For <a href="../api_reference/C/db.html" class="olink">DB</a> operations, record locks are held until -operation completion; for <a href="../api_reference/C/dbc.html" class="olink">DBC</a> operations, record locks are held -until subsequent records are returned or the cursor is closed.</p> - <p>Under non-transaction operations, the access methods do not normally -hold locks across calls to the Berkeley DB interfaces. The one exception to -this rule is when cursors are used. Because cursors maintain a position -in a file, they must hold locks across calls; in fact, they will hold -a lock until the cursor is closed.</p> - <p>In this mode, the assignment of locker IDs to <a href="../api_reference/C/db.html" class="olink">DB</a> and cursor -handles is complicated. If the <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> option was specified -when the <a href="../api_reference/C/db.html" class="olink">DB</a> handle was opened, each use of <a href="../api_reference/C/db.html" class="olink">DB</a> has its -own unique locker ID, and each cursor is assigned its own unique locker -ID when it is created, so <a href="../api_reference/C/db.html" class="olink">DB</a> handle and cursor operations can -all conflict with one another. (This is because when Berkeley DB handles -may be shared by multiple threads of control the Berkeley DB library cannot -identify which operations are performed by which threads of control, -and it must ensure that two different threads of control are not -simultaneously modifying the same data structure. By assigning each -<a href="../api_reference/C/db.html" class="olink">DB</a> handle and cursor its own locker, two threads of control -sharing a handle cannot inadvertently interfere with each other.)</p> - <p>This has important implications. If a single thread of control opens -two cursors, uses a combination of cursor and non-cursor operations, or -begins two separate transactions, the operations are performed on behalf -of different lockers. Conflicts that arise between these different -lockers may not cause actual deadlocks, but can, in fact, permanently -block the thread of control. For example, assume that an application -creates a cursor and uses it to read record A. Now, assume a second -cursor is opened, and the application attempts to write record A using -the second cursor. Unfortunately, the first cursor has a read lock, so -the second cursor cannot obtain its write lock. However, that read lock -is held by the same thread of control, so the read lock can never be -released if we block waiting for the write lock. This might appear to -be a deadlock from the application's perspective, but Berkeley DB cannot -identify it as such because it has no knowledge of which lockers belong -to which threads of control. For this reason, application designers -are encouraged to close cursors as soon as they are done with them.</p> - <p>If the <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> option was not specified when the <a href="../api_reference/C/db.html" class="olink">DB</a> -handle was opened, all uses of the <a href="../api_reference/C/db.html" class="olink">DB</a> handle and all cursors -created using that handle will use the same locker ID for all -operations. In this case, if a single thread of control opens two -cursors or uses a combination of cursor and non-cursor operations, these -operations are performed on behalf of the same locker, and so cannot -deadlock or block the thread of control.</p> - <p>Complicated operations that require multiple cursors (or combinations -of cursor and non-cursor operations) can be performed in two ways. -First, they may be performed within a transaction, in which case all -operations lock on behalf of the designated transaction. Second, they -may be performed using a local <a href="../api_reference/C/db.html" class="olink">DB</a> handle, although, as -<a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> operations are relatively slow, this may not be a good -idea. Finally, the <a href="../api_reference/C/dbcdup.html" class="olink">DBC->dup()</a> function duplicates a cursor, using -the same locker ID as the originating cursor. There is no way to -achieve this duplication functionality through the <a href="../api_reference/C/db.html" class="olink">DB</a> handle -calls, but any <a href="../api_reference/C/db.html" class="olink">DB</a> call can be implemented by one or more calls -through a cursor.</p> - <p>When the access methods use transactions, many of these problems disappear. -The transaction ID is used as the locker ID for all operations performed -on behalf of the transaction. This means that the application may open -multiple cursors on behalf of the same transaction and these cursors will -all share a common locker ID. This is safe because transactions cannot -span threads of control, so the library knows that two cursors in the same -transaction cannot modify the database concurrently.</p> + <p> + All Berkeley DB access methods follow the same conventions + for locking database objects. Applications that do their own + locking and also do locking via the access methods must be + careful to adhere to these conventions. + </p> + <p> + Whenever a Berkeley DB database is opened, the <a href="../api_reference/C/db.html" class="olink">DB</a> handle + is assigned a unique locker ID. Unless transactions are + specified, that ID is used as the locker for all calls that + the Berkeley DB methods make to the lock subsystem. In order + to lock a file, pages in the file, or records in the file, we + must create a unique ID that can be used as the object to be + locked in calls to the lock manager. Under normal operation, + that object is a 28-byte value created by the concatenation of + a unique file identifier, a page or record number, and an + object type (page or record). + </p> + <p> + In a transaction-protected environment, database create and + delete operations are recoverable and single-threaded. This + single-threading is achieved using a single lock for the + entire environment that must be acquired before beginning a + create or delete operation. In this case, the object on which + Berkeley DB will lock is a 4-byte unsigned integer with a + value of 0. + </p> + <p> + If applications are using the lock subsystem directly while + they are also using locking via the access methods, they must + take care not to inadvertently lock objects that happen to be + equal to the unique file IDs used to lock files. This is most + easily accomplished by using a lock object with a length + different from the values used by Berkeley DB. + </p> + <p> + All the access methods other than Queue use standard + read/write locks in a simple multiple-reader/single writer + page-locking scheme. An operation that returns data (for + example, <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a>) obtains a read lock on all the + pages accessed while locating the requested record. When an + update operation is requested (for example, <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> or + <a href="../api_reference/C/dbcdel.html" class="olink">DBC->del()</a>), the page containing the updated (or new) data is + write-locked. As read-modify-write cycles are quite common and + are deadlock-prone under normal circumstances, the Berkeley DB + interfaces allow the application to specify the <a href="../api_reference/C/dbcget.html#dbcget_DB_RMW" class="olink">DB_RMW</a> flag, + which causes operations to immediately obtain a write lock, + even though they are only reading the data. Although this may + reduce concurrency somewhat, it reduces the probability of + deadlock. In the presence of transactions, page locks are held + until transaction commit. + </p> + <p> + The Queue access method does not hold long-term page locks. + Instead, page locks are held only long enough to locate + records or to change metadata on a page, and record locks are + held for the appropriate duration. In the presence of + transactions, record locks are held until transaction commit. + For <a href="../api_reference/C/db.html" class="olink">DB</a> operations, record locks are held until operation + completion; for <a href="../api_reference/C/dbc.html" class="olink">DBC</a> operations, record locks are held until + subsequent records are returned or the cursor is + closed. + </p> + <p> + Under non-transaction operations, the access methods do not + normally hold locks across calls to the Berkeley DB + interfaces. The one exception to this rule is when cursors are + used. Because cursors maintain a position in a file, they must + hold locks across calls; in fact, they will hold a lock until + the cursor is closed. + </p> + <p> + In this mode, the assignment of locker IDs to <a href="../api_reference/C/db.html" class="olink">DB</a> and + cursor handles is complicated. If the <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> option was + specified when the <a href="../api_reference/C/db.html" class="olink">DB</a> handle was opened, each use of <a href="../api_reference/C/db.html" class="olink">DB</a> + has its own unique locker ID, and each cursor is assigned its + own unique locker ID when it is created, so <a href="../api_reference/C/db.html" class="olink">DB</a> handle and + cursor operations can all conflict with one another. (This is + because when Berkeley DB handles may be shared by multiple + threads of control the Berkeley DB library cannot identify + which operations are performed by which threads of control, + and it must ensure that two different threads of control are + not simultaneously modifying the same data structure. By + assigning each <a href="../api_reference/C/db.html" class="olink">DB</a> handle and cursor its own locker, two + threads of control sharing a handle cannot inadvertently + interfere with each other.) + </p> + <p> + This has important implications. If a single thread of + control opens two cursors, uses a combination of cursor and + non-cursor operations, or begins two separate transactions, + the operations are performed on behalf of different lockers. + Conflicts that arise between these different lockers may not + cause actual deadlocks, but can, in fact, permanently block + the thread of control. For example, assume that an application + creates a cursor and uses it to read record A. Now, assume a + second cursor is opened, and the application attempts to write + record A using the second cursor. Unfortunately, the first + cursor has a read lock, so the second cursor cannot obtain its + write lock. However, that read lock is held by the same thread + of control, so the read lock can never be released if we block + waiting for the write lock. This might appear to be a deadlock + from the application's perspective, but Berkeley DB cannot + identify it as such because it has no knowledge of which + lockers belong to which threads of control. For this reason, + application designers are encouraged to close cursors as soon + as they are done with them. + </p> + <p> + If the <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> option was not specified when the <a href="../api_reference/C/db.html" class="olink">DB</a> + handle was opened, all uses of the <a href="../api_reference/C/db.html" class="olink">DB</a> handle and all cursors + created using that handle will use the same locker ID for all + operations. In this case, if a single thread of control opens + two cursors or uses a combination of cursor and non-cursor + operations, these operations are performed on behalf of the + same locker, and so cannot deadlock or block the thread of + control. + </p> + <p> + Complicated operations that require multiple cursors (or + combinations of cursor and non-cursor operations) can be + performed in two ways. First, they may be performed within a + transaction, in which case all operations lock on behalf of + the designated transaction. Second, they may be performed + using a local <a href="../api_reference/C/db.html" class="olink">DB</a> handle, although, as <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> operations + are relatively slow, this may not be a good idea. Finally, the + <a href="../api_reference/C/dbcdup.html" class="olink">DBC->dup()</a> function duplicates a cursor, using the same locker + ID as the originating cursor. There is no way to achieve this + duplication functionality through the <a href="../api_reference/C/db.html" class="olink">DB</a> handle calls, but + any <a href="../api_reference/C/db.html" class="olink">DB</a> call can be implemented by one or more calls through + a cursor. + </p> + <p> + When the access methods use transactions, many of these + problems disappear. The transaction ID is used as the locker + ID for all operations performed on behalf of the transaction. + This means that the application may open multiple cursors on + behalf of the same transaction and these cursors will all + share a common locker ID. This is safe because transactions + cannot span threads of control, so the library knows that two + cursors in the same transaction cannot modify the database + concurrently. + </p> </div> <div class="navfooter"> <hr /> @@ -155,7 +197,8 @@ transaction cannot modify the database concurrently.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Locking and non-Berkeley DB applications</td> + <td width="40%" align="right" valign="top"> Locking and non-Berkeley DB + applications</td> </tr> </table> </div> diff --git a/docs/programmer_reference/lock_cam_conv.html b/docs/programmer_reference/lock_cam_conv.html index a6a3c216..5ad78b9c 100644 --- a/docs/programmer_reference/lock_cam_conv.html +++ b/docs/programmer_reference/lock_cam_conv.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="lock_twopl.html">Prev</a> </td> - <th width="60%" align="center">Chapter 16. - The Locking Subsystem - </th> + <th width="60%" align="center">Chapter 16. The Locking Subsystem </th> <td width="20%" align="right"> <a accesskey="n" href="lock_am_conv.html">Next</a></td> </tr> </table> @@ -38,51 +36,76 @@ </div> </div> </div> - <p>The Berkeley DB Concurrent Data Store product has a simple set of conventions for locking. It -provides multiple-reader/single-writer semantics, but not per-page -locking or transaction recoverability. As such, it does its locking -entirely in the Berkeley DB interface layer.</p> - <p>The object it locks is the file, identified by its unique file number. -The locking matrix is not one of the two standard lock modes, instead, -we use a four-lock set, consisting of the following:</p> + <p> + The Berkeley DB Concurrent Data Store product has a simple + set of conventions for locking. It provides + multiple-reader/single-writer semantics, but not per-page + locking or transaction recoverability. As such, it does its + locking entirely in the Berkeley DB interface layer. + </p> + <p> + The object it locks is the file, identified by its unique + file number. The locking matrix is not one of the two standard + lock modes, instead, we use a four-lock set, consisting of the + following: + </p> <div class="variablelist"> <dl> <dt> <span class="term">DB_LOCK_NG</span> </dt> - <dd>not granted (always 0)</dd> + <dd> + not granted (always 0) + </dd> <dt> <span class="term">DB_LOCK_READ</span> </dt> - <dd>read (shared)</dd> + <dd> + read (shared) + </dd> <dt> <span class="term">DB_LOCK_WRITE</span> </dt> - <dd>write (exclusive)</dd> + <dd> + write (exclusive) + </dd> <dt> <span class="term">DB_LOCK_IWRITE</span> </dt> - <dd>intention-to-write (shared with NG and READ, but conflicts with WRITE and IWRITE)</dd> + <dd> + intention-to-write (shared with NG and READ, but + conflicts with WRITE and IWRITE) + </dd> </dl> </div> - <p>The IWRITE lock is used for cursors that will be used for updating -(IWRITE locks are implicitly obtained for write operations through the -Berkeley DB handles, for example, <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> or <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a>). While -the cursor is reading, the IWRITE lock is held; but as soon as the -cursor is about to modify the database, the IWRITE is upgraded to a -WRITE lock. This upgrade blocks until all readers have exited the -database. Because only one IWRITE lock is allowed at any one time, no -two cursors can ever try to upgrade to a WRITE lock at the same time, -and therefore deadlocks are prevented, which is essential because Berkeley DB Concurrent Data Store -does not include deadlock detection and recovery.</p> - <p>Applications that need to lock compatibly with Berkeley DB Concurrent Data Store must obey the -following rules:</p> + <p> + The IWRITE lock is used for cursors that will be used for + updating (IWRITE locks are implicitly obtained for write + operations through the Berkeley DB handles, for example, + <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> or <a href="../api_reference/C/dbdel.html" class="olink">DB->del()</a>). While the cursor is reading, the IWRITE + lock is held; but as soon as the cursor is about to modify the + database, the IWRITE is upgraded to a WRITE lock. This upgrade + blocks until all readers have exited the database. Because + only one IWRITE lock is allowed at any one time, no two + cursors can ever try to upgrade to a WRITE lock at the same + time, and therefore deadlocks are prevented, which is + essential because Berkeley DB Concurrent Data Store does not + include deadlock detection and recovery. + </p> + <p> + Applications that need to lock compatibly with Berkeley DB + Concurrent Data Store must obey the following rules: + </p> <div class="orderedlist"> <ol type="1"> - <li>Use only lock modes DB_LOCK_NG, DB_LOCK_READ, DB_LOCK_WRITE, -DB_LOCK_IWRITE.</li> - <li>Never attempt to acquire a WRITE lock on an object that is -already locked with a READ lock.</li> + <li> + Use only lock modes DB_LOCK_NG, DB_LOCK_READ, + DB_LOCK_WRITE, DB_LOCK_IWRITE. + </li> + <li> + Never attempt to acquire a WRITE lock on an object + that is already locked with a READ lock. + </li> </ol> </div> </div> @@ -97,11 +120,13 @@ already locked with a READ lock.</li> <td width="40%" align="right"> <a accesskey="n" href="lock_am_conv.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Locking with transactions: two-phase locking </td> + <td width="40%" align="left" valign="top">Locking with transactions: two-phase + locking </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Berkeley DB Transactional Data Store locking conventions</td> + <td width="40%" align="right" valign="top"> Berkeley DB Transactional Data + Store locking conventions</td> </tr> </table> </div> diff --git a/docs/programmer_reference/lock_config.html b/docs/programmer_reference/lock_config.html index 7659a481..d582e11d 100644 --- a/docs/programmer_reference/lock_config.html +++ b/docs/programmer_reference/lock_config.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="lock.html">Prev</a> </td> - <th width="60%" align="center">Chapter 16. - The Locking Subsystem - </th> + <th width="60%" align="center">Chapter 16. The Locking Subsystem </th> <td width="20%" align="right"> <a accesskey="n" href="lock_max.html">Next</a></td> </tr> </table> @@ -38,32 +36,44 @@ </div> </div> </div> - <p>The <a href="../api_reference/C/envset_lk_detect.html" class="olink">DB_ENV->set_lk_detect()</a> method specifies that the deadlock detector -should be run whenever a lock is about to block. This option provides -for rapid detection of deadlocks at the expense of potentially frequent -invocations of the deadlock detector. On a fast processor with a highly -contentious application where response time is critical, this is a good -choice. An option argument to the <a href="../api_reference/C/envset_lk_detect.html" class="olink">DB_ENV->set_lk_detect()</a> method -indicates which lock requests should be rejected.</p> - <p>The application can limit how long it blocks on a contested resource. -The <a href="../api_reference/C/envset_timeout.html" class="olink">DB_ENV->set_timeout()</a> method specifies the length of the timeout. -This value is checked whenever deadlock detection is performed, -so the accuracy of the timeout depends upon the frequency of -deadlock detection.</p> - <p>In general, when applications are not specifying lock and transaction -timeout values, the <a href="../api_reference/C/lockdetect.html#detect_DB_LOCK_DEFAULT" class="olink">DB_LOCK_DEFAULT</a> option is probably the -correct first choice, and other options should only be selected based -on evidence that they improve transaction throughput. If an application -has long-running transactions, <a href="../api_reference/C/lockdetect.html#detect_DB_LOCK_YOUNGEST" class="olink">DB_LOCK_YOUNGEST</a> will guarantee -that transactions eventually complete, but it may do so at the expense -of a large number of lock request rejections (and therefore, transaction -aborts).</p> - <p>The alternative to using the <a href="../api_reference/C/envset_lk_detect.html" class="olink">DB_ENV->set_lk_detect()</a> method is to -explicitly perform deadlock detection using the Berkeley DB -<a href="../api_reference/C/lockdetect.html" class="olink">DB_ENV->lock_detect()</a> method.</p> - <p>The <a href="../api_reference/C/envset_lk_conflicts.html" class="olink">DB_ENV->set_lk_conflicts()</a> method allows you to specify your own -locking conflicts matrix. This is an advanced configuration option, -and is almost never necessary.</p> + <p> + The <a href="../api_reference/C/envset_lk_detect.html" class="olink">DB_ENV->set_lk_detect()</a> method specifies that the deadlock + detector should be run whenever a lock is about to block. This + option provides for rapid detection of deadlocks at the + expense of potentially frequent invocations of the deadlock + detector. On a fast processor with a highly contentious + application where response time is critical, this is a good + choice. An option argument to the <a href="../api_reference/C/envset_lk_detect.html" class="olink">DB_ENV->set_lk_detect()</a> method + indicates which lock requests should be rejected. + </p> + <p> + The application can limit how long it blocks on a contested + resource. The <a href="../api_reference/C/envset_timeout.html" class="olink">DB_ENV->set_timeout()</a> method specifies the length of + the timeout. This value is checked whenever deadlock detection + is performed, so the accuracy of the timeout depends upon the + frequency of deadlock detection. + </p> + <p> + In general, when applications are not specifying lock and + transaction timeout values, the <a href="../api_reference/C/lockdetect.html#detect_DB_LOCK_DEFAULT" class="olink">DB_LOCK_DEFAULT</a> option is + probably the correct first choice, and other options should + only be selected based on evidence that they improve + transaction throughput. If an application has long-running + transactions, <a href="../api_reference/C/lockdetect.html#detect_DB_LOCK_YOUNGEST" class="olink">DB_LOCK_YOUNGEST</a> will guarantee that + transactions eventually complete, but it may do so at the + expense of a large number of lock request rejections (and + therefore, transaction aborts). + </p> + <p> + The alternative to using the <a href="../api_reference/C/envset_lk_detect.html" class="olink">DB_ENV->set_lk_detect()</a> method is to + explicitly perform deadlock detection using the Berkeley DB + <a href="../api_reference/C/lockdetect.html" class="olink">DB_ENV->lock_detect()</a> method. + </p> + <p> + The <a href="../api_reference/C/envset_lk_conflicts.html" class="olink">DB_ENV->set_lk_conflicts()</a> method allows you to specify your + own locking conflicts matrix. This is an advanced + configuration option, and is almost never necessary. + </p> </div> <div class="navfooter"> <hr /> @@ -76,13 +86,12 @@ and is almost never necessary.</p> <td width="40%" align="right"> <a accesskey="n" href="lock_max.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 16. - The Locking Subsystem - </td> + <td width="40%" align="left" valign="top">Chapter 16. The Locking Subsystem </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Configuring locking: sizing the system</td> + <td width="40%" align="right" valign="top"> Configuring locking: sizing the + system</td> </tr> </table> </div> diff --git a/docs/programmer_reference/lock_dead.html b/docs/programmer_reference/lock_dead.html index 7250f42f..b20fcd27 100644 --- a/docs/programmer_reference/lock_dead.html +++ b/docs/programmer_reference/lock_dead.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="lock_stdmode.html">Prev</a> </td> - <th width="60%" align="center">Chapter 16. - The Locking Subsystem - </th> + <th width="60%" align="center">Chapter 16. The Locking Subsystem </th> <td width="20%" align="right"> <a accesskey="n" href="lock_timeout.html">Next</a></td> </tr> </table> @@ -38,67 +36,85 @@ </div> </div> </div> - <p>Practically any application that uses locking may deadlock. The -exceptions to this rule are when all the threads of control accessing -the database are read-only or when the Berkeley DB Concurrent Data Store product is used; the -Berkeley DB Concurrent Data Store product guarantees deadlock-free operation at the expense of -reduced concurrency. While there are data access patterns that are -deadlock free (for example, an application doing nothing but overwriting -fixed-length records in an already existing database), they are -extremely rare.</p> - <p>When a deadlock exists in the system, all the threads of control -involved in the deadlock are, by definition, waiting on a lock. The -deadlock detector examines the state of the lock manager and identifies -a deadlock, and selects one of the lock requests to reject. (See -<a class="xref" href="lock_config.html" title="Configuring locking">Configuring locking</a> for a -discussion of how a participant is selected). The <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a> or -<a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> call for which the selected participant is waiting then -returns a <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a> error. When using the Berkeley DB access -methods, this error return is propagated back through the Berkeley DB database -handle method to the calling application.</p> - <p>The deadlock detector identifies deadlocks by looking for a cycle in -what is commonly referred to as its "waits-for" graph. More precisely, -the deadlock detector reads through the lock table, and reviews each -lock object currently locked. Each object has lockers that currently -hold locks on the object and possibly a list of lockers waiting for a -lock on the object. Each object's list of waiting lockers defines a -partial ordering. That is, for a particular object, every waiting -locker comes after every holding locker because that holding locker must -release its lock before the waiting locker can make forward progress. -Conceptually, after each object has been examined, the partial orderings -are topologically sorted. If this topological sort reveals any cycles, -the lockers forming the cycle are involved in a deadlock. One of the -lockers is selected for rejection.</p> - <p>It is possible that rejecting a single lock request involved in a -deadlock is not enough to allow other lockers to make forward progress. -Unfortunately, at the time a lock request is selected for rejection, -there is not enough information available to determine whether rejecting -that single lock request will allow forward progress or not. Because -most applications have few deadlocks, Berkeley DB takes the conservative -approach, rejecting as few requests as may be necessary to resolve the -existing deadlocks. In particular, for each unique cycle found in the -waits-for graph described in the previous paragraph, only one lock -request is selected for rejection. However, if there are multiple -cycles, one lock request from each cycle is selected for rejection. -Only after the enclosing transactions have received the lock request -rejection return and aborted their transactions can it be determined -whether it is necessary to reject additional lock requests in order to -allow forward progress.</p> - <p>The <a href="../api_reference/C/db_deadlock.html" class="olink">db_deadlock</a> utility performs deadlock detection by calling -the underlying Berkeley DB <a href="../api_reference/C/lockdetect.html" class="olink">DB_ENV->lock_detect()</a> method at regular intervals -(<a href="../api_reference/C/lockdetect.html" class="olink">DB_ENV->lock_detect()</a> runs a single iteration of the Berkeley DB deadlock -detector). Alternatively, applications can create their own deadlock -utility or thread by calling the <a href="../api_reference/C/lockdetect.html" class="olink">DB_ENV->lock_detect()</a> method directly, or by -using the <a href="../api_reference/C/envset_lk_detect.html" class="olink">DB_ENV->set_lk_detect()</a> method to configure Berkeley DB to -automatically run the deadlock detector whenever there is a conflict -over a lock. The tradeoffs between using the <a href="../api_reference/C/lockdetect.html" class="olink">DB_ENV->lock_detect()</a> and -<a href="../api_reference/C/envset_lk_detect.html" class="olink">DB_ENV->set_lk_detect()</a> methods is that automatic deadlock detection will -resolve deadlocks more quickly (because the deadlock detector runs -as soon as the lock request blocks), however, automatic deadlock -detection often runs the deadlock detector when there is no need for -it, and for applications with large numbers of locks and/or where many -operations block temporarily on locks but are soon able to proceed, -automatic detection can decrease performance.</p> + <p> + Practically any application that uses locking may deadlock. + The exceptions to this rule are when all the threads of + control accessing the database are read-only or when the + Berkeley DB Concurrent Data Store product is used; the + Berkeley DB Concurrent Data Store product guarantees + deadlock-free operation at the expense of reduced concurrency. + While there are data access patterns that are deadlock free + (for example, an application doing nothing but overwriting + fixed-length records in an already existing database), they + are extremely rare. + </p> + <p> + When a deadlock exists in the system, all the threads of + control involved in the deadlock are, by definition, waiting + on a lock. The deadlock detector examines the state of the + lock manager and identifies a deadlock, and selects one of the + lock requests to reject. (See <a class="xref" href="lock_config.html" title="Configuring locking">Configuring locking</a> for a discussion of how a + participant is selected). The <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a> or <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> call for + which the selected participant is waiting then returns a <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a> error. + When using the Berkeley DB access methods, this error return is propagated + back through the Berkeley DB database handle method to the calling + application. + </p> + <p> + The deadlock detector identifies deadlocks by looking for a + cycle in what is commonly referred to as its "waits-for" + graph. More precisely, the deadlock detector reads through the + lock table, and reviews each lock object currently locked. + Each object has lockers that currently hold locks on the + object and possibly a list of lockers waiting for a lock on + the object. Each object's list of waiting lockers defines a + partial ordering. That is, for a particular object, every + waiting locker comes after every holding locker because that + holding locker must release its lock before the waiting locker + can make forward progress. Conceptually, after each object has + been examined, the partial orderings are topologically sorted. + If this topological sort reveals any cycles, the lockers + forming the cycle are involved in a deadlock. One of the + lockers is selected for rejection. + </p> + <p> + It is possible that rejecting a single lock request involved + in a deadlock is not enough to allow other lockers to make + forward progress. Unfortunately, at the time a lock request is + selected for rejection, there is not enough information + available to determine whether rejecting that single lock + request will allow forward progress or not. Because most + applications have few deadlocks, Berkeley DB takes the + conservative approach, rejecting as few requests as may be + necessary to resolve the existing deadlocks. In particular, + for each unique cycle found in the waits-for graph described + in the previous paragraph, only one lock request is selected + for rejection. However, if there are multiple cycles, one lock + request from each cycle is selected for rejection. Only after + the enclosing transactions have received the lock request + rejection return and aborted their transactions can it be + determined whether it is necessary to reject additional lock + requests in order to allow forward progress. + </p> + <p> + The <a href="../api_reference/C/db_deadlock.html" class="olink">db_deadlock</a> utility performs deadlock detection by calling the + underlying Berkeley DB <a href="../api_reference/C/lockdetect.html" class="olink">DB_ENV->lock_detect()</a> method at regular + intervals (<a href="../api_reference/C/lockdetect.html" class="olink">DB_ENV->lock_detect()</a> runs a single iteration of the + Berkeley DB deadlock detector). Alternatively, applications + can create their own deadlock utility or thread by calling the + <a href="../api_reference/C/lockdetect.html" class="olink">DB_ENV->lock_detect()</a> method directly, or by using the + <a href="../api_reference/C/envset_lk_detect.html" class="olink">DB_ENV->set_lk_detect()</a> method to configure Berkeley DB to + automatically run the deadlock detector whenever there is a + conflict over a lock. The tradeoffs between using the + <a href="../api_reference/C/lockdetect.html" class="olink">DB_ENV->lock_detect()</a> and <a href="../api_reference/C/envset_lk_detect.html" class="olink">DB_ENV->set_lk_detect()</a> methods is that automatic + deadlock detection will resolve deadlocks more quickly + (because the deadlock detector runs as soon as the lock + request blocks), however, automatic deadlock detection often + runs the deadlock detector when there is no need for it, and + for applications with large numbers of locks and/or where many + operations block temporarily on locks but are soon able to + proceed, automatic detection can decrease performance. + </p> </div> <div class="navfooter"> <hr /> @@ -115,7 +131,8 @@ automatic detection can decrease performance.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Deadlock detection using timers</td> + <td width="40%" align="right" valign="top"> Deadlock detection using + timers</td> </tr> </table> </div> diff --git a/docs/programmer_reference/lock_deaddbg.html b/docs/programmer_reference/lock_deaddbg.html index 655a2caa..93a20f17 100644 --- a/docs/programmer_reference/lock_deaddbg.html +++ b/docs/programmer_reference/lock_deaddbg.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="lock_timeout.html">Prev</a> </td> - <th width="60%" align="center">Chapter 16. - The Locking Subsystem - </th> + <th width="60%" align="center">Chapter 16. The Locking Subsystem </th> <td width="20%" align="right"> <a accesskey="n" href="lock_page.html">Next</a></td> </tr> </table> @@ -38,79 +36,109 @@ </div> </div> </div> - <p>An occasional debugging problem in Berkeley DB applications is unresolvable -deadlock. The output of the <span class="bold"><strong>-Co</strong></span> flags of the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility can be used to detect and debug these problems. The following -is a typical example of the output of this utility:</p> + <p> + An occasional debugging problem in Berkeley DB applications + is unresolvable deadlock. The output of the <span class="bold"><strong>-Co</strong></span> flags of the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility can be + used to detect and debug these problems. The following is a + typical example of the output of this utility: + </p> <pre class="programlisting">Locks grouped by object Locker Mode Count Status ----------- Object ---------- 1 READ 1 HELD a.db handle 0 80000004 WRITE 1 HELD a.db page 3</pre> - <p>In this example, we have opened a database and stored a single key/data -pair in it. Because we have a database handle open, we have a read lock -on that database handle. The database handle lock is the read lock -labeled <span class="emphasis"><em>handle</em></span>. (We can normally ignore handle locks for -the purposes of database debugging, as they will only conflict with -other handle operations, for example, an attempt to remove the database -will block because we are holding the handle locked, but reading and -writing the database will not conflict with the handle lock.)</p> - <p>It is important to note that locker IDs are 32-bit unsigned integers, -and are divided into two name spaces. Locker IDs with the high bit set -(that is, values 80000000 or higher), are locker IDs associated with -transactions. Locker IDs without the high bit set are locker IDs that -are not associated with a transaction. Locker IDs associated with -transactions map one-to-one with the transaction, that is, a transaction -never has more than a single locker ID, and all of the locks acquired -by the transaction will be acquired on behalf of the same locker ID.</p> - <p>We also hold a write lock on the database page where we stored the new -key/data pair. The page lock is labeled <span class="emphasis"><em>page</em></span> and is on page -number 3. If we were to put an additional key/data pair in the -database, we would see the following output:</p> + <p> + In this example, we have opened a database and stored a + single key/data pair in it. Because we have a database handle + open, we have a read lock on that database handle. The + database handle lock is the read lock labeled + <span class="emphasis"><em>handle</em></span>. (We can normally ignore + handle locks for the purposes of database debugging, as they + will only conflict with other handle operations, for example, + an attempt to remove the database will block because we are + holding the handle locked, but reading and writing the + database will not conflict with the handle lock.) + </p> + <p> + It is important to note that locker IDs are 32-bit unsigned + integers, and are divided into two name spaces. Locker IDs + with the high bit set (that is, values 80000000 or higher), + are locker IDs associated with transactions. Locker IDs + without the high bit set are locker IDs that are not + associated with a transaction. Locker IDs associated with + transactions map one-to-one with the transaction, that is, a + transaction never has more than a single locker ID, and all of + the locks acquired by the transaction will be acquired on + behalf of the same locker ID. + </p> + <p> + We also hold a write lock on the database page where we + stored the new key/data pair. The page lock is labeled + <span class="emphasis"><em>page</em></span> and is on page number 3. If we + were to put an additional key/data pair in the database, we + would see the following output: + </p> <pre class="programlisting">Locks grouped by object Locker Mode Count Status ----------- Object ---------- 80000004 WRITE 2 HELD a.db page 3 1 READ 1 HELD a.db handle 0</pre> - <p>That is, we have acquired a second reference count to page number 3, but -have not acquired any new locks. If we add an entry to a different page -in the database, we would acquire additional locks:</p> + <p> + That is, we have acquired a second reference count to page + number 3, but have not acquired any new locks. If we add an + entry to a different page in the database, we would acquire + additional locks: + </p> <pre class="programlisting">Locks grouped by object Locker Mode Count Status ----------- Object ---------- 1 READ 1 HELD a.db handle 0 80000004 WRITE 2 HELD a.db page 3 80000004 WRITE 1 HELD a.db page 2</pre> - <p>Here's a simple example of one lock blocking another one:</p> + <p> + Here's a simple example of one lock blocking another + one: + </p> <pre class="programlisting">Locks grouped by object Locker Mode Count Status ----------- Object ---------- 80000004 WRITE 1 HELD a.db page 2 80000005 WRITE 1 WAIT a.db page 2 1 READ 1 HELD a.db handle 0 80000004 READ 1 HELD a.db page 1</pre> - <p>In this example, there are two different transactional lockers (80000004 and -80000005). Locker 80000004 is holding a write lock on page 2, and -locker 80000005 is waiting for a write lock on page 2. This is not a -deadlock, because locker 80000004 is not blocked on anything. -Presumably, the thread of control using locker 80000004 will proceed, -eventually release its write lock on page 2, at which point the thread -of control using locker 80000005 can also proceed, acquiring a write -lock on page 2.</p> - <p>If lockers 80000004 and 80000005 are not in different threads of -control, the result would be <span class="emphasis"><em>self deadlock</em></span>. Self deadlock -is not a true deadlock, and won't be detected by the Berkeley DB deadlock -detector. It's not a true deadlock because, if work could continue to -be done on behalf of locker 80000004, then the lock would eventually be -released, and locker 80000005 could acquire the lock and itself proceed. -So, the key element is that the thread of control holding the lock -cannot proceed because it is the same thread as is blocked waiting on the -lock.</p> - <p>Here's an example of three transactions reaching true deadlock. First, -three different threads of control opened the database, acquiring three -database handle read locks.</p> + <p> + In this example, there are two different transactional + lockers (80000004 and 80000005). Locker 80000004 is holding a + write lock on page 2, and locker 80000005 is waiting for a + write lock on page 2. This is not a deadlock, because locker + 80000004 is not blocked on anything. Presumably, the thread of + control using locker 80000004 will proceed, eventually release + its write lock on page 2, at which point the thread of control + using locker 80000005 can also proceed, acquiring a write lock + on page 2. + </p> + <p> + If lockers 80000004 and 80000005 are not in different + threads of control, the result would be <span class="emphasis"><em>self + deadlock</em></span>. Self deadlock is not a true deadlock, + and won't be detected by the Berkeley DB deadlock detector. + It's not a true deadlock because, if work could continue to be + done on behalf of locker 80000004, then the lock would + eventually be released, and locker 80000005 could acquire the + lock and itself proceed. So, the key element is that the + thread of control holding the lock cannot proceed because it + is the same thread as is blocked waiting on the lock. + </p> + <p> + Here's an example of three transactions reaching true + deadlock. First, three different threads of control opened the + database, acquiring three database handle read locks. + </p> <pre class="programlisting">Locks grouped by object Locker Mode Count Status ----------- Object ---------- 1 READ 1 HELD a.db handle 0 3 READ 1 HELD a.db handle 0 5 READ 1 HELD a.db handle 0</pre> - <p>The three threads then each began a transaction, and put a key/data pair -on a different page:</p> + <p> + The three threads then each began a transaction, and put a + key/data pair on a different page: + </p> <pre class="programlisting">Locks grouped by object Locker Mode Count Status ----------- Object ---------- 80000008 WRITE 1 HELD a.db page 4 @@ -122,15 +150,21 @@ Locker Mode Count Status ----------- Object ---------- 80000008 READ 1 HELD a.db page 1 80000006 WRITE 1 HELD a.db page 2 80000007 WRITE 1 HELD a.db page 3</pre> - <p>The thread using locker 80000006 put a new key/data pair on page 2, the -thread using locker 80000007, on page 3, and the thread using locker -80000008 on page 4. Because the database is a 2-level Btree, the tree -was searched, and so each transaction acquired a read lock on the Btree -root page (page 1) as part of this operation.</p> - <p>The three threads then each attempted to put a second key/data pair on -a page currently locked by another thread. The thread using locker -80000006 tried to put a key/data pair on page 3, the thread using locker -80000007 on page 4, and the thread using locker 80000008 on page 2:</p> + <p> + The thread using locker 80000006 put a new key/data pair on + page 2, the thread using locker 80000007, on page 3, and the + thread using locker 80000008 on page 4. Because the database + is a 2-level Btree, the tree was searched, and so each + transaction acquired a read lock on the Btree root page (page + 1) as part of this operation. + </p> + <p> + The three threads then each attempted to put a second + key/data pair on a page currently locked by another thread. + The thread using locker 80000006 tried to put a key/data pair + on page 3, the thread using locker 80000007 on page 4, and the + thread using locker 80000008 on page 2: + </p> <pre class="programlisting">Locks grouped by object Locker Mode Count Status ----------- Object ---------- 80000008 WRITE 1 HELD a.db page 4 @@ -145,17 +179,17 @@ Locker Mode Count Status ----------- Object ---------- 80000008 WRITE 1 WAIT a.db page 2 80000007 WRITE 1 HELD a.db page 3 80000006 WRITE 1 WAIT a.db page 3</pre> - <p>Now, each of the threads of control is blocked, waiting on a different -thread of control. -The thread using locker 80000007 is blocked by -the thread using locker 80000008, due to the lock on page 4. -The thread using locker 80000008 is blocked by -the thread using locker 80000006, due to the lock on page 2. -And the thread using locker 80000006 is blocked by -the thread using locker 80000007, due to the lock on page 3. -Since none of the threads of control can make -progress, one of them will have to be killed in order to resolve the -deadlock.</p> + <p> + Now, each of the threads of control is blocked, waiting on a + different thread of control. The thread using locker 80000007 + is blocked by the thread using locker 80000008, due to the + lock on page 4. The thread using locker 80000008 is blocked by + the thread using locker 80000006, due to the lock on page 2. + And the thread using locker 80000006 is blocked by the thread + using locker 80000007, due to the lock on page 3. Since none + of the threads of control can make progress, one of them will + have to be killed in order to resolve the deadlock. + </p> </div> <div class="navfooter"> <hr /> @@ -168,7 +202,8 @@ deadlock.</p> <td width="40%" align="right"> <a accesskey="n" href="lock_page.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Deadlock detection using timers </td> + <td width="40%" align="left" valign="top">Deadlock detection using + timers </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/lock_max.html b/docs/programmer_reference/lock_max.html index 85f094cc..507fe270 100644 --- a/docs/programmer_reference/lock_max.html +++ b/docs/programmer_reference/lock_max.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Configuring locking: sizing the system</th> + <th colspan="3" align="center">Configuring locking: sizing the + system</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="lock_config.html">Prev</a> </td> - <th width="60%" align="center">Chapter 16. - The Locking Subsystem - </th> + <th width="60%" align="center">Chapter 16. The Locking Subsystem </th> <td width="20%" align="right"> <a accesskey="n" href="lock_stdmode.html">Next</a></td> </tr> </table> @@ -34,133 +33,143 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="lock_max"></a>Configuring locking: sizing the system</h2> + <h2 class="title" style="clear: both"><a id="lock_max"></a>Configuring locking: sizing the + system</h2> </div> </div> </div> <p> - The amount of memory available to the locking system is specified - using the <a href="../api_reference/C/envset_memory_max.html" class="olink">DB_ENV->set_memory_max()</a> method. Sizing of the enviroment - using the <a href="../api_reference/C/envset_memory_max.html" class="olink">DB_ENV->set_memory_max()</a> method is discussed in - <a class="xref" href="env_size.html" title="Sizing a database environment">Sizing a database environment</a>. Here we will - discuss how to estimate the number of objects your application is - likely to lock. Since running out of memory for locking structures - is a fatal error requiring reconfiguration and restarting the - environment it is best to overestimate the numbers. + The amount of memory available to the locking system is + specified using the <a href="../api_reference/C/envset_memory_max.html" class="olink">DB_ENV->set_memory_max()</a> method. Sizing of the + enviroment using the <a href="../api_reference/C/envset_memory_max.html" class="olink">DB_ENV->set_memory_max()</a> method is discussed + in <a class="xref" href="env_size.html" title="Sizing a database environment">Sizing a database environment</a>. Here + we will discuss how to estimate the number of objects your + application is likely to lock. Since running out of memory for + locking structures is a fatal error requiring reconfiguration + and restarting the environment it is best to overestimate the + numbers. </p> - <p> - When configuring a Berkeley DB Concurrent Data Store application, - the number of lock objects needed is two per open database (one for - the database lock, and one for the cursor lock when the - <a href="../api_reference/C/envset_flags.html#set_flags_DB_CDB_ALLDB" class="olink">DB_CDB_ALLDB</a> option is not specified). The number of locks - needed is one per open database handle plus one per simultaneous - cursor or non-cursor operation. + <p> + When configuring a Berkeley DB Concurrent Data Store + application, the number of lock objects needed is two per open + database (one for the database lock, and one for the cursor + lock when the <a href="../api_reference/C/envset_flags.html#set_flags_DB_CDB_ALLDB" class="olink">DB_CDB_ALLDB</a> option is not specified). The + number of locks needed is one per open database handle plus + one per simultaneous cursor or non-cursor operation. </p> - <p> - Configuring a Berkeley DB Transactional Data Store application is - more complicated. The recommended algorithm for selecting the - number of locks, lockers, and lock objects is to run the - application under stressful conditions and then review the lock - system's statistics to determine the number of locks, - lockers, and lock objects that were used. Then, double these - values for safety. However, in some large applications, finer - granularity of control is necessary in order to minimize the size - of the Lock subsystem. + <p> + Configuring a Berkeley DB Transactional Data Store + application is more complicated. The recommended algorithm for + selecting the number of locks, lockers, and lock objects is to + run the application under stressful conditions and then review + the lock system's statistics to determine the number of locks, + lockers, and lock objects that were used. Then, double these + values for safety. However, in some large applications, finer + granularity of control is necessary in order to minimize the + size of the Lock subsystem. </p> - <p> - The number of lockers can be estimated as follows: + <p> + The number of lockers can be estimated as follows: </p> <div class="itemizedlist"> <ul type="disc"> - <li> - If the database environment is using transactions, the - number of lockers can be estimated by adding the number of - simultaneously active non-transactional cursors and open database - handles to the number of simultaneously active transactions and - child transactions (where a child transaction is active until - it commits or aborts, not until its parent commits or aborts). - </li> - <li> - If the database environment is not using transactions, the - number of lockers can be estimated by adding the number - of simultaneously active non-transactional cursors and open - database handles to the number of simultaneous non-cursor - operations. + <li> + If the database environment is using transactions, + the number of lockers can be estimated by adding the + number of simultaneously active non-transactional cursors + and open database handles to the number of simultaneously + active transactions and child transactions (where a child + transaction is active until it commits or aborts, not + until its parent commits or aborts). + </li> + <li> + If the database environment is not using + transactions, the number of lockers can be estimated by + adding the number of simultaneously active + non-transactional cursors and open database handles to the + number of simultaneous non-cursor operations. </li> </ul> </div> - <p> - The number of lock objects needed for a transaction - can be estimated as follows: + <p> + The number of lock objects needed for a transaction can be + estimated as follows: </p> <div class="itemizedlist"> <ul type="disc"> <li> - <p> - For each access to a non-Queue database, one lock object is - needed for each page that is read or updated. + <p> + For each access to a non-Queue database, one lock + object is needed for each page that is read or + updated. </p> </li> <li> <p> - For the Queue access method you will need one lock object per - record that is read or updated. Deleted records skipped by a - DB_NEXT or DB_PREV operation do not require a separate lock - object. + For the Queue access method you will need one lock + object per record that is read or updated. Deleted + records skipped by a DB_NEXT or DB_PREV operation do + not require a separate lock object. </p> </li> <li> <p> - For Btree and Recno databases additional lock objects may be - needed for each node in the btree that has to be split due to - an update. + For Btree and Recno databases additional lock + objects may be needed for each node in the btree that + has to be split due to an update. </p> </li> <li> <p> - For Hash and Queue databases, every access must obtain a lock on - the metadata page for the duration of the access. This is not - held to the end of the transaction. + For Hash and Queue databases, every access must + obtain a lock on the metadata page for the duration of + the access. This is not held to the end of the + transaction. </p> </li> <li> - <p> - If the transaction performs an update that needs to allocate a page - to the database then a lock object for the metadata page will - be needed to the end of the transaction. + <p> + If the transaction performs an update that needs to + allocate a page to the database then a lock object for + the metadata page will be needed to the end of the + transaction. </p> </li> </ul> </div> <p> - Note that transactions accumulate locks over the transaction lifetime, - and the lock objects required by a single transaction is the total lock - objects required by all of the database operations in the transaction. - However, a database page (or record, in the case of the Queue access - method), that is accessed multiple times within a transaction only - requires a single lock object for the entire transaction. So if a - transaction in your application typically accesses 10 records, that - transaction will require about 10 lock objects (it may be a few more if - it splits btree nodes). If you have up to 10 concurrent threads in your - application, then you need to configure your system to have about 100 - lock objects. It is always better to configure more than you need so - that you don't run out of lock objects. The memory overhead of - over-allocating lock objects is minimal as they are small structures. + Note that transactions accumulate locks over the + transaction lifetime, and the lock objects required by a + single transaction is the total lock objects required by all + of the database operations in the transaction. However, a + database page (or record, in the case of the Queue access + method), that is accessed multiple times within a transaction + only requires a single lock object for the entire transaction. + So if a transaction in your application typically accesses 10 + records, that transaction will require about 10 lock objects + (it may be a few more if it splits btree nodes). If you have + up to 10 concurrent threads in your application, then you need + to configure your system to have about 100 lock objects. It is + always better to configure more than you need so that you + don't run out of lock objects. The memory overhead of + over-allocating lock objects is minimal as they are small + structures. </p> <p> - The number of locks required by an application cannot be easily - estimated. It is possible to calculate a number of locks by - multiplying the number of lockers, times the number of - lock objects, times two (two for the two possible lock modes for each - object, read and write). However, this is a pessimal value, and real - applications are unlikely to actually need that many locks. Reviewing - the Lock subsystem statistics is the best way to determine this value. + The number of locks required by an application cannot be + easily estimated. It is possible to calculate a number of + locks by multiplying the number of lockers, times the number + of lock objects, times two (two for the two possible lock + modes for each object, read and write). However, this is a + pessimal value, and real applications are unlikely to actually + need that many locks. Reviewing the Lock subsystem statistics + is the best way to determine this value. </p> <p> - By default a minimum number of locking objects are allocated at - startup. To avoid contention due to allocation the application may - use the <a href="../api_reference/C/envset_memory_init.html" class="olink">DB_ENV->set_memory_init()</a> method to preallocate and initialize - the following lock structures: + By default a minimum number of locking objects are + allocated at startup. To avoid contention due to allocation + the application may use the <a href="../api_reference/C/envset_memory_init.html" class="olink">DB_ENV->set_memory_init()</a> method to + preallocate and initialize the following lock structures: </p> <div class="itemizedlist"> <ul type="disc"> @@ -168,52 +177,54 @@ <p> <code class="literal">DB_MEM_LOCK</code> </p> - <p> + <p> Specifies the number of locks that can be - simultaneously requested in the system. + simultaneously requested in the system. </p> </li> <li> <p> <code class="literal">DB_MEM_LOCKER</code> </p> - <p> + <p> Specifies the number of lockers that can - simultaneously request locks in the system. + simultaneously request locks in the system. </p> </li> <li> <p> <code class="literal">DB_MEM_LOCKOBJECTS</code> </p> - <p> + <p> Specifies the number of objects that can - simultaneously be locked in the system. + simultaneously be locked in the system. </p> </li> </ul> </div> <p> - In addition to the above structures, sizing your locking subsystem - also requires specifying the number of lock table partitions. You - do this using the <a href="../api_reference/C/envset_lk_partitions.html" class="olink">DB_ENV->set_lk_partitions()</a> method. Each partition - may be accessed independently by a thread. More partitions can lead - to higher levels of concurrency. The default is to set the number - of partitions to be 10 times the number of cpus that the operating - system reports at the time the environment is created. Having more - than one partition when there is only one cpu is not beneficial - because the locking system is more efficient when there is a single - partition. Some operating systems (Linux, Solaris) may report - thread contexts as cpus, and so it may be necessary to override the - default to force a single partition on a single hyperthreaded cpu - system. Objects and locks are divided among the partitions so it is - best to allocate several locks and objects per partition. The - system will force there to be at least one per partition. If a - partition runs out of locks or objects it will steal what is needed - from the other partitions. This operation could impact performance - if it occurs too often. The final values specified for the locks - and lock objects should be more than or equal to the number of lock - table partitions. + In addition to the above structures, sizing your locking + subsystem also requires specifying the number of lock table + partitions. You do this using the <a href="../api_reference/C/envset_lk_partitions.html" class="olink">DB_ENV->set_lk_partitions()</a> + method. Each partition may be accessed independently by a + thread. More partitions can lead to higher levels of + concurrency. The default is to set the number of partitions to + be 10 times the number of cpus that the operating system + reports at the time the environment is created. Having more + than one partition when there is only one cpu is not + beneficial because the locking system is more efficient when + there is a single partition. Some operating systems (Linux, + Solaris) may report thread contexts as cpus, and so it may be + necessary to override the default to force a single partition + on a single hyperthreaded cpu system. Objects and locks are + divided among the partitions so it is best to allocate several + locks and objects per partition. The system will force there + to be at least one per partition. If a partition runs out of + locks or objects it will steal what is needed from the other + partitions. This operation could impact performance if it + occurs too often. The final values specified for the locks and + lock objects should be more than or equal to the number of + lock table partitions. </p> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/lock_nondb.html b/docs/programmer_reference/lock_nondb.html index 066666a4..7c242f70 100644 --- a/docs/programmer_reference/lock_nondb.html +++ b/docs/programmer_reference/lock_nondb.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Locking and non-Berkeley DB applications</th> + <th colspan="3" align="center">Locking and non-Berkeley DB + applications</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="lock_am_conv.html">Prev</a> </td> - <th width="60%" align="center">Chapter 16. - The Locking Subsystem - </th> + <th width="60%" align="center">Chapter 16. The Locking Subsystem </th> <td width="20%" align="right"> <a accesskey="n" href="log.html">Next</a></td> </tr> </table> @@ -34,38 +33,50 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="lock_nondb"></a>Locking and non-Berkeley DB applications</h2> + <h2 class="title" style="clear: both"><a id="lock_nondb"></a>Locking and non-Berkeley DB + applications</h2> </div> </div> </div> - <p>The Lock subsystem is useful outside the context of Berkeley DB. It can be -used to manage concurrent access to any collection of either ephemeral -or persistent objects. That is, the lock region can persist across -invocations of an application, so it can be used to provide long-term -locking (for example, conference room scheduling).</p> - <p>In order to use the locking subsystem in such a general way, the -applications must adhere to a convention for identifying objects and -lockers. Consider a conference room scheduling problem, in which there -are three conference rooms scheduled in half-hour intervals. The -scheduling application must then select a way to identify each -conference room/time slot combination. In this case, we could describe -the objects being locked as bytestrings consisting of the conference -room name, the date when it is needed, and the beginning of the -appropriate half-hour slot.</p> - <p>Lockers are 32-bit numbers, so we might choose to use the User ID of -the individual running the scheduling program. To schedule half-hour -slots, all the application needs to do is issue a <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a> call -for the appropriate locker/object pair. To schedule a longer slot, the -application needs to issue a <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> call, with one -<a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a> operation per half-hour — up to the total length. If -the <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> call fails, the application would have to release -the parts of the time slot that were obtained.</p> - <p>To cancel a reservation, the application would make the appropriate -<a href="../api_reference/C/lockput.html" class="olink">DB_ENV->lock_put()</a> calls. To reschedule a reservation, the -<a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a> and <a href="../api_reference/C/lockput.html" class="olink">DB_ENV->lock_put()</a> calls could all be made inside of -a single <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> call. The output of <a href="../api_reference/C/lockstat.html" class="olink">DB_ENV->lock_stat()</a> could -be post-processed into a human-readable schedule of conference room -use.</p> + <p> + The Lock subsystem is useful outside the context of Berkeley + DB. It can be used to manage concurrent access to any + collection of either ephemeral or persistent objects. That is, + the lock region can persist across invocations of an + application, so it can be used to provide long-term locking + (for example, conference room scheduling). + </p> + <p> + In order to use the locking subsystem in such a general way, + the applications must adhere to a convention for identifying + objects and lockers. Consider a conference room scheduling + problem, in which there are three conference rooms scheduled + in half-hour intervals. The scheduling application must then + select a way to identify each conference room/time slot + combination. In this case, we could describe the objects being + locked as bytestrings consisting of the conference room name, + the date when it is needed, and the beginning of the + appropriate half-hour slot. + </p> + <p> + Lockers are 32-bit numbers, so we might choose to use the + User ID of the individual running the scheduling program. To + schedule half-hour slots, all the application needs to do is + issue a <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a> call for the appropriate locker/object pair. + To schedule a longer slot, the application needs to issue a + <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> call, with one <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a> operation per half-hour + — up to the total length. If the <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> call fails, + the application would have to release the parts of the time + slot that were obtained. + </p> + <p> + To cancel a reservation, the application would make the + appropriate <a href="../api_reference/C/lockput.html" class="olink">DB_ENV->lock_put()</a> calls. To reschedule a reservation, the + <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a> and <a href="../api_reference/C/lockput.html" class="olink">DB_ENV->lock_put()</a> calls could all be made inside of a + single <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> call. The output of <a href="../api_reference/C/lockstat.html" class="olink">DB_ENV->lock_stat()</a> could be + post-processed into a human-readable schedule of conference + room use. + </p> </div> <div class="navfooter"> <hr /> @@ -78,13 +89,12 @@ use.</p> <td width="40%" align="right"> <a accesskey="n" href="log.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Berkeley DB Transactional Data Store locking conventions </td> + <td width="40%" align="left" valign="top">Berkeley DB Transactional Data + Store locking conventions </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 17. - The Logging Subsystem - </td> + <td width="40%" align="right" valign="top"> Chapter 17. The Logging Subsystem </td> </tr> </table> </div> diff --git a/docs/programmer_reference/lock_notxn.html b/docs/programmer_reference/lock_notxn.html index 0455d4a1..8d1f7959 100644 --- a/docs/programmer_reference/lock_notxn.html +++ b/docs/programmer_reference/lock_notxn.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="lock_page.html">Prev</a> </td> - <th width="60%" align="center">Chapter 16. - The Locking Subsystem - </th> + <th width="60%" align="center">Chapter 16. The Locking Subsystem </th> <td width="20%" align="right"> <a accesskey="n" href="lock_twopl.html">Next</a></td> </tr> </table> @@ -38,30 +36,38 @@ </div> </div> </div> - <p>If an application runs with locking specified, but not transactions (for -example, <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> is callsed with <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOCK" class="olink">DB_INIT_LOCK</a> or -<a href="../api_reference/C/envopen.html#envopen_DB_INIT_CDB" class="olink">DB_INIT_CDB</a> specified, but not <a href="../api_reference/C/envopen.html#envopen_DB_INIT_TXN" class="olink">DB_INIT_TXN</a>), locks are -normally acquired during each Berkeley DB operation and released before the -operation returns to the caller. The only exception is in the case of -cursor operations. Cursors identify a particular position in a file. -For this reason, cursors must retain read locks across cursor calls to -make sure that the position is uniquely identifiable during a subsequent -cursor call, and so that an operation using <a href="../api_reference/C/dbcget.html#dbcget_DB_CURRENT" class="olink">DB_CURRENT</a> will -always refer to the same record as a previous cursor call. These cursor -locks cannot be released until the cursor is either repositioned and a -new cursor lock established (for example, using the <a href="../api_reference/C/dbcget.html#dbcget_DB_NEXT" class="olink">DB_NEXT</a> -or <a href="../api_reference/C/dbcget.html#dbcget_DB_SET" class="olink">DB_SET</a> flags), or the cursor is closed. As a result, -application writers are encouraged to close cursors as soon as -possible.</p> - <p>It is important to realize that concurrent applications that use locking -must ensure that two concurrent threads do not block each other. -However, because Btree and Hash access method page splits can occur at -any time, there is virtually no way to guarantee that an application -that writes the database cannot deadlock. Applications running without -the protection of transactions may deadlock, and can leave the database -in an inconsistent state when they do so. Applications that need -concurrent access, but not transactions, are more safely implemented -using the Berkeley DB Concurrent Data Store Product.</p> + <p> + If an application runs with locking specified, but not + transactions (for example, <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> is callsed with + <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOCK" class="olink">DB_INIT_LOCK</a> or <a href="../api_reference/C/envopen.html#envopen_DB_INIT_CDB" class="olink">DB_INIT_CDB</a> specified, but not + <a href="../api_reference/C/envopen.html#envopen_DB_INIT_TXN" class="olink">DB_INIT_TXN</a>), locks are normally acquired during each + Berkeley DB operation and released before the operation + returns to the caller. The only exception is in the case of + cursor operations. Cursors identify a particular position in a + file. For this reason, cursors must retain read locks across + cursor calls to make sure that the position is uniquely + identifiable during a subsequent cursor call, and so that an + operation using <a href="../api_reference/C/dbcget.html#dbcget_DB_CURRENT" class="olink">DB_CURRENT</a> will always refer to the same + record as a previous cursor call. These cursor locks cannot be + released until the cursor is either repositioned and a new + cursor lock established (for example, using the <a href="../api_reference/C/dbcget.html#dbcget_DB_NEXT" class="olink">DB_NEXT</a> or + <a href="../api_reference/C/dbcget.html#dbcget_DB_SET" class="olink">DB_SET</a> flags), or the cursor is closed. As a result, + application writers are encouraged to close cursors as soon as + possible. + </p> + <p> + It is important to realize that concurrent applications that + use locking must ensure that two concurrent threads do not + block each other. However, because Btree and Hash access + method page splits can occur at any time, there is virtually + no way to guarantee that an application that writes the + database cannot deadlock. Applications running without the + protection of transactions may deadlock, and can leave the + database in an inconsistent state when they do so. + Applications that need concurrent access, but not + transactions, are more safely implemented using the Berkeley + DB Concurrent Data Store Product. + </p> </div> <div class="navfooter"> <hr /> @@ -78,7 +84,8 @@ using the Berkeley DB Concurrent Data Store Product.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Locking with transactions: two-phase locking</td> + <td width="40%" align="right" valign="top"> Locking with transactions: two-phase + locking</td> </tr> </table> </div> diff --git a/docs/programmer_reference/lock_page.html b/docs/programmer_reference/lock_page.html index b1733b70..06c099de 100644 --- a/docs/programmer_reference/lock_page.html +++ b/docs/programmer_reference/lock_page.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="lock_deaddbg.html">Prev</a> </td> - <th width="60%" align="center">Chapter 16. - The Locking Subsystem - </th> + <th width="60%" align="center">Chapter 16. The Locking Subsystem </th> <td width="20%" align="right"> <a accesskey="n" href="lock_notxn.html">Next</a></td> </tr> </table> @@ -38,57 +36,88 @@ </div> </div> </div> - <p>With the exception of the Queue access method, the Berkeley DB access methods -do page-level locking. The size of pages in a database may be set when -the database is created by calling the <a href="../api_reference/C/dbset_pagesize.html" class="olink">DB->set_pagesize()</a> method. If -not specified by the application, Berkeley DB selects a page size that will -provide the best I/O performance by setting the page size equal to the -block size of the underlying file system. Selecting a smaller page size -can result in increased concurrency for some applications.</p> - <p>In the Btree access method, Berkeley DB uses a technique called lock coupling -to improve concurrency. The traversal of a Btree requires reading a -page, searching that page to determine which page to search next, and -then repeating this process on the next page. Once a page has been -searched, it will never be accessed again for this operation, unless a -page split is required. To improve concurrency in the tree, once the -next page to read/search has been determined, that page is locked and -then the original page lock is released atomically (that is, without -relinquishing control of the lock manager). When page splits become -necessary, write locks are reacquired.</p> - <p>Because the Recno access method is built upon Btree, it also uses lock -coupling for read operations. However, because the Recno access method -must maintain a count of records on its internal pages, it cannot -lock-couple during write operations. Instead, it retains write locks -on all internal pages during every update operation. For this reason, -it is not possible to have high concurrency in the Recno access method -in the presence of write operations.</p> - <p>The Queue access method uses only short-term page locks. That is, a page -lock is released prior to requesting another page lock. Record locks are -used for transaction isolation. The provides a high degree of concurrency -for write operations. A metadata page is used to keep track of the head -and tail of the queue. This page is never locked during other locking or -I/O operations.</p> - <p>The Hash access method does not have such traversal issues, but it must -always refer to its metadata while computing a hash function because it -implements dynamic hashing. This metadata is stored on a special page -in the hash database. This page must therefore be read-locked on every -operation. Fortunately, it needs to be write-locked only when new pages -are allocated to the file, which happens in three cases:</p> + <p> + With the exception of the Queue access method, the Berkeley + DB access methods do page-level locking. The size of pages in + a database may be set when the database is created by calling + the <a href="../api_reference/C/dbset_pagesize.html" class="olink">DB->set_pagesize()</a> method. If not specified by the + application, Berkeley DB selects a page size that will provide + the best I/O performance by setting the page size equal to the + block size of the underlying file system. Selecting a smaller + page size can result in increased concurrency for some + applications. + </p> + <p> + In the Btree access method, Berkeley DB uses a technique + called lock coupling to improve concurrency. The traversal of + a Btree requires reading a page, searching that page to + determine which page to search next, and then repeating this + process on the next page. Once a page has been searched, it + will never be accessed again for this operation, unless a page + split is required. To improve concurrency in the tree, once + the next page to read/search has been determined, that page is + locked and then the original page lock is released atomically + (that is, without relinquishing control of the lock manager). + When page splits become necessary, write locks are + reacquired. + </p> + <p> + Because the Recno access method is built upon Btree, it also + uses lock coupling for read operations. However, because the + Recno access method must maintain a count of records on its + internal pages, it cannot lock-couple during write operations. + Instead, it retains write locks on all internal pages during + every update operation. For this reason, it is not possible to + have high concurrency in the Recno access method in the + presence of write operations. + </p> + <p> + The Queue access method uses only short-term page locks. + That is, a page lock is released prior to requesting another + page lock. Record locks are used for transaction isolation. + The provides a high degree of concurrency for write + operations. A metadata page is used to keep track of the head + and tail of the queue. This page is never locked during other + locking or I/O operations. + </p> + <p> + The Hash access method does not have such traversal issues, + but it must always refer to its metadata while computing a + hash function because it implements dynamic hashing. This + metadata is stored on a special page in the hash database. + This page must therefore be read-locked on every operation. + Fortunately, it needs to be write-locked only when new pages + are allocated to the file, which happens in three + cases: + </p> <div class="itemizedlist"> <ul type="disc"> - <li>a hash bucket becomes full and needs to split</li> - <li>a key or data item is too large to fit on a normal page</li> - <li>the number of duplicate items for a fixed key becomes so large that they -are moved to an auxiliary page</li> + <li> + a hash bucket becomes full and needs to + split + </li> + <li> + a key or data item is too large to fit on a normal + page + </li> + <li> + the number of duplicate items for a fixed key + becomes so large that they are moved to an auxiliary + page + </li> </ul> </div> - <p>In this case, the access method must obtain a write lock on the metadata -page, thus requiring that all readers be blocked from entering the tree -until the update completes.</p> - <p>Finally, when traversing duplicate data items for a key, the lock on -the key value also acts as a lock on all duplicates of that key. -Therefore, two conflicting threads of control cannot access the same -duplicate set simultaneously.</p> + <p> + In this case, the access method must obtain a write lock on + the metadata page, thus requiring that all readers be blocked + from entering the tree until the update completes. + </p> + <p> + Finally, when traversing duplicate data items for a key, the + lock on the key value also acts as a lock on all duplicates of + that key. Therefore, two conflicting threads of control cannot + access the same duplicate set simultaneously. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/lock_stdmode.html b/docs/programmer_reference/lock_stdmode.html index 243574f9..921405a3 100644 --- a/docs/programmer_reference/lock_stdmode.html +++ b/docs/programmer_reference/lock_stdmode.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="lock_max.html">Prev</a> </td> - <th width="60%" align="center">Chapter 16. - The Locking Subsystem - </th> + <th width="60%" align="center">Chapter 16. The Locking Subsystem </th> <td width="20%" align="right"> <a accesskey="n" href="lock_dead.html">Next</a></td> </tr> </table> @@ -38,16 +36,22 @@ </div> </div> </div> - <p>The Berkeley DB locking protocol is described by a conflict matrix. A -conflict matrix is an NxN array in which N is the number of different -lock modes supported, and the (i, j)th entry of the array indicates -whether a lock of mode i conflicts with a lock of mode j. In addition, -Berkeley DB defines the type <span class="bold"><strong>db_lockmode_t</strong></span>, which is the type of a -lock mode within a conflict matrix.</p> - <p>The following is an example of a conflict matrix. The actual conflict -matrix used by Berkeley DB to support the underlying access methods is more -complicated, but this matrix shows the lock mode relationships available -to applications using the Berkeley DB Locking subsystem interfaces directly.</p> + <p> + The Berkeley DB locking protocol is described by a conflict + matrix. A conflict matrix is an NxN array in which N is the + number of different lock modes supported, and the (i, j)th + entry of the array indicates whether a lock of mode i + conflicts with a lock of mode j. In addition, Berkeley DB + defines the type <span class="bold"><strong>db_lockmode_t</strong></span>, + which is the type of a lock mode within a conflict matrix. + </p> + <p> + The following is an example of a conflict matrix. The actual + conflict matrix used by Berkeley DB to support the underlying + access methods is more complicated, but this matrix shows the + lock mode relationships available to applications using the + Berkeley DB Locking subsystem interfaces directly. + </p> <div class="variablelist"> <dl> <dt> @@ -76,10 +80,13 @@ to applications using the Berkeley DB Locking subsystem interfaces directly.</p> <dd>intention to read and write (shared)</dd> </dl> </div> - <p>In a conflict matrix, the rows indicate the lock that is held, and the -columns indicate the lock that is requested. A 1 represents a conflict -(that is, do not grant the lock if the indicated lock is held), and a -0 indicates that it is OK to grant the lock.</p> + <p> + In a conflict matrix, the rows indicate the lock that is + held, and the columns indicate the lock that is requested. A 1 + represents a conflict (that is, do not grant the lock if the + indicated lock is held), and a 0 indicates that it is OK to + grant the lock. + </p> <pre class="programlisting"> Notheld Read Write IWrite IRead IRW Notheld 0 0 0 0 0 0 Read* 0 0 1 1 0 1 @@ -92,14 +99,20 @@ Intent RW 0 1 1 0 0 0</pre> <dt> <span class="term">*</span> </dt> - <dd>In this case, suppose that there is a read lock held on an object. A new -request for a read lock would be granted, but a request for a write lock -would not.</dd> + <dd> + In this case, suppose that there is a read lock + held on an object. A new request for a read lock would + be granted, but a request for a write lock would + not. + </dd> <dt> <span class="term">**</span> </dt> - <dd>In this case, suppose that there is a write lock held on an object. A -new request for either a read or write lock would be denied.</dd> + <dd> + In this case, suppose that there is a write lock + held on an object. A new request for either a read or + write lock would be denied. + </dd> </dl> </div> </div> @@ -114,7 +127,8 @@ new request for either a read or write lock would be denied.</dd> <td width="40%" align="right"> <a accesskey="n" href="lock_dead.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Configuring locking: sizing the system </td> + <td width="40%" align="left" valign="top">Configuring locking: sizing the + system </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/lock_timeout.html b/docs/programmer_reference/lock_timeout.html index 1b3001e0..97392f17 100644 --- a/docs/programmer_reference/lock_timeout.html +++ b/docs/programmer_reference/lock_timeout.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Deadlock detection using timers</th> + <th colspan="3" align="center">Deadlock detection using + timers</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="lock_dead.html">Prev</a> </td> - <th width="60%" align="center">Chapter 16. - The Locking Subsystem - </th> + <th width="60%" align="center">Chapter 16. The Locking Subsystem </th> <td width="20%" align="right"> <a accesskey="n" href="lock_deaddbg.html">Next</a></td> </tr> </table> @@ -34,57 +33,78 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="lock_timeout"></a>Deadlock detection using timers</h2> + <h2 class="title" style="clear: both"><a id="lock_timeout"></a>Deadlock detection using + timers</h2> </div> </div> </div> - <p>Lock and transaction timeouts may be used in place of, or in addition -to, regular deadlock detection. If lock timeouts are set, lock requests -will return <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_NOTGRANTED">DB_LOCK_NOTGRANTED</a> from a lock call when it is -detected that the lock's timeout has expired, that is, the lock request -has blocked, waiting, longer than the specified timeout. If transaction -timeouts are set, lock requests will return <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_NOTGRANTED">DB_LOCK_NOTGRANTED</a> -from a lock call when it has been detected that the transaction has been -active longer than the specified timeout.</p> - <p>If lock or transaction timeouts have been set, database operations will -return <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a> when the lock timeout has expired or the -transaction has been active longer than the specified timeout. -Applications wanting to distinguish between true deadlock and timeout -can use the <a href="../api_reference/C/envset_flags.html" class="olink">DB_ENV->set_flags()</a> configuration flag, which causes -database operations to instead return <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_NOTGRANTED">DB_LOCK_NOTGRANTED</a> in the -case of timeout.</p> - <p>As lock and transaction timeouts are only checked when lock requests -first block or when deadlock detection is performed, the accuracy of -the timeout depends on how often deadlock detection is performed. More -specifically, transactions will continue to run after their timeout has -expired if they do not block on a lock request after that time. -A separate deadlock detection thread (or process) should always -be used if the application depends on timeouts; otherwise, if -there are no new blocked lock requests a pending timeout will -never trigger.</p> - <p>If the database environment deadlock detector has been configured with -the <a href="../api_reference/C/lockdetect.html#detect_DB_LOCK_EXPIRE" class="olink">DB_LOCK_EXPIRE</a> option, timeouts are the only mechanism by -which deadlocks will be broken. If the deadlock detector has been -configured with a different option, then regular deadlock detection will -be performed, and in addition, if timeouts have also been specified, -lock requests and transactions will time out as well.</p> - <p>Lock and transaction timeouts may be specified on a database environment -wide basis using the <a href="../api_reference/C/envset_timeout.html" class="olink">DB_ENV->set_timeout()</a> method. Lock timeouts may be -specified on a per-lock request basis using the <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> method. Lock -and transaction timeouts may be specified on a per-transaction basis -using the <a href="../api_reference/C/txnset_timeout.html" class="olink">DB_TXN->set_timeout()</a> method. Per-lock and per-transaction -timeouts supersede environment wide timeouts.</p> - <p>For example, consider that the environment wide transaction timeout has -been set to 20ms, the environment wide lock timeout has been set to -10ms, a transaction has been created in this environment and its timeout -value set to 8ms, and a specific lock request has been made on behalf -of this transaction where the lock timeout was set to 4ms. By default, -transactions in this environment will be timed out if they block waiting -for a lock after 20ms. The specific transaction described will be timed -out if it blocks waiting for a lock after 8ms. By default, any lock -request in this system will be timed out if it blocks longer than 10ms, -and the specific lock described will be timed out if it blocks longer -than 4ms.</p> + <p> + Lock and transaction timeouts may be used in place of, or in + addition to, regular deadlock detection. If lock timeouts are + set, lock requests will return <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_NOTGRANTED">DB_LOCK_NOTGRANTED</a> + from a lock call when it is + detected that the lock's timeout has expired, that is, the + lock request has blocked, waiting, longer than the specified + timeout. If transaction timeouts are set, lock requests will + return <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_NOTGRANTED">DB_LOCK_NOTGRANTED</a> + from a lock call when it has been detected that the transaction has been + active longer than the specified timeout. + </p> + <p> + If lock or transaction timeouts have been set, database + operations will return <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a> + when the lock timeout has expired + or the transaction has been active longer than the specified + timeout. Applications wanting to distinguish between true + deadlock and timeout can use the <a href="../api_reference/C/envset_flags.html" class="olink">DB_ENV->set_flags()</a> configuration + flag, which causes database operations to instead return <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_NOTGRANTED">DB_LOCK_NOTGRANTED</a> + in the case of timeout. + </p> + <p> + As lock and transaction timeouts are only checked when lock + requests first block or when deadlock detection is performed, + the accuracy of the timeout depends on how often deadlock + detection is performed. More specifically, transactions will + continue to run after their timeout has expired if they do not + block on a lock request after that time. A separate deadlock + detection thread (or process) should always be used if the + application depends on timeouts; otherwise, if there are no + new blocked lock requests a pending timeout will never + trigger. + </p> + <p> + If the database environment deadlock detector has been + configured with the <a href="../api_reference/C/lockdetect.html#detect_DB_LOCK_EXPIRE" class="olink">DB_LOCK_EXPIRE</a> option, timeouts are the + only mechanism by which deadlocks will be broken. If the + deadlock detector has been configured with a different option, + then regular deadlock detection will be performed, and in + addition, if timeouts have also been specified, lock requests + and transactions will time out as well. + </p> + <p> + Lock and transaction timeouts may be specified on a database + environment wide basis using the <a href="../api_reference/C/envset_timeout.html" class="olink">DB_ENV->set_timeout()</a> method. Lock + timeouts may be specified on a per-lock request basis using + the <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> method. Lock and transaction timeouts may be + specified on a per-transaction basis using the + <a href="../api_reference/C/txnset_timeout.html" class="olink">DB_TXN->set_timeout()</a> method. Per-lock and per-transaction timeouts + supersede environment wide timeouts. + </p> + <p> + For example, consider that the environment wide transaction + timeout has been set to 20ms, the environment wide lock + timeout has been set to 10ms, a transaction has been created + in this environment and its timeout value set to 8ms, and a + specific lock request has been made on behalf of this + transaction where the lock timeout was set to 4ms. By default, + transactions in this environment will be timed out if they + block waiting for a lock after 20ms. The specific transaction + described will be timed out if it blocks waiting for a lock + after 8ms. By default, any lock request in this system will be + timed out if it blocks longer than 10ms, and the specific lock + described will be timed out if it blocks longer than + 4ms. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/lock_twopl.html b/docs/programmer_reference/lock_twopl.html index 044c68d1..067bd4dd 100644 --- a/docs/programmer_reference/lock_twopl.html +++ b/docs/programmer_reference/lock_twopl.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Locking with transactions: two-phase locking</th> + <th colspan="3" align="center">Locking with transactions: two-phase + locking</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="lock_notxn.html">Prev</a> </td> - <th width="60%" align="center">Chapter 16. - The Locking Subsystem - </th> + <th width="60%" align="center">Chapter 16. The Locking Subsystem </th> <td width="20%" align="right"> <a accesskey="n" href="lock_cam_conv.html">Next</a></td> </tr> </table> @@ -34,38 +33,57 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="lock_twopl"></a>Locking with transactions: two-phase locking</h2> + <h2 class="title" style="clear: both"><a id="lock_twopl"></a>Locking with transactions: two-phase + locking</h2> </div> </div> </div> - <p>Berkeley DB uses a locking protocol called <span class="emphasis"><em>two-phase locking (2PL)</em></span>. This -is the traditional protocol used in conjunction with lock-based transaction -systems.</p> - <p>In a two-phase locking system, transactions are divided into two -distinct phases. During the first phase, the transaction only acquires -locks; during the second phase, the transaction only releases locks. -More formally, once a transaction releases a lock, it may not acquire -any additional locks. Practically, this translates into a system in -which locks are acquired as they are needed throughout a transaction -and retained until the transaction ends, either by committing or -aborting. In Berkeley DB, locks are released during <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> or -<a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a>. The only exception to this protocol occurs when we -use lock-coupling to traverse a data structure. If the locks are held -only for traversal purposes, it is safe to release locks before -transactions commit or abort.</p> - <p>For applications, the implications of 2PL are that long-running -transactions will hold locks for a long time. When designing -applications, lock contention should be considered. In order to reduce -the probability of deadlock and achieve the best level of concurrency -possible, the following guidelines are helpful.</p> + <p> + Berkeley DB uses a locking protocol called + <span class="emphasis"><em>two-phase locking (2PL)</em></span>. This is the + traditional protocol used in conjunction with lock-based + transaction systems. + </p> + <p> + In a two-phase locking system, transactions are divided into + two distinct phases. During the first phase, the transaction + only acquires locks; during the second phase, the transaction + only releases locks. More formally, once a transaction + releases a lock, it may not acquire any additional locks. + Practically, this translates into a system in which locks are + acquired as they are needed throughout a transaction and + retained until the transaction ends, either by committing or + aborting. In Berkeley DB, locks are released during <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> + or <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a>. The only exception to this protocol occurs + when we use lock-coupling to traverse a data structure. If the + locks are held only for traversal purposes, it is safe to + release locks before transactions commit or abort. + </p> + <p> + For applications, the implications of 2PL are that + long-running transactions will hold locks for a long time. + When designing applications, lock contention should be + considered. In order to reduce the probability of deadlock and + achieve the best level of concurrency possible, the following + guidelines are helpful. + </p> <div class="orderedlist"> <ol type="1"> - <li>When accessing multiple databases, design all transactions so that they -access the files in the same order.</li> - <li>If possible, access your most hotly contested resources last (so that -their locks are held for the shortest time possible).</li> - <li>If possible, use nested transactions to protect the parts of your -transaction most likely to deadlock.</li> + <li> + When accessing multiple databases, design all + transactions so that they access the files in the same + order. + </li> + <li> + If possible, access your most hotly contested + resources last (so that their locks are held for the + shortest time possible). + </li> + <li> + If possible, use nested transactions to protect the + parts of your transaction most likely to + deadlock. + </li> </ol> </div> </div> diff --git a/docs/programmer_reference/log.html b/docs/programmer_reference/log.html index 2012ada2..8fd628b5 100644 --- a/docs/programmer_reference/log.html +++ b/docs/programmer_reference/log.html @@ -14,13 +14,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 17. - The Logging Subsystem - </th> + <th colspan="3" align="center">Chapter 17. The Logging Subsystem </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="lock_nondb.html">Prev</a> </td> @@ -34,9 +32,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="log"></a>Chapter 17. - The Logging Subsystem - </h2> + <h2 class="title"><a id="log"></a>Chapter 17. The Logging Subsystem </h2> </div> </div> </div> @@ -47,7 +43,8 @@ <dl> <dt> <span class="sect1"> - <a href="log.html#log_intro">Introduction to the logging subsystem</a> + <a href="log.html#log_intro">Introduction to the logging + subsystem</a> </span> </dt> <dt> @@ -66,27 +63,42 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="log_intro"></a>Introduction to the logging subsystem</h2> + <h2 class="title" style="clear: both"><a id="log_intro"></a>Introduction to the logging + subsystem</h2> </div> </div> </div> - <p>The Logging subsystem is the logging facility used by Berkeley DB. It is -largely Berkeley DB-specific, although it is potentially useful outside of -the Berkeley DB package for applications wanting write-ahead logging support. -Applications wanting to use the log for purposes other than logging file -modifications based on a set of open file descriptors will almost -certainly need to make source code modifications to the Berkeley DB code -base.</p> - <p>A log can be shared by any number of threads of control. The -<a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method is used to open a log. When the log is no longer -in use, it should be closed using the <a href="../api_reference/C/envclose.html" class="olink">DB_ENV->close()</a> method.</p> - <p>Individual log entries are identified by log sequence numbers. Log -sequence numbers are stored in an opaque object, an <a href="../api_reference/C/lsn.html" class="olink">DB_LSN</a>.</p> - <p>The <a href="../api_reference/C/logcursor.html" class="olink">DB_ENV->log_cursor()</a> method is used to allocate a log cursor. Log cursors -have two methods: <a href="../api_reference/C/logcget.html" class="olink">DB_LOGC->get()</a> method to retrieve log records from the -log, and <a href="../api_reference/C/logcclose.html" class="olink">DB_LOGC->close()</a> method to destroy the cursor.</p> - <p>There are additional methods for integrating the log subsystem with a -transaction processing system:</p> + <p> + The Logging subsystem is the logging facility used by + Berkeley DB. It is largely Berkeley DB-specific, although it + is potentially useful outside of the Berkeley DB package for + applications wanting write-ahead logging support. Applications + wanting to use the log for purposes other than logging file + modifications based on a set of open file descriptors will + almost certainly need to make source code modifications to the + Berkeley DB code base. + </p> + <p> + A log can be shared by any number of threads of control. The + <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method is used to open a log. When the log is no + longer in use, it should be closed using the <a href="../api_reference/C/envclose.html" class="olink">DB_ENV->close()</a> + method. + </p> + <p> + Individual log entries are identified by log sequence + numbers. Log sequence numbers are stored in an opaque object, + an <a href="../api_reference/C/lsn.html" class="olink">DB_LSN</a>. + </p> + <p> + The <a href="../api_reference/C/logcursor.html" class="olink">DB_ENV->log_cursor()</a> method is used to allocate a log cursor. Log + cursors have two methods: <a href="../api_reference/C/logcget.html" class="olink">DB_LOGC->get()</a> method to retrieve log + records from the log, and <a href="../api_reference/C/logcclose.html" class="olink">DB_LOGC->close()</a> method to destroy the + cursor. + </p> + <p> + There are additional methods for integrating the log + subsystem with a transaction processing system: + </p> <div class="variablelist"> <dl> <dt> @@ -94,44 +106,66 @@ transaction processing system:</p> <a href="../api_reference/C/logflush.html" class="olink">DB_ENV->log_flush()</a> </span> </dt> - <dd>Flushes the log up to a particular log sequence number.</dd> + <dd> + Flushes the log up to a particular log sequence + number. + </dd> <dt> <span class="term"> <a href="../api_reference/C/logcompare.html" class="olink">DB_ENV->log_compare()</a> </span> </dt> - <dd>Allows applications to compare any two log sequence numbers.</dd> + <dd> + Allows applications to compare any two log + sequence numbers. + </dd> <dt> <span class="term"> <a href="../api_reference/C/logfile.html" class="olink">DB_ENV->log_file()</a> </span> </dt> - <dd>Maps a log sequence number to the specific log file that contains it.</dd> + <dd> + Maps a log sequence number to the specific log + file that contains it. + </dd> <dt> <span class="term"> <a href="../api_reference/C/logarchive.html" class="olink">DB_ENV->log_archive()</a> </span> </dt> - <dd>Returns various sets of log filenames. These methods are used for -database administration; for example, to determine if log files may -safely be removed from the system.</dd> + <dd> + Returns various sets of log filenames. These + methods are used for database administration; for + example, to determine if log files may safely be + removed from the system. + </dd> <dt> <span class="term"> <a href="../api_reference/C/logstat.html" class="olink">DB_ENV->log_stat()</a> </span> </dt> - <dd> The display <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility used the <a href="../api_reference/C/logstat.html" class="olink">DB_ENV->log_stat()</a> method to -display statistics about the log.</dd> + <dd> + The display <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility used the <a href="../api_reference/C/logstat.html" class="olink">DB_ENV->log_stat()</a> method + to display statistics about the log. + </dd> <dt> <span class="term"> <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> </span> </dt> - <dd>The log meta-information (but not the log files themselves) may be -removed using the <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> method.</dd> + <dd> + The log meta-information (but not the log files + themselves) may be removed using the <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> + method. + </dd> </dl> </div> - <p>For more information on the logging subsystem methods, see the <a href="../api_reference/C/lsn.html#loglist" class="olink">Logging Subsystem and Related Methods</a> section in the <em class="citetitle">Berkeley DB C API Reference Guide.</em></p> + <p> + For more information on the logging subsystem methods, see + the <a href="../api_reference/C/lsn.html#loglist" class="olink">Logging + Subsystem and Related Methods</a> section in the + <em class="citetitle">Berkeley DB C API Reference Guide.</em> + </p> </div> </div> <div class="navfooter"> @@ -143,7 +177,8 @@ removed using the <a href="../api_reference/C/envremove.html" class="olink">DB_E <td width="40%" align="right"> <a accesskey="n" href="log_config.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Locking and non-Berkeley DB applications </td> + <td width="40%" align="left" valign="top">Locking and non-Berkeley DB + applications </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/log_config.html b/docs/programmer_reference/log_config.html index 361fb6af..d9052c15 100644 --- a/docs/programmer_reference/log_config.html +++ b/docs/programmer_reference/log_config.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="log.html">Prev</a> </td> - <th width="60%" align="center">Chapter 17. - The Logging Subsystem - </th> + <th width="60%" align="center">Chapter 17. The Logging Subsystem </th> <td width="20%" align="right"> <a accesskey="n" href="log_limits.html">Next</a></td> </tr> </table> @@ -38,39 +36,58 @@ </div> </div> </div> - <p>The aspects of logging that may be configured are the size of the -logging subsystem's region, the size of the log files on disk and the -size of the log buffer in memory. The <a href="../api_reference/C/envset_lg_regionmax.html" class="olink">DB_ENV->set_lg_regionmax()</a> method -specifies the size of the logging subsystem's region, in bytes. The -logging subsystem's default size is approximately 60KB. This value may -need to be increased if a large number of files are registered with the -Berkeley DB log manager, for example, by opening a large number of Berkeley DB -database files in a transactional application.</p> - <p>The <a href="../api_reference/C/envset_lg_max.html" class="olink">DB_ENV->set_lg_max()</a> method specifies the individual log file size for -all the applications sharing the Berkeley DB environment. Setting the log -file size is largely a matter of convenience and a reflection of the -application's preferences in backup media and frequency. However, -setting the log file size too low can potentially cause problems because -it would be possible to run out of log sequence numbers, which requires -a full archival and application restart to reset. See -<a class="xref" href="log_limits.html" title="Log file limits">Log file limits</a> for more -information.</p> - <p>The <a href="../api_reference/C/envset_lg_bsize.html" class="olink">DB_ENV->set_lg_bsize()</a> method specifies the size of the in-memory log -buffer, in bytes. Log information is stored in memory until the buffer -fills up or transaction commit forces the buffer to be written to disk. -Larger buffer sizes can significantly increase throughput in the -presence of long-running transactions, highly concurrent applications, -or transactions producing large amounts of data. By default, the buffer -is approximately 32KB.</p> - <p>The <a href="../api_reference/C/envset_lg_dir.html" class="olink">DB_ENV->set_lg_dir()</a> method specifies the directory in which -log files will be placed. By default, log files are placed in -the environment home directory.</p> - <p>The <a href="../api_reference/C/envset_lg_filemode.html" class="olink">DB_ENV->set_lg_filemode()</a> method specifies the absolute file mode for -created log files. This method is only useful for the rare Berkeley DB -application that does not control its umask value.</p> - <p>The <a href="../api_reference/C/envlog_set_config.html" class="olink">DB_ENV->log_set_config()</a> method configures several boolean parameters -that control the use of file system controls such as O_DIRECT and O_DSYNC, -automatic removal of log files, in-memory logging, and pre-zeroing of logfiles.</p> + <p> + The aspects of logging that may be configured are the size + of the logging subsystem's region, the size of the log files + on disk, the size of the log buffer in memory, and several + operating system settings affecting the behavior of log files. + </p> + <p> + The <a href="../api_reference/C/envset_lg_regionmax.html" class="olink">DB_ENV->set_lg_regionmax()</a> method specifies the size of the logging + subsystem's region, in bytes. The logging subsystem's default size + is approximately 60KB. This value may need to be increased if a + large number of files are registered with the Berkeley DB log + manager, for example, by opening a large number of Berkeley DB + database files in a transactional application. + </p> + <p> + The <a href="../api_reference/C/envset_lg_max.html" class="olink">DB_ENV->set_lg_max()</a> method specifies the individual log file + size for all the applications sharing the Berkeley DB + environment. Setting the log file size is largely a matter of + convenience and a reflection of the application's preferences + in backup media and frequency. However, setting the log file + size too low can potentially cause problems because it would + be possible to run out of log sequence numbers, which requires + a full archival and application restart to reset. See <a class="xref" href="log_limits.html" title="Log file limits">Log file limits</a> for more + information. + </p> + <p> + The <a href="../api_reference/C/envset_lg_bsize.html" class="olink">DB_ENV->set_lg_bsize()</a> method specifies the size of the + in-memory log buffer, in bytes. Log information is stored in + memory until the buffer fills up or transaction commit forces + the buffer to be written to disk. Larger buffer sizes can + significantly increase throughput in the presence of + long-running transactions, highly concurrent applications, or + transactions producing large amounts of data. By default, the + buffer is approximately 32KB. + </p> + <p> + The <a href="../api_reference/C/envset_lg_dir.html" class="olink">DB_ENV->set_lg_dir()</a> method specifies the directory in which + log files will be placed. By default, log files are placed in + the environment home directory. + </p> + <p> + The <a href="../api_reference/C/envset_lg_filemode.html" class="olink">DB_ENV->set_lg_filemode()</a> method specifies the absolute file + mode for created log files. This method is only useful for the + rare Berkeley DB application that does not control its umask + value. + </p> + <p> + The <a href="../api_reference/C/envlog_set_config.html" class="olink">DB_ENV->log_set_config()</a> method configures several boolean + parameters that control the use of file system controls such + as O_DIRECT and O_DSYNC, automatic removal of log files, + in-memory logging, and pre-zeroing of logfiles. + </p> </div> <div class="navfooter"> <hr /> @@ -83,9 +100,7 @@ automatic removal of log files, in-memory logging, and pre-zeroing of logfiles.< <td width="40%" align="right"> <a accesskey="n" href="log_limits.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 17. - The Logging Subsystem - </td> + <td width="40%" align="left" valign="top">Chapter 17. The Logging Subsystem </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/log_limits.html b/docs/programmer_reference/log_limits.html index 54f523d6..f29e3e44 100644 --- a/docs/programmer_reference/log_limits.html +++ b/docs/programmer_reference/log_limits.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="log_config.html">Prev</a> </td> - <th width="60%" align="center">Chapter 17. - The Logging Subsystem - </th> + <th width="60%" align="center">Chapter 17. The Logging Subsystem </th> <td width="20%" align="right"> <a accesskey="n" href="mp.html">Next</a></td> </tr> </table> @@ -38,32 +36,54 @@ </div> </div> </div> - <p>Log filenames and sizes impose a limit on how long databases may be -used in a Berkeley DB database environment. It is quite unlikely that an -application will reach this limit; however, if the limit is reached, -the Berkeley DB environment's databases must be dumped and reloaded.</p> - <p>The log filename consists of <span class="bold"><strong>log.</strong></span> followed by 10 digits, with -a maximum of 2,000,000,000 log files. Consider an application performing -6000 transactions per second for 24 hours a day, logged into 10MB log -files, in which each transaction is logging approximately 500 bytes of data. -The following calculation:</p> + <p> + Log filenames and sizes impose a limit on how long databases + may be used in a Berkeley DB database environment. It is quite + unlikely that an application will reach this limit; however, + if the limit is reached, the Berkeley DB environment's + databases must be dumped and reloaded. + </p> + <p> + The log filename consists of <span class="bold"><strong>log.</strong></span> + followed by 10 digits, with a maximum of + 2,000,000,000 log files. Consider an application performing + 6000 transactions per second for 24 hours a day, logged into + 10MB log files, in which each transaction is logging + approximately 500 bytes of data. The following + calculation: + </p> <pre class="programlisting">(10 * 2^20 * 2000000000) / (6000 * 500 * 365 * 60 * 60 * 24) = ~221</pre> - <p>indicates that the system will run out of log filenames in roughly 221 -years.</p> - <p>There is no way to reset the log filename space in Berkeley DB. If your -application is reaching the end of its log filename space, you must do -the following:</p> + <p> + indicates that the system will run out of log filenames in + roughly 221 years. + </p> + <p> + There is no way to reset the log filename space in Berkeley + DB. If your application is reaching the end of its log + filename space, you must do the following: + </p> <div class="orderedlist"> <ol type="1"> - <li>Archive your databases as if to prepare for catastrophic failure (see -<a class="xref" href="transapp_archival.html" title="Database and log file archival">Database and log file archival</a> -for more information).</li> - <li>Reset the database's log sequence numbers (see the <span class="bold"><strong>-r</strong></span> option -to the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility for more information).</li> - <li>Remove all of the log files from the database environment. (This is the -only situation in which all the log files are removed from an environment; -in all other cases, at least a single log file is retained.)</li> - <li>Restart your application.</li> + <li> + Archive your databases as if to prepare for + catastrophic failure (see <a class="xref" href="transapp_archival.html" title="Database and log file archival">Database and log file + archival</a> for more + information). + </li> + <li> + Reset the database's log sequence numbers (see the + <span class="bold"><strong>-r</strong></span> option to the + <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility for more information). + </li> + <li> + Remove all of the log files from the database + environment. (This is the only situation in which all the + log files are removed from an environment; in all other + cases, at least a single log file is retained.) + </li> + <li> + Restart your application. + </li> </ol> </div> </div> @@ -82,9 +102,7 @@ in all other cases, at least a single log file is retained.)</li> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 18. - The Memory Pool Subsystem - </td> + <td width="40%" align="right" valign="top"> Chapter 18. The Memory Pool Subsystem </td> </tr> </table> </div> diff --git a/docs/programmer_reference/moreinfo.html b/docs/programmer_reference/moreinfo.html index 70c54552..ea956d34 100644 --- a/docs/programmer_reference/moreinfo.html +++ b/docs/programmer_reference/moreinfo.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -45,72 +45,60 @@ </dt> </dl> </div> - <p> - Beyond this manual, you may also find the following sources of - information useful when building a DB application: + <p> + Beyond this manual, you may also find the following + sources of information useful when building a DB + application: </p> <div class="itemizedlist"> <ul type="disc"> <li> <p> - <a class="ulink" href="http://download.oracle.com/docs/cd/E17076_02/html/gsg_txn/C/index.html" target="_top"> - Getting Started with Transaction Processing for C + <a class="ulink" href="http://docs.oracle.com/cd/E17076_02/html/gsg_txn/C/index.html" target="_top"> Getting Started with Transaction Processing for C </a> </p> </li> <li> <p> - <a class="ulink" href="http://download.oracle.com/docs/cd/E17076_02/html/gsg_db_rep/C/index.html" target="_top"> - Berkeley DB Getting Started with Replicated Applications for C - </a> + <a class="ulink" href="http://docs.oracle.com/cd/E17076_02/html/gsg_db_rep/C/index.html" target="_top"> Berkeley DB Getting Started with Replicated Applications for C + </a> </p> </li> <li> <p> - <a class="ulink" href="http://download.oracle.com/docs/cd/E17076_02/html/api_reference/C/frame_main.html" target="_top"> - Berkeley DB C API Reference Guide - </a> - </p> + <a class="ulink" href="http://docs.oracle.com/cd/E17076_02/html/api_reference/C/frame_main.html" target="_top"> Berkeley DB C API Reference Guide </a> + </p> </li> <li> <p> - <a class="ulink" href="http://download.oracle.com/docs/cd/E17076_02/html/api_reference/CXX/frame_main.html" target="_top"> - Berkeley DB C++ API Reference Guide - </a> - </p> + <a class="ulink" href="http://docs.oracle.com/cd/E17076_02/html/api_reference/CXX/frame_main.html" target="_top"> Berkeley DB C++ API Reference Guide </a> + </p> </li> <li> <p> - <a class="ulink" href="http://download.oracle.com/docs/cd/E17076_02/html/api_reference/STL/frame_main.html" target="_top"> - Berkeley DB STL API Reference Guide - </a> - </p> + <a class="ulink" href="http://docs.oracle.com/cd/E17076_02/html/api_reference/STL/frame_main.html" target="_top"> Berkeley DB STL API Reference Guide </a> + </p> </li> <li> <p> - <a class="ulink" href="http://download.oracle.com/docs/cd/E17076_02/html/api_reference/TCL/frame_main.html" target="_top"> - Berkeley DB TCL API Reference Guide - </a> - </p> + <a class="ulink" href="http://docs.oracle.com/cd/E17076_02/html/api_reference/TCL/frame_main.html" target="_top"> Berkeley DB TCL API Reference Guide </a> + </p> </li> <li> <p> - <a class="ulink" href="http://download.oracle.com/docs/cd/E17076_02/html/installation/index.html" target="_top"> - Berkeley DB Installation and Build Guide - </a> + <a class="ulink" href="http://docs.oracle.com/cd/E17076_02/html/installation/index.html" target="_top"> + Berkeley DB Installation and Build Guide </a> </p> </li> <li> <p> - <a class="ulink" href="http://download.oracle.com/docs/cd/E17076_02/html/upgrading/index.html" target="_top"> - Berkeley DB Upgrade Guide - </a> + <a class="ulink" href="http://docs.oracle.com/cd/E17076_02/html/upgrading/index.html" target="_top"> + Berkeley DB Upgrade Guide </a> </p> </li> <li> <p> - <a class="ulink" href="http://download.oracle.com/docs/cd/E17076_02/html/bdb-sql/index.html" target="_top"> - Berkeley DB Getting Started with the SQL APIs + <a class="ulink" href="http://docs.oracle.com/cd/E17076_02/html/bdb-sql/index.html" target="_top"> Berkeley DB Getting Started with the SQL APIs </a> </p> </li> @@ -131,7 +119,7 @@ downloads, visit - <a class="ulink" href="http://www.oracle.com/technetwork/database/berkeleydb/downloads/index.html" target="_top">http://www.oracle.com/technetwork/database/berkeleydb/downloads/index.html</a>. + <a class="ulink" href="http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/downloads/index.html" target="_top">http://www.oracle.com/technetwork/database/database-technologies/berkeleydb/downloads/index.html</a>. </p> </span> <div class="sect2" lang="en" xml:lang="en"> @@ -146,8 +134,8 @@ You can post your comments and questions at the Oracle Technology (OTN) forum for <span> - Oracle Berkeley DB at: <a class="ulink" href="http://forums.oracle.com/forums/forum.jspa?forumID=271" target="_top">http://forums.oracle.com/forums/forum.jspa?forumID=271</a>, - or for Oracle Berkeley DB High Availability at: <a class="ulink" href="http://forums.oracle.com/forums/forum.jspa?forumID=272" target="_top">http://forums.oracle.com/forums/forum.jspa?forumID=272</a>. + Oracle Berkeley DB at: <a class="ulink" href="https://forums.oracle.com/forums/forum.jspa?forumID=271" target="_top">https://forums.oracle.com/forums/forum.jspa?forumID=271</a>, + or for Oracle Berkeley DB High Availability at: <a class="ulink" href="https://forums.oracle.com/forums/forum.jspa?forumID=272" target="_top">https://forums.oracle.com/forums/forum.jspa?forumID=272</a>. </span> @@ -176,9 +164,7 @@ <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 1. - Introduction - </td> + <td width="40%" align="right" valign="top"> Chapter 1. Introduction </td> </tr> </table> </div> diff --git a/docs/programmer_reference/mp.html b/docs/programmer_reference/mp.html index 02e23090..12b2649c 100644 --- a/docs/programmer_reference/mp.html +++ b/docs/programmer_reference/mp.html @@ -14,13 +14,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 18. - The Memory Pool Subsystem - </th> + <th colspan="3" align="center">Chapter 18. The Memory Pool Subsystem </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="log_limits.html">Prev</a> </td> @@ -34,9 +32,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="mp"></a>Chapter 18. - The Memory Pool Subsystem - </h2> + <h2 class="title"><a id="mp"></a>Chapter 18. The Memory Pool Subsystem </h2> </div> </div> </div> @@ -79,64 +75,110 @@ </div> </div> </div> - <p>The Memory Pool subsystem is the general-purpose shared memory buffer -pool used by Berkeley DB. This module is useful outside of the Berkeley DB package -for processes that require page-oriented, shared and cached file -access. (However, such "use outside of Berkeley DB" is not supported -in replicated environments.)</p> - <p>A <span class="emphasis"><em>memory pool</em></span> is a memory cache shared among any number of -threads of control. The <a href="../api_reference/C/envopen.html#envopen_DB_INIT_MPOOL" class="olink">DB_INIT_MPOOL</a> flag to the -<a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method opens and optionally creates a memory pool. When -that pool is no longer in use, it should be closed using the -<a href="../api_reference/C/envclose.html" class="olink">DB_ENV->close()</a> method.</p> - <p>The <a href="../api_reference/C/mempfcreate.html" class="olink">DB_ENV->memp_fcreate()</a> method returns a <a href="../api_reference/C/memp.html" class="olink">DB_MPOOLFILE</a> handle on an -underlying file within the memory pool. The file may be opened using -the <a href="../api_reference/C/mempfopen.html" class="olink">DB_MPOOLFILE->open()</a> method. The <a href="../api_reference/C/mempfget.html" class="olink">DB_MPOOLFILE->get()</a> method is used to retrieve -pages from files in the pool. All retrieved pages must be subsequently -returned using the <a href="../api_reference/C/mempput.html" class="olink">DB_MPOOLFILE->put()</a> method. At the time pages are returned, -they may be marked <span class="bold"><strong>dirty</strong></span>, which causes them to be written to -the underlying file before being discarded from the pool. If there is -insufficient room to bring a new page in the pool, a page is selected -to be discarded from the pool using a least-recently-used algorithm. -All dirty pages in the pool from the file may be flushed using the -<a href="../api_reference/C/mempfsync.html" class="olink">DB_MPOOLFILE->sync()</a> method. When the file handle is no longer in use, it -should be closed using the <a href="../api_reference/C/mempfclose.html" class="olink">DB_MPOOLFILE->close()</a> method.</p> - <p>There are additional configuration interfaces that apply when opening -a new file in the memory pool:</p> + <p> + The Memory Pool subsystem is the general-purpose shared + memory buffer pool used by Berkeley DB. This module is useful + outside of the Berkeley DB package for processes that require + page-oriented, shared and cached file access. (However, such + "use outside of Berkeley DB" is not supported in replicated + environments.) + </p> + <p> + A <span class="emphasis"><em>memory pool</em></span> is a memory cache shared + among any number of threads of control. The <a href="../api_reference/C/envopen.html#envopen_DB_INIT_MPOOL" class="olink">DB_INIT_MPOOL</a> + flag to the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method opens and optionally creates a + memory pool. When that pool is no longer in use, it should be + closed using the <a href="../api_reference/C/envclose.html" class="olink">DB_ENV->close()</a> method. + </p> + <p> + The <a href="../api_reference/C/mempfcreate.html" class="olink">DB_ENV->memp_fcreate()</a> method returns a <a href="../api_reference/C/memp.html" class="olink">DB_MPOOLFILE</a> handle on an + underlying file within the memory pool. The file may be opened + using the <a href="../api_reference/C/mempfopen.html" class="olink">DB_MPOOLFILE->open()</a> method. The <a href="../api_reference/C/mempfget.html" class="olink">DB_MPOOLFILE->get()</a> method is used to + retrieve pages from files in the pool. All retrieved pages + must be subsequently returned using the <a href="../api_reference/C/mempput.html" class="olink">DB_MPOOLFILE->put()</a> method. At + the time pages are returned, they may be marked <span class="bold"><strong>dirty</strong></span>, which causes them to be + written to the underlying file before being discarded from the + pool. If there is insufficient room to bring a new page in the + pool, a page is selected to be discarded from the pool using a + least-recently-used algorithm. All dirty pages in the pool + from the file may be flushed using the <a href="../api_reference/C/mempfsync.html" class="olink">DB_MPOOLFILE->sync()</a> method. + When the file handle is no longer in use, it should be closed + using the <a href="../api_reference/C/mempfclose.html" class="olink">DB_MPOOLFILE->close()</a> method. + </p> + <p> + There are additional configuration interfaces that apply + when opening a new file in the memory pool: + </p> <div class="itemizedlist"> <ul type="disc"> - <li>The <a href="../api_reference/C/mempset_clear_len.html" class="olink">DB_MPOOLFILE->set_clear_len()</a> method specifies the number of bytes to clear -when creating a new page in the memory pool.</li> - <li>The <a href="../api_reference/C/mempset_fileid.html" class="olink">DB_MPOOLFILE->set_fileid()</a> method specifies a unique ID associated with the file.</li> - <li>The <a href="../api_reference/C/mempset_ftype.html" class="olink">DB_MPOOLFILE->set_ftype()</a> method specifies the type of file for the purposes of -page input and output processing.</li> - <li>The <a href="../api_reference/C/mempset_lsn_offset.html" class="olink">DB_MPOOLFILE->set_lsn_offset()</a> method specifies the byte offset of each page's -log sequence number (<a href="../api_reference/C/lsn.html" class="olink">DB_LSN</a>) for the purposes of transaction -checkpoints.</li> - <li>The <a href="../api_reference/C/mempset_pgcookie.html" class="olink">DB_MPOOLFILE->set_pgcookie()</a> method specifies an application provided argument -for the purposes of page input and output processing.</li> + <li> + The <a href="../api_reference/C/mempset_clear_len.html" class="olink">DB_MPOOLFILE->set_clear_len()</a> method specifies the number + of bytes to clear when creating a new page in the memory + pool. + </li> + <li> + The <a href="../api_reference/C/mempset_fileid.html" class="olink">DB_MPOOLFILE->set_fileid()</a> method specifies a unique ID + associated with the file. + </li> + <li> + The <a href="../api_reference/C/mempset_ftype.html" class="olink">DB_MPOOLFILE->set_ftype()</a> method specifies the type of + file for the purposes of page input and output + processing. + </li> + <li> + The <a href="../api_reference/C/mempset_lsn_offset.html" class="olink">DB_MPOOLFILE->set_lsn_offset()</a> method specifies the byte + offset of each page's log sequence number (<a href="../api_reference/C/lsn.html" class="olink">DB_LSN</a>) for the + purposes of transaction checkpoints. + </li> + <li> + The <a href="../api_reference/C/mempset_pgcookie.html" class="olink">DB_MPOOLFILE->set_pgcookie()</a> method specifies an + application provided argument for the purposes of page + input and output processing. + </li> </ul> </div> - <p>There are additional interfaces for the memory pool as a whole:</p> + <p> + There are additional interfaces for the memory pool as a + whole: + </p> <div class="itemizedlist"> <ul type="disc"> - <li>It is possible to gradually flush buffers from the pool in order to -maintain a consistent percentage of clean buffers in the pool using -the <a href="../api_reference/C/memptrickle.html" class="olink">DB_ENV->memp_trickle()</a> method.</li> - <li>Because special-purpose processing may be necessary when pages are read -or written (for example, endian conversion, or page checksums), the -<a href="../api_reference/C/mempregister.html" class="olink">DB_ENV->memp_register()</a> function allows applications to specify automatic -input and output processing in these cases.</li> - <li>The <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility uses the <a href="../api_reference/C/mempstat.html" class="olink">DB_ENV->memp_stat()</a> method to display -statistics about the efficiency of the pool.</li> - <li>All dirty pages in the pool may be flushed using the <a href="../api_reference/C/mempsync.html" class="olink">DB_ENV->memp_sync()</a> method. -In addition, <a href="../api_reference/C/mempsync.html" class="olink">DB_ENV->memp_sync()</a> takes an argument that is specific to -database systems, and which allows the memory pool to be flushed up to -a specified log sequence number (<a href="../api_reference/C/lsn.html" class="olink">DB_LSN</a>).</li> - <li>The entire pool may be discarded using the <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> method.</li> + <li> + It is possible to gradually flush buffers from the + pool in order to maintain a consistent percentage of clean + buffers in the pool using the <a href="../api_reference/C/memptrickle.html" class="olink">DB_ENV->memp_trickle()</a> + method. + </li> + <li> + Because special-purpose processing may be necessary + when pages are read or written (for example, endian + conversion, or page checksums), the <a href="../api_reference/C/mempregister.html" class="olink">DB_ENV->memp_register()</a> + function allows applications to specify automatic input + and output processing in these cases. + </li> + <li> + The <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility uses the <a href="../api_reference/C/mempstat.html" class="olink">DB_ENV->memp_stat()</a> method to display + statistics about the efficiency of the pool. + </li> + <li> + All dirty pages in the pool may be flushed using the + <a href="../api_reference/C/mempsync.html" class="olink">DB_ENV->memp_sync()</a> method. In addition, <a href="../api_reference/C/mempsync.html" class="olink">DB_ENV->memp_sync()</a> takes an + argument that is specific to database systems, and which + allows the memory pool to be flushed up to a specified log + sequence number (<a href="../api_reference/C/lsn.html" class="olink">DB_LSN</a>). + </li> + <li> + The entire pool may be discarded using the + <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> method. + </li> </ul> </div> - <p>For more information on the memory pool subsystem methods, see the <a href="../api_reference/C/memp.html#memplist" class="olink">Memory Pools and Related Methods</a> section in the <em class="citetitle">Berkeley DB C API Reference Guide.</em> </p> + <p> + For more information on the memory pool subsystem methods, + see the <a href="../api_reference/C/memp.html#memplist" class="olink">Memory + Pools and Related Methods</a> section in the + <em class="citetitle">Berkeley DB C API Reference Guide.</em> + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/mp_config.html b/docs/programmer_reference/mp_config.html index ae7083aa..1c585e4b 100644 --- a/docs/programmer_reference/mp_config.html +++ b/docs/programmer_reference/mp_config.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="mp.html">Prev</a> </td> - <th width="60%" align="center">Chapter 18. - The Memory Pool Subsystem - </th> + <th width="60%" align="center">Chapter 18. The Memory Pool Subsystem </th> <td width="20%" align="right"> <a accesskey="n" href="mp_warm.html">Next</a></td> </tr> </table> @@ -38,52 +36,65 @@ </div> </div> </div> - <p>There are two issues to consider when configuring the memory pool.</p> - <p>The first issue, the most important tuning parameter for Berkeley DB -applications, is the size of the memory pool. There are two ways to -specify the pool size. First, calling the <a href="../api_reference/C/envset_cachesize.html" class="olink">DB_ENV->set_cachesize()</a> method -specifies the pool size for all of the applications sharing the Berkeley DB -environment. Second, the <a href="../api_reference/C/dbset_cachesize.html" class="olink">DB->set_cachesize()</a> method only specifies a -pool size for the specific database. Note: It is meaningless to call -<a href="../api_reference/C/dbset_cachesize.html" class="olink">DB->set_cachesize()</a> for a database opened inside of a Berkeley DB -environment because the environment pool size will override any pool -size specified for a single database. For information on tuning the -Berkeley DB cache size, see -<a class="xref" href="general_am_conf.html#am_conf_cachesize" title="Selecting a cache size">Selecting a cache size</a>.</p> <p> - Note the memory pool defaults to assuming that the average page size is - 4k. This factor is used to determine the size of the hash table used - to locate pages in the memory pool. The size of the hash table is - calculated to so that on average 2.5 pages will be in each hash table - entry. Each page requires a mutex be allocated to it and the average - page size is used to determine the number of mutexes to allocate to the - memory pool. -</p> + There are two issues to consider when configuring the memory + pool. + </p> <p> - Normally you should see good results by using the default values for - the page size, but in some cases you may be able to achieve better - performance by manually configuring the page size. The expected page - size, hash table size and mutex count can be set via the methods: - <a href="../api_reference/C/envset_mp_pagesize.html" class="olink">DB_ENV->set_mp_pagesize()</a>, <a href="../api_reference/C/envset_mp_tablesize.html" class="olink">DB_ENV->set_mp_tablesize()</a>, and <a href="../api_reference/C/envset_mp_mtxcount.html" class="olink">DB_ENV->set_mp_mtxcount()</a>. -</p> - <p>The second memory pool configuration issue is the maximum size an -underlying file can be and still be mapped into the process address -space (instead of reading the file's pages into the cache). Mapping -files into the process address space can result in better performance -because available virtual memory is often much larger than the local -cache, and page faults are faster than page copying on many systems. -However, in the presence of limited virtual memory, it can cause -resource starvation; and in the presence of large databases, it can -result in immense process sizes. In addition, because of the -requirements of the Berkeley DB transactional implementation, only read-only -files can be mapped into process memory.</p> - <p>To specify that no files are to be mapped into the process address -space, specify the <a href="../api_reference/C/dbopen.html#open_DB_NOMMAP" class="olink">DB_NOMMAP</a> flag to the -<a href="../api_reference/C/envset_flags.html" class="olink">DB_ENV->set_flags()</a> method. To specify that any individual file should -not be mapped into the process address space, specify the -<a href="../api_reference/C/dbopen.html#open_DB_NOMMAP" class="olink">DB_NOMMAP</a> flag to the <a href="../api_reference/C/mempfopen.html" class="olink">DB_MPOOLFILE->open()</a> interface. To limit -the size of files mapped into the process address space, use the -<a href="../api_reference/C/envset_mp_mmapsize.html" class="olink">DB_ENV->set_mp_mmapsize()</a> method.</p> + The first issue, the most important tuning parameter for + Berkeley DB applications, is the size of the memory pool. + There are two ways to specify the pool size. First, calling + the <a href="../api_reference/C/envset_cachesize.html" class="olink">DB_ENV->set_cachesize()</a> method specifies the pool size for all + of the applications sharing the Berkeley DB environment. + Second, the <a href="../api_reference/C/dbset_cachesize.html" class="olink">DB->set_cachesize()</a> method only specifies a pool + size for the specific database. Note: It is meaningless to + call <a href="../api_reference/C/dbset_cachesize.html" class="olink">DB->set_cachesize()</a> for a database opened inside of a + Berkeley DB environment because the environment pool size will + override any pool size specified for a single database. For + information on tuning the Berkeley DB cache size, see <a class="xref" href="general_am_conf.html#am_conf_cachesize" title="Selecting a cache size">Selecting a cache size</a>. + </p> + <p> + Note the memory pool defaults to assuming that the average + page size is 4k. This factor is used to determine the size of + the hash table used to locate pages in the memory pool. The + size of the hash table is calculated to so that on average 2.5 + pages will be in each hash table entry. Each page requires a + mutex be allocated to it and the average page size is used to + determine the number of mutexes to allocate to the memory + pool. + </p> + <p> + Normally you should see good results by using the default + values for the page size, but in some cases you may be able to + achieve better performance by manually configuring the page + size. The expected page size, hash table size and mutex count + can be set via the methods: <a href="../api_reference/C/envset_mp_pagesize.html" class="olink">DB_ENV->set_mp_pagesize()</a>, + <a href="../api_reference/C/envset_mp_tablesize.html" class="olink">DB_ENV->set_mp_tablesize()</a>, and <a href="../api_reference/C/envset_mp_mtxcount.html" class="olink">DB_ENV->set_mp_mtxcount()</a>. + </p> + <p> + The second memory pool configuration issue is the maximum + size an underlying file can be and still be mapped into the + process address space (instead of reading the file's pages + into the cache). Mapping files into the process address space + can result in better performance because available virtual + memory is often much larger than the local cache, and page + faults are faster than page copying on many systems. However, + in the presence of limited virtual memory, it can cause + resource starvation; and in the presence of large databases, + it can result in immense process sizes. In addition, because + of the requirements of the Berkeley DB transactional + implementation, only read-only files can be mapped into + process memory. + </p> + <p> + To specify that no files are to be mapped into the process + address space, specify the <a href="../api_reference/C/dbopen.html#open_DB_NOMMAP" class="olink">DB_NOMMAP</a> flag to the + <a href="../api_reference/C/envset_flags.html" class="olink">DB_ENV->set_flags()</a> method. To specify that any individual file + should not be mapped into the process address space, specify + the <a href="../api_reference/C/dbopen.html#open_DB_NOMMAP" class="olink">DB_NOMMAP</a> flag to the <a href="../api_reference/C/mempfopen.html" class="olink">DB_MPOOLFILE->open()</a> interface. To limit + the size of files mapped into the process address space, use + the <a href="../api_reference/C/envset_mp_mmapsize.html" class="olink">DB_ENV->set_mp_mmapsize()</a> method. + </p> </div> <div class="navfooter"> <hr /> @@ -96,9 +107,7 @@ the size of files mapped into the process address space, use the <td width="40%" align="right"> <a accesskey="n" href="mp_warm.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 18. - The Memory Pool Subsystem - </td> + <td width="40%" align="left" valign="top">Chapter 18. The Memory Pool Subsystem </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/mp_warm.html b/docs/programmer_reference/mp_warm.html index 402f4203..e4d9fa81 100644 --- a/docs/programmer_reference/mp_warm.html +++ b/docs/programmer_reference/mp_warm.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="mp_config.html">Prev</a> </td> - <th width="60%" align="center">Chapter 18. - The Memory Pool Subsystem - </th> + <th width="60%" align="center">Chapter 18. The Memory Pool Subsystem </th> <td width="20%" align="right"> <a accesskey="n" href="txn.html">Next</a></td> </tr> </table> @@ -47,58 +45,61 @@ </dt> </dl> </div> - <p> - Some applications find it is useful to pre-load the memory pool - upon application startup. This is a strictly optional activity that - provides faster initial access to your data at the expense of - longer application startup times. + <p> + Some applications find it is useful to pre-load the memory + pool upon application startup. This is a strictly optional + activity that provides faster initial access to your data at + the expense of longer application startup times. </p> - <p> - To warm the cache, you simply have to read the records that your - application will operate on most frequently. You can do this with - normal database reads, and you can also use cursors. But the most - efficient way to warm the cache is to use memory pool APIs to get - the pages that contain your most frequently accessed records. + <p> + To warm the cache, you simply have to read the records that + your application will operate on most frequently. You can do + this with normal database reads, and you can also use cursors. + But the most efficient way to warm the cache is to use memory + pool APIs to get the pages that contain your most frequently + accessed records. </p> - <p> + <p> You read pages into the memory pool using the - <code class="methodname">DB_MPOOLFILE->get()</code> method. This method - acquires locks on the page, so immediately upon getting the page - you need to put it so as to release the locks. + <code class="methodname">DB_MPOOLFILE->get()</code> method. This + method acquires locks on the page, so immediately upon getting + the page you need to put it so as to release the locks. </p> <p> - Also, you obtain a memory pool file handle using a database handle. - This means that if your data is contained in more than one Berkeley - DB database, you must operate on each database handle in turn. + Also, you obtain a memory pool file handle using a database + handle. This means that if your data is contained in more than + one Berkeley DB database, you must operate on each database + handle in turn. </p> <p> - The following example code illustrates this. It does the following: + The following example code illustrates this. It does the + following: </p> <div class="itemizedlist"> <ul type="disc"> <li> - <p> + <p> Opens an environment and two database handles. </p> </li> <li> <p> - Determines how many database pages can fit into the memory - pool. + Determines how many database pages can fit into the + memory pool. </p> </li> <li> - <p> - Uses <code class="methodname">DB_MPOOLFILE->get()</code> and - <code class="methodname">DB_MPOOLFILE->put()</code> - to load that number of pages into the memory pool. + <p> + Uses <code class="methodname">DB_MPOOLFILE->get()</code> + and <code class="methodname">DB_MPOOLFILE->put()</code> to + load that number of pages into the memory pool. </p> </li> </ul> </div> <p> - First, we include the libraries that we need, forward declare some - functions, and intialize some variables. + First, we include the libraries that we need, forward + declare some functions, and intialize some variables. </p> <a id="prog_mp01-1"></a> <pre class="programlisting">#include <stdio.h> @@ -117,12 +118,12 @@ main(void) DB_ENV *envp = 0; u_int32_t env_flags, pagesize, gbytes, bytes; int ret = 0, ret_t = 0, numcachepages, pagecount; </pre> - <p> + <p> Then we open the environment and our databases. The - <code class="methodname">open_db()</code> function that we use here simply - opens a database. We will provide that code at the end of this - example, but it should hold no surprises for you. - We only use the function so as to reuse the code. + <code class="methodname">open_db()</code> function that we use + here simply opens a database. We will provide that code at the + end of this example, but it should hold no surprises for you. + We only use the function so as to reuse the code. </p> <a id="prog_mp01-2"></a> <pre class="programlisting"> /* @@ -161,9 +162,9 @@ main(void) if (ret != 0) goto err; </pre> <p> - Next we determine how many database pages we can fit into the - cache. We do this by finding out how large our pages are, and then - finding out how large our cache can be. + Next we determine how many database pages we can fit into + the cache. We do this by finding out how large our pages are, + and then finding out how large our cache can be. </p> <a id="prog_mp01-3"></a> <pre class="programlisting"> /* Find out how many pages can fit at most in the cache */ @@ -183,12 +184,12 @@ main(void) /* Avoid an overflow by first calculating pages per gigabyte. */ numcachepages = gbytes * ((1024 * 1024 * 1024) / pagesize); numcachepages += bytes / pagesize; </pre> - <p> + <p> Now we call our <code class="methodname">warm_cache()</code> - function. We will describe this function in a little while, but - note that we call <code class="methodname">warm_cache()</code> + function. We will describe this function in a little while, + but note that we call <code class="methodname">warm_cache()</code> twice. This is because our example uses two databases, and the - memory pool methods operate on a per-handle basis. + memory pool methods operate on a per-handle basis. </p> <a id="prog_mp01-4"></a> <pre class="programlisting"> /* @@ -209,7 +210,7 @@ main(void) db_strerror(ret)); goto err; } </pre> - <p> + <p> Now we close all our handles and finish our <code class="methodname">main()</code> function. Again, this is straight-forward boilerplate code that we provide simply to be @@ -250,11 +251,11 @@ main(void) printf("I'm all done.\n"); return (ret == 0 ? EXIT_SUCCESS : EXIT_FAILURE); } </pre> - <p> - As noted above, this example uses an <code class="methodname">open_db()</code> - function, which opens a database handle inside the provided - environment. To be complete, this is the implementation of that - function: + <p> + As noted above, this example uses an + <code class="methodname">open_db()</code> function, which opens a + database handle inside the provided environment. To be + complete, this is the implementation of that function: </p> <a id="prog_mp01-6"></a> <pre class="programlisting">/* Open a database handle */ @@ -304,11 +305,11 @@ open_db(DB_ENV *envp, DB **dbpp, const char *file_name) </div> <p> In this section we provide the implementation of the - <code class="methodname">warm_cache()</code> function. This example - function simply loads all the database pages that will fit into - the memory pool. It starts from the first database page and - continues until it either runs out of database pages or it runs - out of room in the memory pool. + <code class="methodname">warm_cache()</code> function. This + example function simply loads all the database pages that + will fit into the memory pool. It starts from the first + database page and continues until it either runs out of + database pages or it runs out of room in the memory pool. </p> <a id="prog_mp01-7"></a> <pre class="programlisting">/* Warm the cache */ @@ -378,9 +379,7 @@ warm_cache(DB *dbp, int *pagecountp, int numcachepages) <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 19. - The Transaction Subsystem - </td> + <td width="40%" align="right" valign="top"> Chapter 19. The Transaction Subsystem </td> </tr> </table> </div> diff --git a/docs/programmer_reference/preface.html b/docs/programmer_reference/preface.html index 2815d535..ae2c8b38 100644 --- a/docs/programmer_reference/preface.html +++ b/docs/programmer_reference/preface.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -62,15 +62,15 @@ </dd> </dl> </div> + <p> + Welcome to Berkeley DB (DB). This document provides an + introduction and usage notes for skilled programmers who wish + to use the Berkeley DB APIs. + </p> <p> - Welcome to Berkeley DB (DB). This document provides an - introduction and usage notes for skilled programmers who wish - to use the Berkeley DB APIs. - </p> - <p> - This document reflects Berkeley DB 11<span class="emphasis"><em>g</em></span> Release 2, which - provides DB library version 11.2.5.3. - </p> + This document reflects Berkeley DB 12<span class="emphasis"><em>c</em></span> Release 1, which provides + DB library version 12.1.6.1. + </p> <div class="sect1" lang="en" xml:lang="en"> <div class="titlepage"> <div> @@ -79,23 +79,27 @@ </div> </div> </div> - <p> - The following typographical conventions are used within in this manual: - </p> - <p> - Structure names are represented in <code class="classname">monospaced font</code>, as are - <code class="methodname">method names</code>. For example: - "<code class="methodname">DB->open()</code> is a method - on a <code class="classname">DB</code> handle." - </p> - <p> - Variable or non-literal text is presented in <span class="emphasis"><em>italics</em></span>. For example: - "Go to your <span class="emphasis"><em>DB_INSTALL</em></span> directory." - </p> - <p> - Program examples are displayed in a <code class="classname">monospaced font</code> on a shaded background. - For example: - </p> + <p> + The following typographical conventions are used within + in this manual: + </p> + <p> + Structure names are represented in + <code class="classname">monospaced font</code>, as are + <code class="methodname">method names</code>. For example: + "<code class="methodname">DB->open()</code> is a method on + a <code class="classname">DB</code> handle." + </p> + <p> + Variable or non-literal text is presented in + <span class="emphasis"><em>italics</em></span>. For example: "Go to your + <span class="emphasis"><em>DB_INSTALL</em></span> directory." + </p> + <p> + Program examples are displayed in a + <code class="classname">monospaced font</code> on a shaded + background. For example: + </p> <pre class="programlisting">/* File: gettingstarted_common.h */ typedef struct stock_dbs { DB *inventory_dbp; /* Database containing inventory information */ @@ -107,10 +111,10 @@ typedef struct stock_dbs { } STOCK_DBS; </pre> <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> <h3 class="title">Note</h3> - <p> - Finally, notes of interest are represented using a note block such - as this. - </p> + <p> + Finally, notes of interest are represented using a + note block such as this. + </p> </div> </div> </div> diff --git a/docs/programmer_reference/program.html b/docs/programmer_reference/program.html index 9680f55b..59c18f99 100644 --- a/docs/programmer_reference/program.html +++ b/docs/programmer_reference/program.html @@ -14,13 +14,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 15. - Programmer Notes - </th> + <th colspan="3" align="center">Chapter 15. Programmer Notes </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="apprec_config.html">Prev</a> </td> @@ -34,9 +32,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="program"></a>Chapter 15. - Programmer Notes - </h2> + <h2 class="title"><a id="program"></a>Chapter 15. Programmer Notes </h2> </div> </div> </div> @@ -52,11 +48,36 @@ </dt> <dt> <span class="sect1"> - <a href="program_errorret.html">Error returns to applications</a> + <a href="program_errorret.html">Error returns to + applications</a> </span> </dt> <dt> <span class="sect1"> + <a href="program_i18n.html">Globalization Support</a> + </span> + </dt> + <dd> + <dl> + <dt> + <span class="sect2"> + <a href="program_i18n.html#idp2473976">Message Format</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="program_i18n.html#idp2493888">Enable Globalization Support</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="program_i18n.html#localization_example">Localization Example</a> + </span> + </dt> + </dl> + </dd> + <dt> + <span class="sect1"> <a href="program_environ.html">Environment variables</a> </span> </dt> @@ -79,19 +100,21 @@ <dl> <dt> <span class="sect2"> - <a href="program_namespace.html#idp2805992">C Language Name Space</a> + <a href="program_namespace.html#idp2522952">C Language Name + Space</a> </span> </dt> <dt> <span class="sect2"> - <a href="program_namespace.html#idp2778712">Filesystem Name Space</a> + <a href="program_namespace.html#idp2540280">Filesystem Name Space</a> </span> </dt> </dl> </dd> <dt> <span class="sect1"> - <a href="program_ram.html">Memory-only or Flash configurations</a> + <a href="program_ram.html">Memory-only or Flash + configurations</a> </span> </dt> <dt> @@ -116,7 +139,8 @@ </dt> <dt> <span class="sect1"> - <a href="program_perfmon.html">Performance Event Monitoring</a> + <a href="program_perfmon.html">Performance Event + Monitoring</a> </span> </dt> <dd> @@ -133,7 +157,8 @@ </dt> <dt> <span class="sect2"> - <a href="program_perfmon.html#program_perfmon_examples">Example Scripts</a> + <a href="program_perfmon.html#program_perfmon_examples">Example + Scripts</a> </span> </dt> <dt> @@ -158,22 +183,32 @@ </div> </div> </div> - <p>When applications using Berkeley DB receive signals, it is important that they -exit gracefully, discarding any Berkeley DB locks that they may hold. This is -normally done by setting a flag when a signal arrives and then checking -for that flag periodically within the application. Because Berkeley DB is not -re-entrant, the signal handler should not attempt to release locks and/or -close the database handles itself. Re-entering Berkeley DB is not guaranteed to -work correctly, and the results are undefined.</p> - <p>If an application exits holding a lock, the situation is no different -than if the application crashed, and all applications participating in -the database environment must be shut down, and then recovery must be -performed. If this is not done, databases may be left in an -inconsistent state, or locks the application held may cause unresolvable -deadlocks inside the environment, causing applications to hang.</p> - <p>Berkeley DB restarts all system calls interrupted by signals, that is, any -underlying system calls that return failure with errno set to EINTR will -be restarted rather than failing.</p> + <p> + When applications using Berkeley DB receive signals, it is + important that they exit gracefully, discarding any Berkeley + DB locks that they may hold. This is normally done by setting + a flag when a signal arrives and then checking for that flag + periodically within the application. Because Berkeley DB is + not re-entrant, the signal handler should not attempt to + release locks and/or close the database handles itself. + Re-entering Berkeley DB is not guaranteed to work correctly, + and the results are undefined. + </p> + <p> + If an application exits holding a lock, the situation is no + different than if the application crashed, and all + applications participating in the database environment must be + shut down, and then recovery must be performed. If this is not + done, databases may be left in an inconsistent state, or locks + the application held may cause unresolvable deadlocks inside + the environment, causing applications to hang. + </p> + <p> + Berkeley DB restarts all system calls interrupted by + signals, that is, any underlying system calls that return + failure with errno set to EINTR will be restarted rather than + failing. + </p> </div> </div> <div class="navfooter"> @@ -189,7 +224,8 @@ be restarted rather than failing.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Error returns to applications</td> + <td width="40%" align="right" valign="top"> Error returns to + applications</td> </tr> </table> </div> diff --git a/docs/programmer_reference/program_cache.html b/docs/programmer_reference/program_cache.html index 5bfbaf9e..a278318d 100644 --- a/docs/programmer_reference/program_cache.html +++ b/docs/programmer_reference/program_cache.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="program_ram.html">Prev</a> </td> - <th width="60%" align="center">Chapter 15. - Programmer Notes - </th> + <th width="60%" align="center">Chapter 15. Programmer Notes </th> <td width="20%" align="right"> <a accesskey="n" href="program_copy.html">Next</a></td> </tr> </table> @@ -38,19 +36,25 @@ </div> </div> </div> - <p>Many disk drives contain onboard caches. Some of these drives include -battery-backup or other functionality that guarantees that all cached -data will be completely written if the power fails. These drives can -offer substantial performance improvements over drives without caching -support. However, some caching drives rely on capacitors or other -mechanisms that guarantee only that the write of the current sector -will complete. These drives can endanger your database and potentially -cause corruption of your data.</p> - <p>To avoid losing your data, make sure the caching on your disk drives is -properly configured so the drive will never report that data has been written -unless the data is guaranteed to be written in the face of a power failure. -Many times, this means that write-caching on the disk drive must -be disabled.</p> + <p> + Many disk drives contain onboard caches. Some of these + drives include battery-backup or other functionality that + guarantees that all cached data will be completely written if + the power fails. These drives can offer substantial + performance improvements over drives without caching support. + However, some caching drives rely on capacitors or other + mechanisms that guarantee only that the write of the current + sector will complete. These drives can endanger your database + and potentially cause corruption of your data. + </p> + <p> + To avoid losing your data, make sure the caching on your + disk drives is properly configured so the drive will never + report that data has been written unless the data is + guaranteed to be written in the face of a power failure. Many + times, this means that write-caching on the disk drive must be + disabled. + </p> </div> <div class="navfooter"> <hr /> @@ -63,7 +67,8 @@ be disabled.</p> <td width="40%" align="right"> <a accesskey="n" href="program_copy.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Memory-only or Flash configurations </td> + <td width="40%" align="left" valign="top">Memory-only or Flash + configurations </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/program_compatible.html b/docs/programmer_reference/program_compatible.html index 5dc688d1..1d24d081 100644 --- a/docs/programmer_reference/program_compatible.html +++ b/docs/programmer_reference/program_compatible.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="program_copy.html">Prev</a> </td> - <th width="60%" align="center">Chapter 15. - Programmer Notes - </th> + <th width="60%" align="center">Chapter 15. Programmer Notes </th> <td width="20%" align="right"> <a accesskey="n" href="program_runtime.html">Next</a></td> </tr> </table> @@ -39,19 +37,21 @@ </div> </div> <p> - The Berkeley DB version 2 library provides backward-compatible - interfaces for the historic UNIX <a href="../api_reference/C/dbm.html" class="olink">dbm</a>, <a href="../api_reference/C/dbm.html" class="olink">ndbm</a> and <a href="../api_reference/C/hsearch.html" class="olink">hsearch</a> - interfaces. It also provides a backward-compatible interface for the - historic Berkeley DB 1.85 release. -</p> + The Berkeley DB version 2 library provides + backward-compatible interfaces for the historic UNIX <a href="../api_reference/C/dbm.html" class="olink">dbm</a>, + <a href="../api_reference/C/dbm.html" class="olink">ndbm</a> and <a href="../api_reference/C/hsearch.html" class="olink">hsearch</a> interfaces. It also provides a + backward-compatible interface for the historic Berkeley DB + 1.85 release. + </p> <p> - Berkeley DB version 2 does not provide database compatibility for any - of the previous interfaces, and existing databases must be converted - manually. To convert existing databases from the Berkeley DB 1.85 - format to the Berkeley DB version 2 format, review the <a href="../api_reference/C/db_dump.html" class="olink">db_dump185</a> utility and - the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility information. No utilities are provided to convert UNIX - <a href="../api_reference/C/dbm.html" class="olink">dbm</a>, <a href="../api_reference/C/dbm.html" class="olink">ndbm</a> or <a href="../api_reference/C/hsearch.html" class="olink">hsearch</a> databases. -</p> + Berkeley DB version 2 does not provide database + compatibility for any of the previous interfaces, and existing + databases must be converted manually. To convert existing + databases from the Berkeley DB 1.85 format to the Berkeley DB + version 2 format, review the <a href="../api_reference/C/db_dump.html" class="olink">db_dump185</a> utility and the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility + information. No utilities are provided to convert UNIX <a href="../api_reference/C/dbm.html" class="olink">dbm</a>, + <a href="../api_reference/C/dbm.html" class="olink">ndbm</a> or <a href="../api_reference/C/hsearch.html" class="olink">hsearch</a> databases. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/program_copy.html b/docs/programmer_reference/program_copy.html index 03baefb6..d9db8746 100644 --- a/docs/programmer_reference/program_copy.html +++ b/docs/programmer_reference/program_copy.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="program_cache.html">Prev</a> </td> - <th width="60%" align="center">Chapter 15. - Programmer Notes - </th> + <th width="60%" align="center">Chapter 15. Programmer Notes </th> <td width="20%" align="right"> <a accesskey="n" href="program_compatible.html">Next</a></td> </tr> </table> @@ -38,68 +36,96 @@ </div> </div> </div> - <p>There are two issues with copying or moving databases: database page log -sequence numbers (LSNs), and database file identification strings.</p> - <p>Because database pages contain references to the database environment -log records (LSNs), databases cannot be copied or moved from one -transactional database environment to another without first clearing the -LSNs. Note that this is not a concern for non-transactional database -environments and applications, and can be ignored if the database is not -being used transactionally. Specifically, databases created and written -non-transactionally (for example, as part of a bulk load procedure), can -be copied or moved into a transactional database environment without -resetting the LSNs. The database's LSNs may be reset in one of three -ways: the application can call the <a href="../api_reference/C/envlsn_reset.html" class="olink">DB_ENV->lsn_reset()</a> method to reset the -LSNs in place, or a system administrator can reset the LSNs in place -using the <span class="bold"><strong>-r</strong></span> option to the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility, or by -dumping and reloading the database (using the <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility and the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility).</p> - <p>Because system file identification information (for example, filenames, -device and inode numbers, volume and file IDs, and so on) are not -necessarily unique or maintained across system reboots, each Berkeley DB -database file contains a unique 20-byte file identification bytestring. -When multiple processes or threads open the same database file in Berkeley DB, -it is this bytestring that is used to ensure the same underlying pages -are updated in the database environment cache, no matter which Berkeley DB -handle is used for the operation.</p> - <p>The database file identification string is not a concern when moving -databases, and databases may be moved or renamed without resetting the -identification string. However, when copying a database, you must -ensure there are never two databases with the same file identification -bytestring in the same cache at the same time. Copying databases is -further complicated because Berkeley DB caches do not discard cached database -pages when database handles are closed. Cached pages are only discarded -when the database is removed by calling the <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> or -<a href="../api_reference/C/dbremove.html" class="olink">DB->remove()</a> methods.</p> - <p>Before physically copying a database file, first ensure that all -modified pages have been written from the cache to the backing database -file. This is done using the <a href="../api_reference/C/dbsync.html" class="olink">DB->sync()</a> or <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> methods.</p> - <p>Before using a copy of a database file in a database environment, you -must ensure that all pages from any other database with the same -bytestring have been removed from the memory pool cache. If the -environment in which you will open the copy of the database has pages -from files with identical bytestrings to the copied database, there are -a few possible solutions:</p> + <p> + There are two issues with copying or moving databases: + database page log sequence numbers (LSNs), and database file + identification strings. + </p> + <p> + Because database pages contain references to the database + environment log records (LSNs), databases cannot be copied or + moved from one transactional database environment to another + without first clearing the LSNs. Note that this is not a + concern for non-transactional database environments and + applications, and can be ignored if the database is not being + used transactionally. Specifically, databases created and + written non-transactionally (for example, as part of a bulk + load procedure), can be copied or moved into a transactional + database environment without resetting the LSNs. The + database's LSNs may be reset in one of three ways: the + application can call the <a href="../api_reference/C/envlsn_reset.html" class="olink">DB_ENV->lsn_reset()</a> method to reset the + LSNs in place, or a system administrator can reset the LSNs in + place using the <span class="bold"><strong>-r</strong></span> option to + the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility, or by dumping and reloading the database (using + the <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility and the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility). + </p> + <p> + Because system file identification information (for example, + filenames, device and inode numbers, volume and file IDs, and + so on) are not necessarily unique or maintained across system + reboots, each Berkeley DB database file contains a unique + 20-byte file identification bytestring. When multiple + processes or threads open the same database file in Berkeley + DB, it is this bytestring that is used to ensure the same + underlying pages are updated in the database environment + cache, no matter which Berkeley DB handle is used for the + operation. + </p> + <p> + The database file identification string is not a concern + when moving databases, and databases may be moved or renamed + without resetting the identification string. However, when + copying a database, you must ensure there are never two + databases with the same file identification bytestring in the + same cache at the same time. Copying databases is further + complicated because Berkeley DB caches do not discard cached + database pages when database handles are closed. Cached pages + are only discarded when the database is removed by calling the + <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> or <a href="../api_reference/C/dbremove.html" class="olink">DB->remove()</a> methods. + </p> + <p> + Before physically copying a database file, first ensure that + all modified pages have been written from the cache to the + backing database file. This is done using the <a href="../api_reference/C/dbsync.html" class="olink">DB->sync()</a> or + <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> methods. + </p> + <p> + Before using a copy of a database file in a database + environment, you must ensure that all pages from any other + database with the same bytestring have been removed from the + memory pool cache. If the environment in which you will open + the copy of the database has pages from files with identical + bytestrings to the copied database, there are a few possible + solutions: + </p> <div class="orderedlist"> <ol type="1"> - <li>Remove the environment, either using system utilities or by calling the -<a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> method. Obviously, this will not allow you to access -both the original database and the copy of the database at the same -time.</li> <li> - Create a new file that will have a new bytestring. The simplest way to - create a new file that will have a new bytestring is to call the - <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility to dump out the contents of the database and then use the - <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility to load the dumped output into a new file. This allows you - to access both the original and copy of the database at the same time. -</li> + Remove the environment, either using system + utilities or by calling the <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> method. Obviously, + this will not allow you to access both the original + database and the copy of the database at the same + time. + </li> <li> - If your database is too large to be dumped and reloaded, you can copy - the database by other means, and then reset the bytestring in the - copied database to a new bytestring. There are two ways to reset the - bytestring in the copy: the application can call the <a href="../api_reference/C/envfileid_reset.html" class="olink">DB_ENV->fileid_reset()</a> - method, or a system administrator can use the <span class="bold"><strong>-r</strong></span> option to the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility. This allows you - to access both the original and copy of the database at the same time. -</li> + Create a new file that will have a new bytestring. + The simplest way to create a new file that will have a new + bytestring is to call the <a href="../api_reference/C/db_dump.html" class="olink">db_dump</a> utility to dump out the + contents of the database and then use the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility to + load the dumped output into a new file. This allows you to + access both the original and copy of the database at the + same time. + </li> + <li> + If your database is too large to be dumped and + reloaded, you can copy the database by other means, and + then reset the bytestring in the copied database to a new + bytestring. There are two ways to reset the bytestring in + the copy: the application can call the <a href="../api_reference/C/envfileid_reset.html" class="olink">DB_ENV->fileid_reset()</a> + method, or a system administrator can use the <span class="bold"><strong>-r</strong></span> option to the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility. + This allows you to access both the original and copy of + the database at the same time. + </li> </ol> </div> </div> diff --git a/docs/programmer_reference/program_environ.html b/docs/programmer_reference/program_environ.html index cb9ff44e..027073ec 100644 --- a/docs/programmer_reference/program_environ.html +++ b/docs/programmer_reference/program_environ.html @@ -8,23 +8,21 @@ <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> <link rel="up" href="program.html" title="Chapter 15. Programmer Notes" /> - <link rel="prev" href="program_errorret.html" title="Error returns to applications" /> + <link rel="prev" href="program_i18n.html" title="Globalization Support" /> <link rel="next" href="program_mt.html" title="Multithreaded applications" /> </head> <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> <th colspan="3" align="center">Environment variables</th> </tr> <tr> - <td width="20%" align="left"><a accesskey="p" href="program_errorret.html">Prev</a> </td> - <th width="60%" align="center">Chapter 15. - Programmer Notes - </th> + <td width="20%" align="left"><a accesskey="p" href="program_i18n.html">Prev</a> </td> + <th width="60%" align="center">Chapter 15. Programmer Notes </th> <td width="20%" align="right"> <a accesskey="n" href="program_mt.html">Next</a></td> </tr> </table> @@ -38,23 +36,31 @@ </div> </div> </div> - <p>The Berkeley DB library uses the following environment variables:</p> + <p> + The Berkeley DB library uses the following environment + variables: + </p> <div class="variablelist"> <dl> <dt> <span class="term">DB_HOME</span> </dt> - <dd>If the environment variable DB_HOME is set, it is used as part of -<a class="xref" href="env_naming.html" title="File naming">File naming</a>. -Note: For the DB_HOME variable to take effect, either the -<a href="../api_reference/C/envopen.html#envopen_DB_USE_ENVIRON" class="olink">DB_USE_ENVIRON</a> or <a href="../api_reference/C/envopen.html#envopen_DB_USE_ENVIRON_ROOT" class="olink">DB_USE_ENVIRON_ROOT</a> flags must be -specified to <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a>.</dd> + <dd> + If the environment variable DB_HOME is set, it + is used as part of <a class="xref" href="env_naming.html" title="File naming">File naming</a>. Note: For the + DB_HOME variable to take effect, either the + <a href="../api_reference/C/envopen.html#envopen_DB_USE_ENVIRON" class="olink">DB_USE_ENVIRON</a> or <a href="../api_reference/C/envopen.html#envopen_DB_USE_ENVIRON_ROOT" class="olink">DB_USE_ENVIRON_ROOT</a> flags must + be specified to <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a>. + </dd> <dt> <span class="term">TMPDIR, TEMP, TMP, TempFolder</span> </dt> - <dd>The TMPDIR, TEMP, TMP, and TempFolder environment variables are all -checked as locations in which to create temporary files. See -<a href="../api_reference/C/envset_tmp_dir.html" class="olink">DB_ENV->set_tmp_dir()</a> for more information.</dd> + <dd> + The TMPDIR, TEMP, TMP, and TempFolder + environment variables are all checked as locations in + which to create temporary files. See <a href="../api_reference/C/envset_tmp_dir.html" class="olink">DB_ENV->set_tmp_dir()</a> + for more information. + </dd> </dl> </div> </div> @@ -62,14 +68,14 @@ checked as locations in which to create temporary files. See <hr /> <table width="100%" summary="Navigation footer"> <tr> - <td width="40%" align="left"><a accesskey="p" href="program_errorret.html">Prev</a> </td> + <td width="40%" align="left"><a accesskey="p" href="program_i18n.html">Prev</a> </td> <td width="20%" align="center"> <a accesskey="u" href="program.html">Up</a> </td> <td width="40%" align="right"> <a accesskey="n" href="program_mt.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Error returns to applications </td> + <td width="40%" align="left" valign="top">Globalization Support </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/program_errorret.html b/docs/programmer_reference/program_errorret.html index 2980806a..7e02b372 100644 --- a/docs/programmer_reference/program_errorret.html +++ b/docs/programmer_reference/program_errorret.html @@ -9,23 +9,22 @@ <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> <link rel="up" href="program.html" title="Chapter 15. Programmer Notes" /> <link rel="prev" href="program.html" title="Chapter 15. Programmer Notes" /> - <link rel="next" href="program_environ.html" title="Environment variables" /> + <link rel="next" href="program_i18n.html" title="Globalization Support" /> </head> <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Error returns to applications</th> + <th colspan="3" align="center">Error returns to + applications</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="program.html">Prev</a> </td> - <th width="60%" align="center">Chapter 15. - Programmer Notes - </th> - <td width="20%" align="right"> <a accesskey="n" href="program_environ.html">Next</a></td> + <th width="60%" align="center">Chapter 15. Programmer Notes </th> + <td width="20%" align="right"> <a accesskey="n" href="program_i18n.html">Next</a></td> </tr> </table> <hr /> @@ -34,148 +33,187 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="program_errorret"></a>Error returns to applications</h2> + <h2 class="title" style="clear: both"><a id="program_errorret"></a>Error returns to + applications</h2> </div> </div> </div> - <p> - Except for the historic <a href="../api_reference/C/dbm.html" class="olink">dbm</a>, <a href="../api_reference/C/dbm.html" class="olink">ndbm</a> and <a href="../api_reference/C/hsearch.html" class="olink">hsearch</a> interfaces, Berkeley DB - does not use the global variable <code class="literal">errno</code> to return - error values. The return values for all Berkeley DB functions are - grouped into the following three categories: -</p> + <p> + Except for the historic <a href="../api_reference/C/dbm.html" class="olink">dbm</a>, <a href="../api_reference/C/dbm.html" class="olink">ndbm</a> and <a href="../api_reference/C/hsearch.html" class="olink">hsearch</a> + interfaces, Berkeley DB does not use the global variable + <code class="literal">errno</code> to return error values. The + return values for all Berkeley DB functions are grouped into + the following three categories: + </p> <div class="variablelist"> <dl> <dt> <span class="term">0</span> </dt> - <dd>A return value of 0 indicates that the operation was successful.</dd> + <dd>A return value of 0 indicates that the operation + was successful.</dd> <dt> <span class="term">> 0</span> </dt> - <dd>A return value that is greater than 0 indicates that there was a system -error. The <span class="bold"><strong>errno</strong></span> value returned by the system is returned by -the function; for example, when a Berkeley DB function is unable to allocate -memory, the return value from the function will be ENOMEM.</dd> + <dd> + A return value that is greater than 0 indicates + that there was a system error. The <span class="bold"><strong>errno</strong></span> value returned by the + system is returned by the function; for example, when + a Berkeley DB function is unable to allocate memory, + the return value from the function will be + ENOMEM. + </dd> <dt> <span class="term">< 0</span> </dt> - <dd>A return value that is less than 0 indicates a condition that was not -a system failure, but was not an unqualified success, either. For -example, a routine to retrieve a key/data pair from the database may -return DB_NOTFOUND when the key/data pair does not appear in -the database; as opposed to the value of 0, which would be returned if -the key/data pair were found in the database. -<p> -All values returned by Berkeley DB functions are less than 0 in order to avoid -conflict with possible values of <span class="bold"><strong>errno</strong></span>. Specifically, Berkeley DB -reserves all values from -30,800 to -30,999 to itself as possible error -values. There are a few Berkeley DB interfaces where it is possible for an -application function to be called by a Berkeley DB function and subsequently -fail with an application-specific return. Such failure returns will be -passed back to the function that originally called a Berkeley DB interface. -To avoid ambiguity about the cause of the error, error values separate -from the Berkeley DB error name space should be used.</p></dd> + <dd> + A return value that is less than 0 indicates a + condition that was not a system failure, but was not + an unqualified success, either. For example, a routine + to retrieve a key/data pair from the database may + return DB_NOTFOUND when the key/data pair does not + appear in the database; as opposed to the value of 0, + which would be returned if the key/data pair were + found in the database. + <p> + All values returned by + Berkeley DB functions are less than 0 in order to + avoid conflict with possible values of <span class="bold"><strong>errno</strong></span>. Specifically, + Berkeley DB reserves all values from -30,800 to + -30,999 to itself as possible error values. There + are a few Berkeley DB interfaces where it is + possible for an application function to be called + by a Berkeley DB function and subsequently fail + with an application-specific return. Such failure + returns will be passed back to the function that + originally called a Berkeley DB interface. To + avoid ambiguity about the cause of the error, + error values separate from the Berkeley DB error + name space should be used. + </p></dd> </dl> </div> + <p> + Although possible error returns are specified by each + individual function's manual page, there are a few error + returns that deserve general mention: + </p> <p> - Although possible error returns are specified by each individual - function's manual page, there are a few error returns that deserve - general mention: -</p> + <span class="bold"><strong>DB_NOTFOUND and DB_KEYEMPTY</strong></span> + </p> <p> - <span class="bold"><strong>DB_NOTFOUND and DB_KEYEMPTY</strong></span> -</p> - <p>There are two special return values that are similar in meaning and that -are returned in similar situations, and therefore might be confused: -DB_NOTFOUND and DB_KEYEMPTY.</p> - <p><a id="program_errorret.DB_NOTFOUND"></a>The DB_NOTFOUND error return indicates that the requested key/data -pair did not exist in the database or that start-of- or end-of-file has -been reached by a cursor.</p> - <p><a id="program_errorret.DB_KEYEMPTY"></a>The DB_KEYEMPTY error return indicates that the requested -key/data pair logically exists but was never explicitly created by the -application (the Recno and Queue access methods will automatically -create key/data pairs under some circumstances; see <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> -for more information), or that the requested key/data pair was deleted -and never re-created. In addition, the Queue access method will return -DB_KEYEMPTY for records that were created as part of a -transaction that was later aborted and never re-created.</p> + There are two special return values that are similar in + meaning and that are returned in similar situations, and + therefore might be confused: DB_NOTFOUND and + DB_KEYEMPTY. + </p> + <p><a id="program_errorret.DB_NOTFOUND"></a> + The DB_NOTFOUND error + return indicates that the requested key/data pair did not + exist in the database or that start-of- or end-of-file has + been reached by a cursor. + </p> + <p><a id="program_errorret.DB_KEYEMPTY"></a> + The DB_KEYEMPTY error + return indicates that the requested key/data pair logically + exists but was never explicitly created by the application + (the Recno and Queue access methods will automatically create + key/data pairs under some circumstances; see <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> for more + information), or that the requested key/data pair was deleted + and never re-created. In addition, the Queue access method + will return DB_KEYEMPTY for records that were created as part + of a transaction that was later aborted and never + re-created. + </p> <p> - <span class="bold"><strong>DB_KEYEXIST</strong></span> -</p> - <p><a id="program_errorret.DB_KEYEXIST"></a> - The DB_KEYEXIST error return indicates the <a href="../api_reference/C/dbput.html#put_DB_NOOVERWRITE" class="olink">DB_NOOVERWRITE</a> option was specified - when inserting a key/data pair into the database and the key already - exists in the database, or the <a href="../api_reference/C/dbput.html#put_DB_NODUPDATA" class="olink">DB_NODUPDATA</a> option was specified and the - key/data pair already exists in the data. -</p> + <span class="bold"><strong>DB_KEYEXIST</strong></span> + </p> + <p><a id="program_errorret.DB_KEYEXIST"></a> + The DB_KEYEXIST error + return indicates the <a href="../api_reference/C/dbput.html#put_DB_NOOVERWRITE" class="olink">DB_NOOVERWRITE</a> option was specified + when inserting a key/data pair into the database and the key + already exists in the database, or the <a href="../api_reference/C/dbput.html#put_DB_NODUPDATA" class="olink">DB_NODUPDATA</a> option + was specified and the key/data pair already exists in the + data. + </p> <p> - <span class="bold"><strong>DB_LOCK_DEADLOCK</strong></span> -</p> + <span class="bold"><strong>DB_LOCK_DEADLOCK</strong></span> + </p> <p><a id="program_errorret.DB_LOCK_DEADLOCK"></a> - When multiple threads of control are modifying the database, there is - normally the potential for deadlock. In Berkeley DB, deadlock is - signified by an error return from the Berkeley DB function of the value - DB_LOCK_DEADLOCK. Whenever a Berkeley DB function returns - DB_LOCK_DEADLOCK, the enclosing transaction should be aborted. -</p> - <p> - Any Berkeley DB function that attempts to acquire locks can potentially - return DB_LOCK_DEADLOCK. Practically speaking, the safest way to deal - with applications that can deadlock is to anticipate a DB_LOCK_DEADLOCK - return from any <a href="../api_reference/C/db.html" class="olink">DB</a> or <a href="../api_reference/C/dbc.html" class="olink">DBC</a> handle method call, or any <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> - handle method call that references a database, including the database's - backing physical file. -</p> + When multiple + threads of control are modifying the database, there is + normally the potential for deadlock. In Berkeley DB, deadlock + is signified by an error return from the Berkeley DB function + of the value DB_LOCK_DEADLOCK. Whenever a Berkeley DB function + returns DB_LOCK_DEADLOCK, the enclosing transaction should be + aborted. + </p> + <p> + Any Berkeley DB function that attempts to acquire locks can + potentially return DB_LOCK_DEADLOCK. Practically speaking, the + safest way to deal with applications that can deadlock is to + anticipate a DB_LOCK_DEADLOCK return from any <a href="../api_reference/C/db.html" class="olink">DB</a> or <a href="../api_reference/C/dbc.html" class="olink">DBC</a> + handle method call, or any <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handle method call that + references a database, including the database's backing + physical file. + </p> <p> - <span class="bold"><strong>DB_LOCK_NOTGRANTED</strong></span> -</p> + <span class="bold"><strong>DB_LOCK_NOTGRANTED</strong></span> + </p> <p><a id="program_errorret.DB_LOCK_NOTGRANTED"></a> - If a lock is requested from the <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a> or <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> methods with the - <a href="../api_reference/C/lockvec.html#vec_DB_LOCK_NOWAIT" class="olink">DB_LOCK_NOWAIT</a> flag specified, the method will return DB_LOCK_NOTGRANTED if the lock - is not immediately available. -</p> - <p> - If the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TIME_NOTGRANTED" class="olink">DB_TIME_NOTGRANTED</a> flag is specified to the <a href="../api_reference/C/envset_flags.html" class="olink">DB_ENV->set_flags()</a> method, - database calls timing out based on lock or transaction timeout values - will return DB_LOCK_NOTGRANTED instead of DB_LOCK_DEADLOCK. -</p> + If a lock is + requested from the <a href="../api_reference/C/lockget.html" class="olink">DB_ENV->lock_get()</a> or <a href="../api_reference/C/lockvec.html" class="olink">DB_ENV->lock_vec()</a> methods with the + <a href="../api_reference/C/lockvec.html#vec_DB_LOCK_NOWAIT" class="olink">DB_LOCK_NOWAIT</a> flag specified, the method will return + DB_LOCK_NOTGRANTED if the lock is not immediately available. + </p> + <p> + If the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TIME_NOTGRANTED" class="olink">DB_TIME_NOTGRANTED</a> flag is specified to the + <a href="../api_reference/C/envset_flags.html" class="olink">DB_ENV->set_flags()</a> method, database calls timing out based on lock + or transaction timeout values will return DB_LOCK_NOTGRANTED + instead of DB_LOCK_DEADLOCK. + </p> <p> - <span class="bold"><strong>DB_RUNRECOVERY</strong></span> -</p> + <span class="bold"><strong>DB_RUNRECOVERY</strong></span> + </p> <p><a id="program_errorret.DB_RUNRECOVERY"></a> - There exists a class of errors that Berkeley DB considers fatal to an - entire Berkeley DB environment. An example of this type of error is a - corrupted database page. The only way to recover from these failures - is to have all threads of control exit the Berkeley DB environment, run - recovery of the environment, and re-enter Berkeley DB. (It is not - strictly necessary that the processes exit, although that is the only - way to recover system resources, such as file descriptors and memory, - allocated by Berkeley DB.) -</p> - <p> - When this type of error is encountered, the error value DB_RUNRECOVERY - is returned. This error can be returned by any Berkeley DB interface. - Once DB_RUNRECOVERY is returned by any interface, it will be returned - from all subsequent Berkeley DB calls made by any threads of control - participating in the environment. -</p> - <p> - Applications can handle such fatal errors in one of two ways: first, by - checking for DB_RUNRECOVERY as part of their normal Berkeley DB error - return checking, similarly to DB_LOCK_DEADLOCK or any other error. - Alternatively, applications can specify a fatal-error callback function - using the <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> method. Applications with no cleanup - processing of their own should simply exit from the callback function. -</p> + There exists a class + of errors that Berkeley DB considers fatal to an entire + Berkeley DB environment. An example of this type of error is a + corrupted database page. The only way to recover from these + failures is to have all threads of control exit the Berkeley + DB environment, run recovery of the environment, and re-enter + Berkeley DB. (It is not strictly necessary that the processes + exit, although that is the only way to recover system + resources, such as file descriptors and memory, allocated by + Berkeley DB.) + </p> + <p> + When this type of error is encountered, the error value + DB_RUNRECOVERY is returned. This error can be returned by any + Berkeley DB interface. Once DB_RUNRECOVERY is returned by any + interface, it will be returned from all subsequent Berkeley DB + calls made by any threads of control participating in the + environment. + </p> + <p> + Applications can handle such fatal errors in one of two + ways: first, by checking for DB_RUNRECOVERY as part of their + normal Berkeley DB error return checking, similarly to + DB_LOCK_DEADLOCK or any other error. Alternatively, + applications can specify a fatal-error callback function using + the <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> method. Applications with no cleanup + processing of their own should simply exit from the callback + function. + </p> <p> - <span class="bold"><strong>DB_SECONDARY_BAD</strong></span> -</p> + <span class="bold"><strong>DB_SECONDARY_BAD</strong></span> + </p> <p><a id="program_error_ret.DB_SECONDARY_BAD"></a> - The DB_SECONDARY_BAD error is returned if a secondary index has been - corrupted. This may be the result of an application operating on - related databases without first associating them. -</p> + The + DB_SECONDARY_BAD error is returned if a secondary index has + been corrupted. This may be the result of an application + operating on related databases without first associating them. + </p> </div> <div class="navfooter"> <hr /> @@ -185,16 +223,14 @@ transaction that was later aborted and never re-created.</p> <td width="20%" align="center"> <a accesskey="u" href="program.html">Up</a> </td> - <td width="40%" align="right"> <a accesskey="n" href="program_environ.html">Next</a></td> + <td width="40%" align="right"> <a accesskey="n" href="program_i18n.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 15. - Programmer Notes - </td> + <td width="40%" align="left" valign="top">Chapter 15. Programmer Notes </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Environment variables</td> + <td width="40%" align="right" valign="top"> Globalization Support</td> </tr> </table> </div> diff --git a/docs/programmer_reference/program_faq.html b/docs/programmer_reference/program_faq.html index 39628368..c15e4ff1 100644 --- a/docs/programmer_reference/program_faq.html +++ b/docs/programmer_reference/program_faq.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="program_perfmon.html">Prev</a> </td> - <th width="60%" align="center">Chapter 15. - Programmer Notes - </th> + <th width="60%" align="center">Chapter 15. Programmer Notes </th> <td width="20%" align="right"> <a accesskey="n" href="lock.html">Next</a></td> </tr> </table> @@ -41,45 +39,80 @@ <div class="orderedlist"> <ol type="1"> <li> - <span class="bold"> - <strong>What priorities should threads/tasks executing Berkeley DB functions -be given?</strong> - </span> - <p>Tasks executing Berkeley DB functions should have the same, or roughly -equivalent, system priorities. For example, it can be dangerous to give -tasks of control performing checkpoints a lower priority than tasks of -control doing database lookups, and starvation can sometimes result.</p> + <p> + <span class="bold"><strong>What priorities should + threads/tasks executing Berkeley DB functions be + given?</strong></span> + </p> + <p> + Tasks executing Berkeley DB functions should have + the same, or roughly equivalent, system priorities. + For example, it can be dangerous to give tasks of + control performing checkpoints a lower priority than + tasks of control doing database lookups, and + starvation can sometimes result. + </p> </li> <li> - <span class="bold"> - <strong>Why isn't the C++ API exception safe?</strong> - </span> - <p>The Berkeley DB C++ API is a thin wrapper around the C API that maps most -return values to exceptions, and gives the C++ handles the same -lifecycles as their C counterparts. One consequence is that if an -exception occurs while a cursor or transaction handle is open, the -application must explicitly close the cursor or abort the transaction.</p> - <p>Applications can be simplified and bugs avoided by creating wrapper -classes around <a href="../api_reference/C/dbc.html" class="olink">DBC</a> and <a href="../api_reference/C/txn.html" class="olink">TXN</a> that call the appropriate -cleanup method in the wrapper's destructor. By creating an instance -of the wrappers on the stack, C++ scoping rules will ensure that the -destructor is called before exception handling unrolls the block that -contains the wrapper object.</p> + <p> + <span class="bold"><strong>Why isn't the C++ API exception + safe?</strong></span> + </p> + <p> + The Berkeley DB C++ API is a thin wrapper around + the C API that maps most return values to exceptions, + and gives the C++ handles the same lifecycles as their + C counterparts. One consequence is that if an + exception occurs while a cursor or transaction handle + is open, the application must explicitly close the + cursor or abort the transaction. + </p> + <p> + Applications can be simplified and bugs avoided by + creating wrapper classes around <a href="../api_reference/C/dbc.html" class="olink">DBC</a> and <a href="../api_reference/C/txn.html" class="olink">TXN</a> that + call the appropriate cleanup method in the wrapper's + destructor. By creating an instance of the wrappers on + the stack, C++ scoping rules will ensure that the + destructor is called before exception handling unrolls + the block that contains the wrapper object. + </p> + </li> + <li> + <p> + <span class="bold"><strong> How do I handle a "pass 4" + error when trying to run one of the example + Performance Event Monitoring scripts on my Linux + system? The library was configured with + <code class="literal">--enable-dtrace</code> and built + without error. </strong></span> + </p> + <p> + A Linux installation can have SystemTap support for + kernel probe points without including the kernel + "utrace" module needed to use userspace probes. Pass 4 + errors can occur when this required userspace support + is not present. + </p> </li> <li> - <span class="bold"> - <strong> - How do I handle a "pass 4" error when trying to run one of the - example Performance Event Monitoring scripts on my Linux system? - The library configured with --enable-dtrace and built without error. - </strong> - </span> <p> - A Linux installation can have SystemTap support for kernel probe points - without including the kernel "utrace" module needed to use userspace - probes. Pass 4 errors can occur when this required userspace - support is not present. - </p> + <span class="bold"><strong> I have a program with multiple + threads running on multi-core systems, and it uses + a shared DB handle for all the threads. The + program does not behave as well as I expect and I + see a lot of contention on the dbp->mutex, what + should I do to reduce this contention ? + </strong></span> + </p> + <p> + When running multi-threaded program on a + multi-core/multi-processor system, we suggest using a + DB handle per thread instead of using a shared handle + accorss threads. Since many operations(like creating + cursors) requires to lock the DB handle first, and if + using a shared handle, there could be a lot of lock + contention on the handle. + </p> </li> </ol> </div> @@ -95,13 +128,12 @@ contains the wrapper object.</p> <td width="40%" align="right"> <a accesskey="n" href="lock.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Performance Event Monitoring </td> + <td width="40%" align="left" valign="top">Performance Event + Monitoring </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 16. - The Locking Subsystem - </td> + <td width="40%" align="right" valign="top"> Chapter 16. The Locking Subsystem </td> </tr> </table> </div> diff --git a/docs/programmer_reference/program_i18n.html b/docs/programmer_reference/program_i18n.html new file mode 100644 index 00000000..8ce738e8 --- /dev/null +++ b/docs/programmer_reference/program_i18n.html @@ -0,0 +1,302 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> +<html xmlns="http://www.w3.org/1999/xhtml"> + <head> + <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> + <title>Globalization Support</title> + <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> + <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> + <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> + <link rel="up" href="program.html" title="Chapter 15. Programmer Notes" /> + <link rel="prev" href="program_errorret.html" title="Error returns to applications" /> + <link rel="next" href="program_environ.html" title="Environment variables" /> + </head> + <body> + <div xmlns="" class="navheader"> + <div class="libver"> + <p>Library Version 12.1.6.1</p> + </div> + <table width="100%" summary="Navigation header"> + <tr> + <th colspan="3" align="center">Globalization Support</th> + </tr> + <tr> + <td width="20%" align="left"><a accesskey="p" href="program_errorret.html">Prev</a> </td> + <th width="60%" align="center">Chapter 15. Programmer Notes </th> + <td width="20%" align="right"> <a accesskey="n" href="program_environ.html">Next</a></td> + </tr> + </table> + <hr /> + </div> + <div class="sect1" lang="en" xml:lang="en"> + <div class="titlepage"> + <div> + <div> + <h2 class="title" style="clear: both"><a id="program_i18n"></a>Globalization Support</h2> + </div> + </div> + </div> + <div class="toc"> + <dl> + <dt> + <span class="sect2"> + <a href="program_i18n.html#idp2473976">Message Format</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="program_i18n.html#idp2493888">Enable Globalization Support</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="program_i18n.html#localization_example">Localization Example</a> + </span> + </dt> + </dl> + </div> + <p> + Berkeley DB globalization support allows you to translate + error and informational message text to the language of your + choosing, and then use the translated text instead of the + default English text. This section describes Berkeley DB's + globalization support. Berkeley DB's error and informational + message text is captured in the <span> + <a class="ulink" href="http://docs.oracle.com/cd/E17076_02/html/articles/mssgtxt/index.html" target="_top">Berkeley DB Message Reference Guide</a>. + </span> + + </p> + <div class="sect2" lang="en" xml:lang="en"> + <div class="titlepage"> + <div> + <div> + <h3 class="title"><a id="idp2473976"></a>Message Format</h3> + </div> + </div> + </div> + <p> + By default, Berkeley DB messages are comprised of a + message number followed by message text in English. For + example: + </p> + <pre class="programlisting">BDB1001 illegal record number size</pre> + <p> + It is possible to build Berkeley DB with stripped + messages. When messages are stripped, the message text is + removed from the library, leaving behind only the message + number. When building a stripped library, there is no + message text available so localization will not work. + </p> + <p> + If localization is enabled, the translated message is + substituted for the original message text. + </p> + </div> + <div class="sect2" lang="en" xml:lang="en"> + <div class="titlepage"> + <div> + <div> + <h3 class="title"><a id="idp2493888"></a>Enable Globalization Support</h3> + </div> + </div> + </div> + <p> + To output messages in a language other than the default + English, follow the steps below: + </p> + <div class="orderedlist"> + <ol type="1"> + <li> + <p> + Provide an i18n component containing a + localization function used to translate messages, + and translation files that map existing messages + to localized messages. The localization function + can be added to the current Berkeley DB project, + or as a dynamic library that is called at run + time. + </p> + </li> + <li> + <p> + Add the name of the localization function as + the prefix for "(msg)" when + <code class="literal">HAVE_LOCALIZATION</code> is + defined in <code class="literal">build_unix/db_int.in</code> + on *nux, or in + <code class="literal">build_windows/db_int.h</code> on + Windows. + </p> + </li> + <li> + <p> + On *nix, specify + <code class="literal">-DHAVE_LOCALIZATION</code> to + CFLAGS. On Windows, specify <code class="literal">/D + HAVE_LOCALIZATION</code> to the + <code class="literal">C/C++ Additional Options</code> in + the db project properties. + </p> + </li> + <li> + <p> + Within your application code, use + <a href="../api_reference/C/envset_errcall.html" class="olink">DB_ENV->set_errcall()</a> or <a href="../api_reference/C/dbset_errcall.html" class="olink">DB->set_errcall()</a> to print the + messages. + </p> + </li> + </ol> + </div> + <p> + Note that Berkeley DB supports only UTF-8 for its + message text. If your localization requires UTF-16 + Unicode, the UTF-16 characters must be converted to UTF-8 + Unicode by your localization function. If necessary, the + error reporting function you specify to <a href="../api_reference/C/envset_errcall.html" class="olink">DB_ENV->set_errcall()</a> + or <a href="../api_reference/C/dbset_errcall.html" class="olink">DB->set_errcall()</a> can be used to revert the UTF-8 Unicode + back to the UTF-16 Unicode. + </p> + </div> + <div class="sect2" lang="en" xml:lang="en"> + <div class="titlepage"> + <div> + <div> + <h3 class="title"><a id="localization_example"></a>Localization Example</h3> + </div> + </div> + </div> + <p> + The following example walks you through providing + localization support for a single Berkeley DB error + message: + </p> + <div class="itemizedlist"> + <ul type="disc"> + <li> + <p> + Make the resource bundles. These provide the + actual text translation: + </p> + <p> + es.txt: + </p> + <pre class="programlisting">es { +BDB1002 illegal record number of 0 {"BDB1002 illegal record number of 0"} +} </pre> + <p> + de_CH.txt: + </p> + <pre class="programlisting">de_CH { +BDB1002 illegal record number of 0 {"BDB1002 illegale Rekordzahl von 0"} +} </pre> + <p> + root.txt: + </p> + <pre class="programlisting">root { +BDB1002 illegal record number of 0 {"BDB1002 illegal record number of 0"} +} </pre> + </li> + <li> + <p> + Write and compile your localization functions: + Note that the "es", "de_CH" and "root" tags are + used as the locale name in + <code class="literal">ures_open()</code>. + </p> + <p> + Also notice that because + <code class="literal">ures_getStringByKey()</code> + returns UTF-16 Unicode, its output is converted to + UTF-8 using <code class="literal">ucnv_fromUChars()</code>. + </p> + <pre class="programlisting">UConverter *conv; +UResourceBundle *rhandle; +char *mbuf; + +initialize() { + /* Open ICU resource, specify the locale. */ + rhandle = ures_open(resource, "de_CH", &status); + /* Open an ICU converter. */ + conv = ucnv_open("iso-8859-3", &status); + mbuf = malloc(len * sizeof(char)); + memset(mbuf, 0, 100 * sizeof(char)); +} + +translate() { + const UChar *wmsg; + /* Get the translated message from the resource. */ + wmsg = ures_getStringByKey(rhandle, src, &len, &status); + /* Convert UChar * to char. */ + len = ucnv_fromUChars(conv, wmsg, 100, , -1, &status); +} + +close() { + ucnv_close(conv); + ures_close(rhandle); + free(mbuf); +} </pre> + </li> + <li> + <p> + Update <code class="literal">db_int.h</code> so that + <code class="literal">_(msg)</code> is defined to use + the <code class="literal">translate()</code> that we created + in the previous step. + </p> + <pre class="programlisting">#ifdef HAVE_LOCALIZATION +#define _(msg) translate(msg) +#else +#define _(msg) msg +#endif </pre> + </li> + <li> + <p> + Rebuild Berkeley DB, making sure to specify the + <code class="literal">HAVE_LOCALIZATION</code> compile + option. + </p> + </li> + <li> + <p> + Specify the error callback. + </p> + <pre class="programlisting">dbp->set_errcall(dbp, print_err); + +print_err() { + const UChar *wmsg; + len = ucnv_toUChars(conv, wmsg, 100, src, len, &status); + u_stdout = u_finit(stdout, NULL, NULL); + u_file_write((UChar *)wmsg, len, u_stdout); +} </pre> + </li> + </ul> + </div> + <p> + The result of this is if you input an incorrect recno + and reach the error 1002, the message "BDB1002 illegale + Rekordzahl von 0" is output. + </p> + </div> + </div> + <div class="navfooter"> + <hr /> + <table width="100%" summary="Navigation footer"> + <tr> + <td width="40%" align="left"><a accesskey="p" href="program_errorret.html">Prev</a> </td> + <td width="20%" align="center"> + <a accesskey="u" href="program.html">Up</a> + </td> + <td width="40%" align="right"> <a accesskey="n" href="program_environ.html">Next</a></td> + </tr> + <tr> + <td width="40%" align="left" valign="top">Error returns to + applications </td> + <td width="20%" align="center"> + <a accesskey="h" href="index.html">Home</a> + </td> + <td width="40%" align="right" valign="top"> Environment variables</td> + </tr> + </table> + </div> + </body> +</html> diff --git a/docs/programmer_reference/program_mt.html b/docs/programmer_reference/program_mt.html index 7c8c4443..4bb44a64 100644 --- a/docs/programmer_reference/program_mt.html +++ b/docs/programmer_reference/program_mt.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="program_environ.html">Prev</a> </td> - <th width="60%" align="center">Chapter 15. - Programmer Notes - </th> + <th width="60%" align="center">Chapter 15. Programmer Notes </th> <td width="20%" align="right"> <a accesskey="n" href="program_scope.html">Next</a></td> </tr> </table> @@ -38,73 +36,115 @@ </div> </div> </div> - <p>Berkeley DB fully supports multithreaded applications. The Berkeley DB library is -not itself multithreaded, and was deliberately architected to not use -threads internally because of the portability problems that would -introduce. Database environment and database object handles returned -from Berkeley DB library functions are free-threaded. No other object handles -returned from the Berkeley DB library are free-threaded. The following rules -should be observed when using threads to access the Berkeley DB library:</p> + <p> + Berkeley DB fully supports multithreaded applications. The + Berkeley DB library is not itself multithreaded, and was + deliberately architected to not use threads internally because + of the portability problems that would introduce. Database + environment and database object handles returned from Berkeley + DB library functions are free-threaded. No other object + handles returned from the Berkeley DB library are + free-threaded. The following rules should be observed when + using threads to access the Berkeley DB library: + </p> <div class="orderedlist"> <ol type="1"> <li> - <p>The <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flag must be specified to the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> -and <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> methods if the Berkeley DB handles returned by those interfaces -will be used in the context of more than one thread. Setting the -<a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flag inconsistently may result in database corruption. -</p> - <p>Threading is assumed in the Java API, so no special flags are required; -and Berkeley DB functions will always behave as if the <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flag -was specified.</p> - <p>Only a single thread may call the <a href="../api_reference/C/envclose.html" class="olink">DB_ENV->close()</a> or <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> methods -for a returned environment or database handle.</p> - <p>No other Berkeley DB handles are free-threaded.</p> + <p> + The <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flag must be specified to the + <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> and <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> methods if the Berkeley DB + handles returned by those interfaces will be used in + the context of more than one thread. Setting the + <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flag inconsistently may result in database + corruption. + </p> + <p> + Threading is assumed in the Java API, so no special + flags are required; and Berkeley DB functions will + always behave as if the <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flag was + specified. + </p> + <p> + Only a single thread may call the <a href="../api_reference/C/envclose.html" class="olink">DB_ENV->close()</a> or + <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> methods for a returned environment or + database handle. + </p> + <p> + No other Berkeley DB handles are + free-threaded. + </p> </li> <li> + <p> + When using the non-cursor Berkeley DB calls to + retrieve key/data items (for example, <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a>), the + memory to which the pointer stored into the Dbt refers + is valid only until the next call using the <a href="../api_reference/C/db.html" class="olink">DB</a> + handle returned by <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a>. This includes <span class="bold"><strong>any</strong></span> use of the returned + <a href="../api_reference/C/db.html" class="olink">DB</a> handle, including by another thread within the + process. + </p> <p> - When using the non-cursor Berkeley DB calls to retrieve key/data items - (for example, <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a>), the memory to which the pointer stored into the - Dbt refers is valid only until the next call using the <a href="../api_reference/C/db.html" class="olink">DB</a> - handle returned by <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a>. This includes <span class="bold"><strong>any</strong></span> use of the returned <a href="../api_reference/C/db.html" class="olink">DB</a> handle, - including by another thread within the process. - </p> - <p>For this reason, if the <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> handle was specified to the -<a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> method, either <a href="../api_reference/C/dbt.html#dbt_DB_DBT_MALLOC" class="olink">DB_DBT_MALLOC</a>, <a href="../api_reference/C/dbt.html#dbt_DB_DBT_REALLOC" class="olink">DB_DBT_REALLOC</a> or <a href="../api_reference/C/dbt.html#dbt_DB_DBT_USERMEM" class="olink">DB_DBT_USERMEM</a> -must be specified in the <a href="../api_reference/C/dbt.html" class="olink">DBT</a> when -performing any non-cursor key or data retrieval.</p> + For this reason, if the <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> handle was + specified to the <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> method, either + <a href="../api_reference/C/dbt.html#dbt_DB_DBT_MALLOC" class="olink">DB_DBT_MALLOC</a>, <a href="../api_reference/C/dbt.html#dbt_DB_DBT_REALLOC" class="olink">DB_DBT_REALLOC</a> or <a href="../api_reference/C/dbt.html#dbt_DB_DBT_USERMEM" class="olink">DB_DBT_USERMEM</a> + must be specified in the <a href="../api_reference/C/dbt.html" class="olink">DBT</a> when performing any + non-cursor key or data retrieval. + </p> </li> <li> - <p>Cursors may not span transactions. Each cursor must be - allocated and deallocated within the same transaction.</p> - <p>Transactions and cursors may span threads, but only serially, that is, -the application must serialize access to the <a href="../api_reference/C/txn.html" class="olink">TXN</a> and -<a href="../api_reference/C/dbc.html" class="olink">DBC</a> handles. In the case of nested transactions, since all -child transactions are part of the same parent transaction, they must observe -the same constraints. That is, children may execute in different threads -only if each child executes serially.</p> + <p> + Cursors may not span transactions. Each cursor must + be allocated and deallocated within the same + transaction. + </p> + <p> + Transactions and cursors may span threads, but only + serially, that is, the application must serialize + access to the <a href="../api_reference/C/txn.html" class="olink">TXN</a> and <a href="../api_reference/C/dbc.html" class="olink">DBC</a> handles. In the case of + nested transactions, since all child transactions are + part of the same parent transaction, they must observe + the same constraints. That is, children may execute in + different threads only if each child executes + serially. + </p> </li> <li> - <p>User-level synchronization mutexes must have been implemented for the -compiler/architecture combination. Attempting to specify the DB_THREAD -flag will fail if fast mutexes are not available. -</p> - <p>If blocking mutexes are available (for example POSIX pthreads), they -will be used. Otherwise, the Berkeley DB library will make a system call to -pause for some amount of time when it is necessary to wait on a lock. -This may not be optimal, especially in a thread-only environment, in -which it is usually more efficient to explicitly yield the processor to -another thread.</p> - <p>It is possible to specify a yield function on an per-application basis. -See <a href="../api_reference/C/db_env_set_func_yield.html" class="olink">db_env_set_func_yield</a> for more information.</p> - <p>It is possible to specify the number of attempts that will be made to -acquire the mutex before waiting. See <a href="../api_reference/C/mutexset_tas_spins.html" class="olink">DB_ENV->mutex_set_tas_spins()</a> for -more information.</p> + <p> + User-level synchronization mutexes must have been + implemented for the compiler/architecture combination. + Attempting to specify the DB_THREAD flag will fail if + fast mutexes are not available. + </p> + <p> + If blocking mutexes are available (for example POSIX + pthreads), they will be used. Otherwise, the Berkeley + DB library will make a system call to pause for some + amount of time when it is necessary to wait on a lock. + This may not be optimal, especially in a thread-only + environment, in which it is usually more efficient to + explicitly yield the processor to another + thread. + </p> + <p> + It is possible to specify a yield function on an + per-application basis. See <a href="../api_reference/C/db_env_set_func_yield.html" class="olink">db_env_set_func_yield</a> for more + information. + </p> + <p> + It is possible to specify the number of attempts + that will be made to acquire the mutex before waiting. + See <a href="../api_reference/C/mutexset_tas_spins.html" class="olink">DB_ENV->mutex_set_tas_spins()</a> for more information. + </p> </li> </ol> </div> - <p>When creating multiple databases in a single physical file, multithreaded -programs may have additional requirements. For more information, see -<a class="xref" href="am_opensub.html" title="Opening multiple databases in a single file">Opening multiple databases in a single file</a></p> + <p> + When creating multiple databases in a single physical file, + multithreaded programs may have additional requirements. For + more information, see <a class="xref" href="am_opensub.html" title="Opening multiple databases in a single file">Opening multiple databases in a + single file</a> + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/program_namespace.html b/docs/programmer_reference/program_namespace.html index 46cedbe4..8363cd50 100644 --- a/docs/programmer_reference/program_namespace.html +++ b/docs/programmer_reference/program_namespace.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="program_scope.html">Prev</a> </td> - <th width="60%" align="center">Chapter 15. - Programmer Notes - </th> + <th width="60%" align="center">Chapter 15. Programmer Notes </th> <td width="20%" align="right"> <a accesskey="n" href="program_ram.html">Next</a></td> </tr> </table> @@ -42,12 +40,13 @@ <dl> <dt> <span class="sect2"> - <a href="program_namespace.html#idp2805992">C Language Name Space</a> + <a href="program_namespace.html#idp2522952">C Language Name + Space</a> </span> </dt> <dt> <span class="sect2"> - <a href="program_namespace.html#idp2778712">Filesystem Name Space</a> + <a href="program_namespace.html#idp2540280">Filesystem Name Space</a> </span> </dt> </dl> @@ -56,50 +55,75 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2805992"></a>C Language Name Space</h3> + <h3 class="title"><a id="idp2522952"></a>C Language Name + Space</h3> </div> </div> </div> - <p>The Berkeley DB library is careful to avoid C language programmer name spaces, -but there are a few potential areas for concern, mostly in the Berkeley DB -include file db.h. The db.h include file defines a number of types and -strings. Where possible, all of these types and strings are prefixed with -"DB_" or "db_". There are a few notable exceptions.</p> - <p>The Berkeley DB library uses a macro named "__P" to configure for systems that -do not provide ANSI C function prototypes. This could potentially collide -with other systems using a "__P" macro for similar or different purposes.</p> - <p>The Berkeley DB library needs information about specifically sized types for -each architecture. If they are not provided by the system, they are -typedef'd in the db.h include file. The types that may be typedef'd -by db.h include the following: u_int8_t, int16_t, u_int16_t, int32_t, -u_int32_t, u_char, u_short, u_int, and u_long.</p> - <p>The Berkeley DB library declares a few external routines. All these routines -are prefixed with the strings "db_". All internal Berkeley DB routines are -prefixed with the strings "__XXX_", where "XXX" is the subsystem prefix -(for example, "__db_XXX_" and "__txn_XXX_").</p> + <p> + The Berkeley DB library is careful to avoid C language + programmer name spaces, but there are a few potential + areas for concern, mostly in the Berkeley DB include file + db.h. The db.h include file defines a number of types and + strings. Where possible, all of these types and strings + are prefixed with "DB_" or "db_". There are a few notable + exceptions. + </p> + <p> + The Berkeley DB library uses a macro named "__P" to + configure for systems that do not provide ANSI C function + prototypes. This could potentially collide with other + systems using a "__P" macro for similar or different + purposes. + </p> + <p> + The Berkeley DB library needs information about + specifically sized types for each architecture. If they + are not provided by the system, they are typedef'd in the + db.h include file. The types that may be typedef'd by db.h + include the following: u_int8_t, int16_t, u_int16_t, + int32_t, u_int32_t, u_char, u_short, u_int, and + u_long. + </p> + <p> + The Berkeley DB library declares a few external + routines. All these routines are prefixed with the strings + "db_". All internal Berkeley DB routines are prefixed with + the strings "__XXX_", where "XXX" is the subsystem prefix + (for example, "__db_XXX_" and "__txn_XXX_"). + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2778712"></a>Filesystem Name Space</h3> + <h3 class="title"><a id="idp2540280"></a>Filesystem Name Space</h3> </div> </div> </div> - <p>Berkeley DB environments create or use some number of files in environment -home directories. These files are named <a class="link" href="env_db_config.html" title="DB_CONFIG configuration file">DB_CONFIG</a>, "log.NNNNN" -(for example, log.0000000003, where the number of digits following the -dot is unspecified), or with the string prefix "__db" (for example, -__db.001). Applications should never create files or databases in -database environment home directories with names beginning with the -characters "log" or "__db".</p> - <p>In some cases, applications may choose to remove Berkeley DB files as part of -their cleanup procedures, using system utilities instead of Berkeley DB -interfaces (for example, using the UNIX rm utility instead of the -<a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> method). This is not a problem, as long as applications -limit themselves to removing only files named "__db.###", where "###" -are the digits 0 through 9. Applications should never remove any files -named with the prefix "__db" or "log", other than "__db.###" files.</p> + <p> + Berkeley DB environments create or use some number of + files in environment home directories. These files are + named <a class="link" href="env_db_config.html" title="DB_CONFIG configuration file">DB_CONFIG</a>, + "log.NNNNN" (for example, log.0000000003, where the number + of digits following the dot is unspecified), or with the + string prefix "__db" (for example, __db.001). Applications + should never create files or databases in database + environment home directories with names beginning with the + characters "log" or "__db". + </p> + <p> + In some cases, applications may choose to remove + Berkeley DB files as part of their cleanup procedures, + using system utilities instead of Berkeley DB interfaces + (for example, using the UNIX rm utility instead of the + <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> method). This is not a problem, as long as + applications limit themselves to removing only files named + "__db.###", where "###" are the digits 0 through 9. + Applications should never remove any files named with the + prefix "__db" or "log", other than "__db.###" + files. + </p> </div> </div> <div class="navfooter"> @@ -117,7 +141,8 @@ named with the prefix "__db" or "log", other than "__db.###" files.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Memory-only or Flash configurations</td> + <td width="40%" align="right" valign="top"> Memory-only or Flash + configurations</td> </tr> </table> </div> diff --git a/docs/programmer_reference/program_perfmon.html b/docs/programmer_reference/program_perfmon.html index a72e35d2..1a91ba16 100644 --- a/docs/programmer_reference/program_perfmon.html +++ b/docs/programmer_reference/program_perfmon.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Performance Event Monitoring</th> + <th colspan="3" align="center">Performance Event + Monitoring</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="program_runtime.html">Prev</a> </td> - <th width="60%" align="center">Chapter 15. - Programmer Notes - </th> + <th width="60%" align="center">Chapter 15. Programmer Notes </th> <td width="20%" align="right"> <a accesskey="n" href="program_faq.html">Next</a></td> </tr> </table> @@ -34,7 +33,8 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="program_perfmon"></a>Performance Event Monitoring</h2> + <h2 class="title" style="clear: both"><a id="program_perfmon"></a>Performance Event + Monitoring</h2> </div> </div> </div> @@ -52,7 +52,8 @@ </dt> <dt> <span class="sect2"> - <a href="program_perfmon.html#program_perfmon_examples">Example Scripts</a> + <a href="program_perfmon.html#program_perfmon_examples">Example + Scripts</a> </span> </dt> <dt> @@ -62,24 +63,24 @@ </dt> </dl> </div> - <p> - The Performance Event Monitoring feature uses Solaris DTrace or - Linux SystemTap to "publish" interesting events as they occur - inside of Berkeley DB. - The operating system utilities <span class="command"><strong>dtrace</strong></span> or - <span class="command"><strong>stap</strong></span> run scripts which select, analyze, - and display events. There is no need to modify the application. - Any application which uses that Berkeley DB library can be monitored. - For more information about these instrumentation tools refer to the - following pages: - </p> + <p> + The Performance Event Monitoring feature uses Solaris + DTrace or Linux SystemTap to "publish" interesting events as + they occur inside of Berkeley DB. The operating system + utilities <span class="command"><strong>dtrace</strong></span> or <span class="command"><strong>stap</strong></span> + run scripts which select, analyze, and display events. There + is no need to modify the application. Any application which + uses that Berkeley DB library can be monitored. For more + information about these instrumentation tools refer to the + following pages: + </p> <div class="variablelist"> <dl> <dt> <span class="term">DTrace</span> </dt> <dd> - <a class="ulink" href="http://www.oracle.com/technetwork/systems/dtrace/dtrace/index-jsp-137532.html" target="_top">http://www.oracle.com/technetwork/systems/dtrace/dtrace/index-jsp-137532.html</a> + <a class="ulink" href="http://www.oracle.com/technetwork/server-storage/solaris11/technologies/dtrace-1930301.html" target="_top">http://www.oracle.com/technetwork/server-storage/solaris11/technologies/dtrace-1930301.html</a> </dd> <dt> <span class="term">SystemTap</span> @@ -90,59 +91,64 @@ </dl> </div> <p> -</p> - <p> - Performance Event Monitoring is available for operating systems where - DTrace or SystemTap supports static probe points in user applications, - such as Solaris 10 and OpenSolaris, some versions of Linux, - and Mac OS X 10.6 and later. - By including --enable-dtrace to the configuration options, the resulting - libraries will include probe points for these event categories: </p> + <p> + Performance Event Monitoring is available for operating + systems where DTrace or SystemTap supports static probe points + in user applications, such as Solaris 10 and OpenSolaris, some + versions of Linux, and Mac OS X 10.6 and later. By including + --enable-dtrace to the configuration options, the resulting + libraries will include probe points for these event + categories: + </p> <div class="itemizedlist"> <ul type="disc"> + <li> + database operations: opening a database or + cursor, get, put, delete. + </li> <li> - database operations: opening a database or - cursor, get, put, delete. - </li> - <li> - internal operations regarding disk allocation, transactions, - concurrency control, and caching. - </li> - <li> - the beginning and ending of waiting periods due to conflicting - transactional consistency locks (for example, page locks), - mutexes, or shared latches. - </li> + internal operations regarding disk allocation, + transactions, concurrency control, and caching. + </li> <li> - timing dependent code paths which are expected to be infrequent. - These could raise concerns if they were to happen too often. - </li> + the beginning and ending of waiting periods due + to conflicting transactional consistency locks (for + example, page locks), mutexes, or shared latches. + </li> + <li> + timing dependent code paths which are expected + to be infrequent. These could raise concerns if they + were to happen too often. + </li> </ul> </div> + <p> + These probe points are implemented as + user-level statically defined traces (USDT's) for DTrace, and + static userspace markers for SystemTap. + </p> <p> - These probe points are implemented as user-level statically defined traces - (USDT's) for DTrace, and static userspace markers for SystemTap. -</p> - <p> - To monitor the statistics values as they are updated include - the --enable-perfmon-statistics configuration option. - This option generates probe points for updates to many of the counters - whose values are displayed by the db_stat utility - or returned by the various statistics functions. - The "cache" example script uses a few of these probe points. -</p> - <p> - Performance Event Monitoring is intended to be suitable for production - applications. Running Berkeley DB with DTrace or SystemTap support - built in has little effect on execution speed until probes are enabled - at runtime by the <span class="command"><strong>dtrace</strong></span> - or <span class="command"><strong>stap</strong></span> programs. -</p> + To monitor the statistics values as they are updated + include the --enable-perfmon-statistics configuration option. + This option generates probe points for updates to many of the + counters whose values are displayed by the db_stat utility or + returned by the various statistics functions. The "cache" + example script uses a few of these probe points. + </p> <p> - The list of available events may be displayed by running 'make listprobes' - after building the libdb-5.3 shared library. -</p> + Performance Event Monitoring is intended to be suitable for + production applications. Running Berkeley DB with DTrace or + SystemTap support built in has little effect on execution + speed until probes are enabled at runtime by the + <span class="command"><strong>dtrace</strong></span> or <span class="command"><strong>stap</strong></span> + programs. + </p> + <p> + The list of available events may be displayed by running + 'make listprobes' after building the libdb-6.1 + shared library. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> @@ -151,30 +157,32 @@ </div> </div> </div> + <p> + The DTrace probe provider for Berkeley DB is named + 'bdb'. A simple dtrace command to monitor all + dtrace-enabled Berkeley DB activity in the system is: + </p> <p> - The DTrace probe provider for Berkeley DB is named 'bdb'. A simple - dtrace command to monitor all dtrace-enabled Berkeley DB activity - in the system is: - </p> + <code class="code"> dtrace -Zn 'bdb*::: { printf("%s", probename); }' + </code> + </p> <p> - <code class="code"> - dtrace -Zn 'bdb*::: { printf("%s", probename); }' - </code> - </p> + DTrace requires elevated privileges in order to run. On + Solaris you can avoid running as root by giving any users + who need to run <span class="command"><strong>dtrace</strong></span> the dtrace_proc + or dtrace_user privilege in + <span class="command"><strong>/etc/user_attr</strong></span>. + </p> <p> - DTrace requires elevated privileges in order to run. On Solaris you can - avoid running as root by giving any users who need to run - <span class="command"><strong>dtrace</strong></span> the dtrace_proc or dtrace_user - privilege in <span class="command"><strong>/etc/user_attr</strong></span>. - </p> - <p> - DTrace works on both 32 and 64 bit applications. However, when tracing - a 32-bit application on a 64-bit processor it might be necessary to pass - a "32 bit" option to <span class="command"><strong>dtrace</strong></span>. Without this option - the D language might use a pointer size of 8 bytes, which could cause - pointer values (and structures containing them) to be processed incorrectly. - Use the <code class="code">-32</code> option on Solaris; on Mac OS X use <code class="code">-arch i386</code>. - </p> + DTrace works on both 32 and 64 bit applications. + However, when tracing a 32-bit application on a 64-bit + processor it might be necessary to pass a "32 bit" option + to <span class="command"><strong>dtrace</strong></span>. Without this option the D + language might use a pointer size of 8 bytes, which could + cause pointer values (and structures containing them) to + be processed incorrectly. Use the <code class="code">-32</code> option + on Solaris; on Mac OS X use <code class="code">-arch i386</code>. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> @@ -184,141 +192,159 @@ </div> </div> </div> + <p> + SystemTap looks up its static userspace markers in the + library name specified in the <span class="command"><strong>stap</strong></span> + script. A simple <span class="command"><strong>stap</strong></span> command to list + the probes of the default Berkeley DB installation is: + </p> <p> - SystemTap looks up its static userspace markers in the library name - specified in the <span class="command"><strong>stap</strong></span> script. - A simple <span class="command"><strong>stap</strong></span> command to list the probes - of the default Berkeley DB installation is: - </p> + <code class="code"> stap -l 'process("/usr/local/BerkeleyDB.6.1/lib/libdb-6.1.so").mark("*")' + </code> + </p> <p> - <code class="code"> - stap -l 'process("/usr/local/BerkeleyDB.5.3/lib/libdb-5.3.so").mark("*")' - </code> - </p> - <p> - Berkeley DB supports SystemTap version 1.1 or later. Building with - userspace marker support requires <code class="filename">sys/sdt.h</code>, - which is often available in the package <code class="filename">systemtap-sdt-devel</code>. - Running <span class="command"><strong>stap</strong></span> with userspace markers requires that the - kernel have "utrace" support; see - <a class="ulink" href="http://sourceware.org/systemtap/wiki/utrace" target="_top">http://sourceware.org/systemtap/wiki/utrace</a> for more information. - </p> - <p> - SystemTap needs elevated privileges in order to run. You can avoid running as - root by adding the users who need to run <span class="command"><strong>stap</strong></span> - to the group stapdev. - </p> + Berkeley DB supports SystemTap version 1.1 or later. + Building with userspace marker support requires + <code class="filename">sys/sdt.h</code>, which is often + available in the package + <code class="filename">systemtap-sdt-devel</code>. Running + <span class="command"><strong>stap</strong></span> with userspace markers + requires that the kernel have "utrace" support; see <a class="ulink" href="http://sourceware.org/systemtap/wiki/utrace" target="_top">http://sourceware.org/systemtap/wiki/utrace</a> + for more information. + </p> + <p> + SystemTap needs elevated privileges in order to run. + You can avoid running as root by adding the users who need + to run <span class="command"><strong>stap</strong></span> to the group stapdev. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="program_perfmon_examples"></a>Example Scripts</h3> + <h3 class="title"><a id="program_perfmon_examples"></a>Example + Scripts</h3> </div> </div> </div> - <p> - Berkeley DB includes several example scripts, in both DTrace and SystemTap versions. - The DTrace examples end with a <code class="filename">.d</code> suffix and are located in - <code class="filename">util/dtrace</code>. The SystemTap examples have a - <code class="filename">.stp</code> suffix and can found be in - <code class="filename">util/systemtap</code>. The Berkeley DB shared library name, - including any necessary path, - is expected as the first parameter to the SystemTap examples. - </p> + <p> + Berkeley DB includes several example scripts, in both + DTrace and SystemTap versions. The DTrace examples end + with a <code class="filename">.d</code> suffix and are located in + <code class="filename">util/dtrace</code>. The SystemTap + examples have a <code class="filename">.stp</code> suffix and can + found be in <code class="filename">util/systemtap</code>. The + Berkeley DB shared library name, including any necessary + path, is expected as the first parameter to the SystemTap + examples. + </p> <div class="variablelist"> <dl> <dt> <span class="term">apicalls</span> </dt> <dd> - This script graphs the count of the main API calls. The result - is grouped by thread of the target process. - </dd> + This script graphs the count of the + main API calls. The result is grouped by + thread of the target process. + </dd> <dt> <span class="term">apitimes</span> </dt> <dd> - This script graphs the time spent in the main API calls, - grouped by thread. - </dd> + This script graphs the time spent in + the main API calls, grouped by thread. + </dd> <dt> <span class="term"> apitrace </span> </dt> <dd> - This script displays the entry to and return from each of the - main API calls. - </dd> + This script displays the entry to and + return from each of the main API calls. + </dd> <dt> <span class="term"> cache </span> </dt> <dd> - This script displays overall and per-file buffer cache statistics - every <span class="emphasis"><em>N</em></span> (default: 1) seconds for - <span class="emphasis"><em>M</em></span> (default: 60) intervals. - It prints the number of cache hits, misses, and evictions for - each file with any activity during the interval. - </dd> + This script displays overall and + per-file buffer cache statistics every + <span class="emphasis"><em>N</em></span> (default: 1) + seconds for <span class="emphasis"><em>M</em></span> (default: + 60) intervals. It prints the number of cache + hits, misses, and evictions for each file with + any activity during the interval. + </dd> <dt> <span class="term"> dbdefs</span> </dt> - <dd> - This contains DTrace-compatible declarations of Berkeley DB - data structures returned from probe points. - There is no Linux equivalent; SystemTap can obtain type - information directly from the debugging symbols compiled - into the libdb*-5.3.so shared library. - </dd> + <dd> + This contains DTrace-compatible + declarations of Berkeley DB data structures + returned from probe points. There is no Linux + equivalent; SystemTap can obtain type + information directly from the debugging + symbols compiled into the + libdb*-6.1.so shared library. + </dd> <dt> <span class="term"> locktimes </span> </dt> - <dd> - This script graphs the time spent waiting for DB page locks. - The result times in nanoseconds are grouped by filename, - pgno, and lock_mode. The optional integer maxcount parameter - directs the script to exit once that many page lock waits - have been measured. - </dd> + <dd> + This script graphs the time spent + waiting for DB page locks. The result times in + nanoseconds are grouped by filename, pgno, and + lock_mode. The optional integer maxcount + parameter directs the script to exit once that + many page lock waits have been measured. + </dd> <dt> <span class="term"> locktimesid </span> </dt> <dd> - This is similar to the locktimes script above, except that it displays - the 20 byte file identifier rather than the file name. This can be - useful when there are several environments involved, or when database - files are recreated during the monitoring period. - </dd> + This is similar to the locktimes script + above, except that it displays the 20 byte + file identifier rather than the file name. + This can be useful when there are several + environments involved, or when database files + are recreated during the monitoring period. + </dd> <dt> <span class="term"> mutex </span> </dt> - <dd> - This script measures mutex wait periods, summarizing the results two ways. - <div class="itemizedlist"><ul type="disc"><li> - The first grouping is by mutex, mode (exclusive or shared), - and thread id. - </li><li> - The second grouping is by the mutex category and - mode (exclusive or shared). - The mutex categories are the MTX_XXX definitions in - <code class="filename">dbinc/mutex.h</code>. - </li></ul></div></dd> + <dd> + This script measures mutex wait + periods, summarizing the results two ways. + <div class="itemizedlist"><ul type="disc"><li> + The first grouping is by mutex, + mode (exclusive or shared), and thread + id. + </li><li> + The second grouping is by the + mutex category and mode (exclusive or + shared). The mutex categories are the + MTX_XXX definitions in + <code class="filename">dbinc/mutex.h</code>. + </li></ul></div></dd> <dt> <span class="term"> showerror </span> </dt> <dd> - This script displays the application stack when the basic error - routines are called. It provides additional information about - an error, beyond the string sent to the diagnostic output. - </dd> + This script displays the application + stack when the basic error routines are + called. It provides additional information + about an error, beyond the string sent to the + diagnostic output. + </dd> </dl> </div> <p> -</p> - <p> - These examples are designed to trace a single process. They run until - interrupted, or the monitored process exits, or the limit as given - in an optional 'maximum count' argument has been reached. - </p> + </p> + <p> + These examples are designed to trace a single process. + They run until interrupted, or the monitored process + exits, or the limit as given in an optional 'maximum + count' argument has been reached. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> @@ -329,622 +355,613 @@ </div> </div> <p> - The events are described below as if they were functions with ANSI C-style - signatures. The values each event provides to DTrace or SystemTap are - the arguments to the functions. - </p> - <p> + The events are described below as if they were + functions with ANSI C-style signatures. The values each + event provides to DTrace or SystemTap are the arguments to + the functions. + </p> + <p> + + + </p> <div class="variablelist"> <dl> <dt> - <span class="term"> -alloc -</span> + <span class="term"> alloc </span> </dt> <dd> - <p> -The alloc class covers the allocation of "on disk" database pages. -</p> + <p> + The alloc class covers the allocation of "on disk" + database pages. + </p> <div class="variablelist"> <dl> <dt> - <span class="term">alloc-new (char *file, char *db, unsigned pgno, unsigned type, - struct _db_page *pg, int ret); -</span> + <span class="term">alloc-new (char *file, char *db, unsigned + pgno, unsigned type, struct _db_page *pg, int + ret); </span> </dt> <dd> -An attempt to allocate a database page of type 'type' for database 'db' -returned 'ret'. If the allocation succeeded then ret is 0, pgno is the -location of the new page, and pg is the address of the new page. -Details of the page can be extracted from the pg pointer. -</dd> + An attempt to allocate a database page + of type 'type' for database 'db' returned + 'ret'. If the allocation succeeded then ret is + 0, pgno is the location of the new page, and + pg is the address of the new page. Details of + the page can be extracted from the pg pointer. + </dd> <dt> - <span class="term"> -alloc-free (char *file, char *db, unsigned pgno, unsigned ret); -</span> + <span class="term"> alloc-free (char *file, char *db, unsigned + pgno, unsigned ret); </span> </dt> - <dd> -An attempt to free the page 'pgno' of 'db' returned 'ret'. -When successful the page is returned to the free list -or the file is truncated. -</dd> + <dd> + An attempt to free the page 'pgno' of + 'db' returned 'ret'. When successful the page + is returned to the free list or the file is + truncated. + </dd> <dt> - <span class="term">alloc-btree_split (char *file, char *db, unsigned pgno, unsigned parent, - unsigned level); -</span> + <span class="term">alloc-btree_split (char *file, char *db, + unsigned pgno, unsigned parent, unsigned + level); </span> </dt> <dd> -A btree split of pgno in db is being attempted. The parent page number -and the level in the btree are also provided. -</dd> + A btree split of pgno in db is being + attempted. The parent page number and the + level in the btree are also provided. + </dd> </dl> </div> </dd> <dt> - <span class="term"> -db -</span> + <span class="term"> db </span> </dt> <dd> - <p> -These DB API calls provide the name of the file and database being accessed. -In-memory databases will have a NULL (0) file name address. The db name will -be null unless subdatabases are in use. -</p> + <p> + These DB API calls provide the name of the file and + database being accessed. In-memory databases will have + a NULL (0) file name address. The db name will be null + unless subdatabases are in use. + </p> <div class="variablelist"> <dl> <dt> - <span class="term"> -db-open (char *file, char *db, unsigned flags, uint8_t *fileid); -</span> + <span class="term"> db-open (char *file, char *db, unsigned + flags, uint8_t *fileid); </span> </dt> <dd> -The database or file name was opened. The 20 byte unique fileid can be -used to keep track of databases as they are created and destroyed. -</dd> + The database or file name was opened. + The 20 byte unique fileid can be used to keep + track of databases as they are created and + destroyed. + </dd> <dt> - <span class="term"> -db-close (char *file, char *db, unsigned flags, uint8_t *fileid); -</span> + <span class="term"> db-close (char *file, char *db, unsigned + flags, uint8_t *fileid); </span> </dt> - <dd> -The database or file name was closed. -</dd> + <dd> + The database or file name was closed. + </dd> <dt> - <span class="term">db-cursor (char *file, char *db, unsigned txnid, unsigned flags, - uint8_t *fileid); -</span> + <span class="term">db-cursor (char *file, char *db, unsigned + txnid, unsigned flags, uint8_t *fileid); </span> </dt> - <dd> -An attempt is being made to open a cursor on the database or file. -</dd> + <dd> + An attempt is being made to open a + cursor on the database or file. + </dd> <dt> - <span class="term">db-get (char *file, char *db, unsigned txnid, DBT *key, DBT *data, - unsigned flags); -</span> + <span class="term">db-get (char *file, char *db, unsigned + txnid, DBT *key, DBT *data, unsigned flags); </span> </dt> - <dd> -An attempt is being made to get data from a db. -</dd> + <dd> + An attempt is being made to get data + from a db. + </dd> <dt> - <span class="term">db-put (char *file, char *db, unsigned txnid, DBT *key, DBT *data, - unsigned flags); -</span> + <span class="term">db-put (char *file, char *db, unsigned + txnid, DBT *key, DBT *data, unsigned flags); </span> </dt> <dd> -An attempt is being made to put data to a db. -</dd> + An attempt is being made to put data to + a db. + </dd> <dt> - <span class="term"> -db-del (char *file, char *db, unsigned txnid, DBT *key, unsigned flags); -</span> + <span class="term"> db-del (char *file, char *db, unsigned + txnid, DBT *key, unsigned flags); </span> </dt> <dd> -An attempt is being made to delete data from a db. -</dd> + An attempt is being made to delete data + from a db. + </dd> </dl> </div> </dd> <dt> - <span class="term"> -lock -</span> + <span class="term"> lock </span> </dt> <dd> - <p> -The lock class monitors the transactional consistency locks: page, record, -and database. It also monitors the non-transactional file handle locks. -</p> + <p> + The lock class monitors the transactional + consistency locks: page, record, and database. It also + monitors the non-transactional file handle locks. + </p> <div class="variablelist"> <dl> <dt> - <span class="term"> -lock-suspend (DBT *lock, db_lockmode_t lock_mode); -</span> + <span class="term"> lock-suspend (DBT *lock, db_lockmode_t + lock_mode); </span> </dt> <dd> -The thread is about to suspend itself because another locker already has -a conflicting lock on object 'lock'. The lock DBT's data points to -a __db_ilock structure, except for the atypical program which uses -application specific locking. -</dd> + The thread is about to suspend itself + because another locker already has a + conflicting lock on object 'lock'. The lock + DBT's data points to a __db_ilock structure, + except for the atypical program which uses + application specific locking. + </dd> <dt> - <span class="term"> -lock-resume (DBT *lock, db_lockmode_t lock_mode); -</span> + <span class="term"> lock-resume (DBT *lock, db_lockmode_t + lock_mode); </span> </dt> - <dd> -The thread is awakening from a suspend. -</dd> + <dd> + The thread is awakening from a suspend. + </dd> <dt> - <span class="term"> -lock-put (struct __sh_dbt *lock, unsigned flags); -</span> + <span class="term"> lock-put (struct __sh_dbt *lock, unsigned + flags); </span> </dt> - <dd> -The lock is being freed. -</dd> + <dd> + The lock is being freed. + </dd> <dt> - <span class="term"> -lock-put_reduce_count (struct __sh_dbt *lock, unsigned flags); -</span> + <span class="term"> lock-put_reduce_count (struct __sh_dbt + *lock, unsigned flags); </span> </dt> - <dd> -The lock would have been freed except that its refcount was greater -than 1. -</dd> + <dd> + The lock would have been freed except + that its refcount was greater than 1. + </dd> </dl> </div> - <p> -These lock counters are included by --enable-perfmon-statistics. -</p> + <p> + These lock counters are included by + --enable-perfmon-statistics. + </p> <div class="variablelist"> <dl> <dt> - <span class="term">lock-deadlock (unsigned st_ndeadlocks, unsigned locker_id, - struct __sh_dbt *lock_obj); -</span> + <span class="term">lock-deadlock (unsigned st_ndeadlocks, + unsigned locker_id, struct __sh_dbt + *lock_obj); </span> </dt> - <dd> -The locker_id's lock request in lock_obj is about to be aborted in -order to resolve a deadlock. The lock region's st_ndeadlocks has -been incremented. -</dd> + <dd> + The locker_id's lock request in + lock_obj is about to be aborted in order to + resolve a deadlock. The lock region's + st_ndeadlocks has been incremented. + </dd> <dt> - <span class="term"> -lock-nowait_notgranted (unsigned count, DBT *lock, unsigned locker_id); -</span> + <span class="term"> lock-nowait_notgranted (unsigned count, DBT + *lock, unsigned locker_id); </span> </dt> <dd> -A DB_LOCK_NOWAIT lock request by locker_id would have had to wait. -The lock regions's st_lock_nowait has been incremented and -the request returns DB_LOCK_NOTGRANTED. -</dd> + A DB_LOCK_NOWAIT lock request by + locker_id would have had to wait. The lock + regions's st_lock_nowait has been incremented + and the request returns DB_LOCK_NOTGRANTED. + </dd> <dt> - <span class="term"> -lock-steal (unsigned st_locksteals, unsigned from, unsigned to); -</span> + <span class="term"> lock-steal (unsigned st_locksteals, + unsigned from, unsigned to); </span> </dt> <dd> -A lock is being stolen from one partition for another one. -The 'from' lock partition's st_locksteals has been incremented. -</dd> + A lock is being stolen from one + partition for another one. The 'from' lock + partition's st_locksteals has been + incremented. + </dd> <dt> - <span class="term"> -lock-object_steal (unsigned st_objectsteals, unsigned from, unsigned to); -</span> + <span class="term"> lock-object_steal (unsigned + st_objectsteals, unsigned from, unsigned to); </span> </dt> <dd> -A lock object is being stolen from one partition for another one. -The 'from' lock partition's st_objectsteals has been incremented. -</dd> + A lock object is being stolen from one + partition for another one. The 'from' lock + partition's st_objectsteals has been + incremented. + </dd> <dt> - <span class="term"> -lock-locktimeout (unsigned st_nlocktimeouts, const DBT *lock); -</span> + <span class="term"> lock-locktimeout (unsigned + st_nlocktimeouts, const DBT *lock); </span> </dt> <dd> -A lock wait expired due to the lock request timeout. -</dd> + A lock wait expired due to the lock + request timeout. + </dd> <dt> - <span class="term"> -lock-txntimeout (unsigned st_ntxntimeouts, const DBT *lock); -</span> + <span class="term"> lock-txntimeout (unsigned st_ntxntimeouts, + const DBT *lock); </span> </dt> <dd> -A lock wait expired due to the transaction's timeout. -</dd> + A lock wait expired due to the + transaction's timeout. + </dd> <dt> - <span class="term"> -lock-nlockers (unsigned active, unsigned locker_id); -</span> + <span class="term"> lock-nlockers (unsigned active, unsigned + locker_id); </span> </dt> - <dd> -The allocation or deallocation of the locker id changed the number -of active locker identifiers. -</dd> + <dd> The allocation or deallocation of the + locker id changed the number of active locker + identifiers. </dd> <dt> - <span class="term"> -lock-maxnlockers (unsigned new_max_active, unsigned locker_id); -</span> + <span class="term"> lock-maxnlockers (unsigned new_max_active, + unsigned locker_id); </span> </dt> <dd> -The allocation of the locker id set a new maximum -number of active locker identifiers. -</dd> + The allocation of the locker id set a + new maximum number of active locker + identifiers. + </dd> </dl> </div> </dd> <dt> - <span class="term"> -mpool -</span> + <span class="term"> mpool </span> </dt> <dd> <p> -The mpool class monitors the allocation and management of memory, -including the cache. -</p> + The mpool class monitors the allocation and + management of memory, including the cache. + </p> <div class="variablelist"> <dl> <dt> - <span class="term"> -mpool-read (char *file, unsigned pgno, struct __bh *buf); -</span> + <span class="term"> mpool-read (char *file, unsigned pgno, + struct __bh *buf); </span> </dt> <dd> -Read a page from file into buf. -</dd> + Read a page from file into buf. + </dd> <dt> - <span class="term"> -mpool-write (char *file, unsigned pgno, struct __bh *buf); -</span> + <span class="term"> mpool-write (char *file, unsigned pgno, + struct __bh *buf); </span> </dt> <dd> -Write a page from buf to file. -</dd> + Write a page from buf to file. + </dd> <dt> - <span class="term"> -mpool-env_alloc (unsigned size, unsigned region_id, unsigned reg_type); -</span> + <span class="term"> mpool-env_alloc (unsigned size, unsigned + region_id, unsigned reg_type); </span> </dt> - <dd> -This is an attempt to allocate size bytes from region_id. -The reg_type is one of the reg_type_t enum values. -</dd> + <dd> + This is an attempt to allocate size + bytes from region_id. The reg_type is one of + the reg_type_t enum values. + </dd> <dt> - <span class="term"> -mpool-evict (char *file, unsigned pgno, struct __bh *buf); -</span> + <span class="term"> mpool-evict (char *file, unsigned pgno, + struct __bh *buf); </span> </dt> <dd> -The page is about to be removed from the cache. -</dd> + The page is about to be removed from + the cache. + </dd> <dt> - <span class="term">mpool-alloc_wrap (unsigned alloc_len, int region_id, int wrap_count, - int put_counter); -</span> + <span class="term">mpool-alloc_wrap (unsigned alloc_len, int + region_id, int wrap_count, int put_counter); </span> </dt> <dd> -The memory allocator has incremented wrap_count after searching through -the entire region without being able to fulfill the request for -alloc_len bytes. As wrap_count increases the library makes more effort -to allocate space. -</dd> + The memory allocator has incremented + wrap_count after searching through the entire + region without being able to fulfill the + request for alloc_len bytes. As wrap_count + increases the library makes more effort to + allocate space. + </dd> </dl> </div> - <p> -These mpool counters are included by --enable-perfmon-statistics. -</p> + <p> + These mpool counters are included by + --enable-perfmon-statistics. + </p> <div class="variablelist"> <dl> <dt> - <span class="term"> -mpool-clean_eviction (unsigned st_ro_evict, unsigned region_id); -</span> + <span class="term"> mpool-clean_eviction (unsigned st_ro_evict, + unsigned region_id); </span> </dt> - <dd> -The eviction of a clean page from a cache incremented st_ro_evict. -</dd> + <dd> + The eviction of a clean page from a + cache incremented st_ro_evict. + </dd> <dt> - <span class="term"> -mpool-dirty_eviction (unsigned st_rw_evict, unsigned region_id); -</span> + <span class="term"> mpool-dirty_eviction (unsigned st_rw_evict, + unsigned region_id); </span> </dt> - <dd> -The eviction of a dirty page from a cache incremented st_rw_evict. -The page has already been written out. -</dd> + <dd> + The eviction of a dirty page from a + cache incremented st_rw_evict. The page has + already been written out. + </dd> <dt> - <span class="term"> -mpool-fail (unsigned failure_count, unsigned alloc_len, unsigned region_id); -</span> + <span class="term"> mpool-fail (unsigned failure_count, + unsigned alloc_len, unsigned region_id); </span> </dt> <dd> -An attempt to allocate memory from region_id failed. -</dd> + An attempt to allocate memory from + region_id failed. + </dd> <dt> - <span class="term"> -mpool-hash_search (unsigned st_hash_searches, char *file, unsigned pgno); -</span> + <span class="term"> mpool-hash_search (unsigned + st_hash_searches, char *file, unsigned pgno); </span> </dt> <dd> -A search for pgno of file incremented st_hash_searches. -</dd> + A search for pgno of file incremented + st_hash_searches. </dd> <dt> - <span class="term"> -mpool-hash_examined (unsigned st_hash_examined, char *file, unsigned pgno); -</span> + <span class="term"> mpool-hash_examined (unsigned + st_hash_examined, char *file, unsigned pgno); </span> </dt> <dd> -A search for pgno of file increased st_hash_examined by the number -of hash buckets examined. -</dd> + A search for pgno of file increased + st_hash_examined by the number of hash buckets + examined. + </dd> <dt> - <span class="term"> -mpool-hash_longest (unsigned st_hash_longest, char *file, unsigned pgno); -</span> + <span class="term"> mpool-hash_longest (unsigned + st_hash_longest, char *file, unsigned pgno); </span> </dt> <dd> -A search for pgno of file set a new maximum st_hash_longest value. -</dd> + A search for pgno of file set a new + maximum st_hash_longest value. + </dd> <dt> - <span class="term"> -mpool-map (unsigned st_map, char *file, unsigned pgno); -</span> + <span class="term"> mpool-map (unsigned st_map, char *file, + unsigned pgno); </span> </dt> - <dd> -A file's st_map count was incremented after a page was mapped into -memory. The mapping might have caused disk I/O. -</dd> + <dd> + A file's st_map count was incremented + after a page was mapped into memory. The + mapping might have caused disk I/O. + </dd> <dt> - <span class="term"> -mpool-hit (unsigned st_cache_hit, char *file, unsigned pgno); -</span> + <span class="term"> mpool-hit (unsigned st_cache_hit, char + *file, unsigned pgno); </span> </dt> - <dd> -The hit count was incremented because pgno from file was found -in the cache. -</dd> + <dd> + The hit count was incremented because + pgno from file was found in the cache. + </dd> <dt> - <span class="term"> -mpool-miss (unsigned st_cache_miss, char *file, unsigned pgno); -</span> + <span class="term"> mpool-miss (unsigned st_cache_miss, char + *file, unsigned pgno); </span> </dt> <dd> -The miss count was incremented because pgno from file was -not already present in the cache. -</dd> + The miss count was incremented because + pgno from file was not already present in the + cache. + </dd> <dt> - <span class="term"> -mpool-page_create (unsigned st_page_create, char *file, unsigned pgno); -</span> + <span class="term"> mpool-page_create (unsigned st_page_create, + char *file, unsigned pgno); </span> </dt> <dd> -The st_page_create field was incremented because -the pgno of file was created in the cache. -</dd> + The st_page_create field was + incremented because the pgno of file was + created in the cache. + </dd> <dt> - <span class="term"> -mpool-page_in (unsigned st_page_in, char *file, unsigned pgno); -</span> + <span class="term"> mpool-page_in (unsigned st_page_in, char + *file, unsigned pgno); </span> </dt> - <dd> -The st_page_in field was incremented because -the pgno from file was read into the cache. -</dd> + <dd> + The st_page_in field was incremented + because the pgno from file was read into the + cache. + </dd> <dt> - <span class="term"> -mpool-page_out (unsigned st_page_out, char *file, unsigned pgno); -</span> + <span class="term"> mpool-page_out (unsigned st_page_out, char + *file, unsigned pgno); </span> </dt> - <dd> -The st_page_out field was incremented because -the pgno from file was written out. -</dd> + <dd> + The st_page_out field was incremented + because the pgno from file was written out. + </dd> </dl> </div> </dd> <dt> - <span class="term"> -mutex -</span> + <span class="term"> mutex </span> </dt> <dd> - <p> -The mutex category monitors includes shared latches. The alloc_id value -is one of the MTX_XXX definitions from dbinc/mutex.h -</p> + <p> + The mutex category monitors includes shared + latches. The alloc_id value is one of the MTX_XXX + definitions from dbinc/mutex.h + </p> <div class="variablelist"> <dl> <dt> - <span class="term">mutex-suspend (unsigned mutex, unsigned excl, unsigned alloc_id, - struct __db_mutex_t *mutexp); -</span> + <span class="term">mutex-suspend (unsigned mutex, unsigned + excl, unsigned alloc_id, struct __db_mutex_t + *mutexp); </span> </dt> <dd> -This thread is about to suspend itself because a thread has the -mutex or shared latch locked in a mode which conflicts with the -this request. -</dd> + This thread is about to suspend itself + because a thread has the mutex or shared latch + locked in a mode which conflicts with the this + request. + </dd> <dt> - <span class="term">mutex-resume (unsigned mutex, unsigned excl, unsigned alloc_id, - struct __db_mutex_t *mutexp); -</span> + <span class="term">mutex-resume (unsigned mutex, unsigned excl, + unsigned alloc_id, struct __db_mutex_t + *mutexp); </span> </dt> - <dd> -The thread is returning from a suspend and will attempt to obtain -the mutex or shared latch again. It might need to suspend again. -</dd> + <dd> + The thread is returning from a suspend + and will attempt to obtain the mutex or shared + latch again. It might need to suspend again. + </dd> </dl> </div> - <p> -These mutex counters are included by --enable-perfmon-statistics. -</p> + <p> + These mutex counters are included by + --enable-perfmon-statistics. + </p> <div class="variablelist"> <dl> <dt> - <span class="term"> -mutex-set_nowait (unsigned mutex_set_nowait, unsigned mutex); -</span> + <span class="term"> mutex-set_nowait (unsigned + mutex_set_nowait, unsigned mutex); </span> </dt> - <dd> -Increment the count of times that the mutex was free when trying -to lock it. -</dd> + <dd> + Increment the count of times that the + mutex was free when trying to lock it. + </dd> <dt> - <span class="term"> -mutex-set_wait (unsigned mutex_set_wait, unsigned mutex); -</span> + <span class="term"> mutex-set_wait (unsigned mutex_set_wait, + unsigned mutex); </span> </dt> - <dd> -Increment the count of times that the mutex was busy when trying -to lock it. -</dd> + <dd> + Increment the count of times that the + mutex was busy when trying to lock it. + </dd> <dt> - <span class="term"> -mutex-set_rd_nowait (unsigned mutex_set_rd_nowait, unsigned mutex); -</span> + <span class="term"> mutex-set_rd_nowait (unsigned + mutex_set_rd_nowait, unsigned mutex); </span> </dt> - <dd> -Increment the count of times that the shared latch was free -when trying to get a shared lock on it. -</dd> + <dd> + Increment the count of times that the + shared latch was free when trying to get a + shared lock on it. + </dd> <dt> - <span class="term"> -mutex-set_rd_wait (unsigned mutex_set_rd_wait, unsigned mutex); -</span> + <span class="term"> mutex-set_rd_wait (unsigned + mutex_set_rd_wait, unsigned mutex); </span> </dt> <dd> -Increment the count of times that the shared latch was already -exclusively latched when trying to get a shared lock on it. -</dd> + Increment the count of times that the + shared latch was already exclusively latched + when trying to get a shared lock on it. + </dd> <dt> - <span class="term"> -mutex-hybrid_wait (unsigned hybrid_wait, unsigned mutex); -</span> + <span class="term"> mutex-hybrid_wait (unsigned hybrid_wait, + unsigned mutex); </span> </dt> - <dd> -Increment the count of times that a hybrid mutex had to block -on its condition variable. n a busy system this might happen -several times before the corresponding hybrid_wakeup. -</dd> + <dd> + Increment the count of times that a + hybrid mutex had to block on its condition + variable. n a busy system this might happen + several times before the corresponding + hybrid_wakeup. + </dd> <dt> - <span class="term"> -mutex-hybrid_wakeup (unsigned hybrid_wakeup, unsigned mutex); -</span> + <span class="term"> mutex-hybrid_wakeup (unsigned + hybrid_wakeup, unsigned mutex); </span> </dt> <dd> -Increment the count of times that a hybrid mutex finished -one or more waits for its condition variable. -</dd> + Increment the count of times that a + hybrid mutex finished one or more waits for + its condition variable. + </dd> </dl> </div> </dd> <dt> - <span class="term"> -txn -</span> + <span class="term"> txn </span> </dt> <dd> - <p> -The txn category covers the basic transaction operations. -</p> + <p> + The txn category covers the basic transaction + operations. + </p> <div class="variablelist"> <dl> <dt> - <span class="term"> -txn-begin (unsigned txnid, unsigned flags); -</span> + <span class="term"> txn-begin (unsigned txnid, unsigned flags); </span> </dt> - <dd> -A transaction was successfully begun. -</dd> + <dd> + A transaction was successfully begun. + </dd> <dt> - <span class="term"> -txn-commit (unsigned txnid, unsigned flags); -</span> + <span class="term"> txn-commit (unsigned txnid, unsigned + flags); </span> </dt> - <dd> -A transaction is starting to commit. -</dd> + <dd> + A transaction is starting to commit. + </dd> <dt> - <span class="term"> -txn-prepare (unsigned txnid, uint8_t *gid); -</span> + <span class="term"> txn-prepare (unsigned txnid, uint8_t *gid); </span> </dt> - <dd> -The transaction is starting to prepare, flushing the log -so that a future commit can be guaranteed to succeed. -The global identifier field is 128 bytes long. -</dd> + <dd> + The transaction is starting to prepare, + flushing the log so that a future commit can + be guaranteed to succeed. The global + identifier field is 128 bytes long. + </dd> <dt> - <span class="term"> -txn-abort (unsigned txnid); -</span> + <span class="term"> txn-abort (unsigned txnid); </span> </dt> <dd> -The transaction is about to abort. -</dd> + The transaction is about to abort. + </dd> </dl> </div> <p> -These txn counters are included by --enable-perfmon-statistics. -</p> + These txn counters are included by + --enable-perfmon-statistics. + </p> <div class="variablelist"> <dl> <dt> - <span class="term"> -txn-nbegins (unsigned st_nbegins, unsigned txnid); -</span> + <span class="term"> txn-nbegins (unsigned st_nbegins, unsigned + txnid); </span> </dt> - <dd> -Beginning the transaction incremented st_nbegins. -</dd> + <dd> + Beginning the transaction incremented + st_nbegins. + </dd> <dt> - <span class="term"> -txn-naborts (unsigned st_nbegins, unsigned txnid); -</span> + <span class="term"> txn-naborts (unsigned st_nbegins, unsigned + txnid); </span> </dt> - <dd> -Aborting the transaction incremented st_naborts. -</dd> + <dd> + Aborting the transaction incremented + st_naborts. + </dd> <dt> - <span class="term"> -txn-ncommits (unsigned st_ncommits, unsigned txnid); -</span> + <span class="term"> txn-ncommits (unsigned st_ncommits, + unsigned txnid); </span> </dt> <dd> -Committing the transaction incremented st_ncommits. -</dd> + Committing the transaction incremented + st_ncommits. + </dd> <dt> - <span class="term"> -txn-nactive (unsigned st_nactive, unsigned txnid); -</span> + <span class="term"> txn-nactive (unsigned st_nactive, unsigned + txnid); </span> </dt> - <dd> -Beginning or ending the transaction updated the number of active -transactions. -</dd> + <dd> + Beginning or ending the transaction + updated the number of active transactions. + </dd> <dt> - <span class="term"> -txn-maxnactive (unsigned st_maxnactive, unsigned txnid); -</span> + <span class="term"> txn-maxnactive (unsigned st_maxnactive, + unsigned txnid); </span> </dt> - <dd> -The creation of the transaction set a new maximum number -of active transactions. -</dd> + <dd> + The creation of the transaction set a + new maximum number of active transactions. + </dd> </dl> </div> </dd> </dl> </div> <p> - - </p> + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/program_ram.html b/docs/programmer_reference/program_ram.html index f94e863e..698d75b4 100644 --- a/docs/programmer_reference/program_ram.html +++ b/docs/programmer_reference/program_ram.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Memory-only or Flash configurations</th> + <th colspan="3" align="center">Memory-only or Flash + configurations</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="program_namespace.html">Prev</a> </td> - <th width="60%" align="center">Chapter 15. - Programmer Notes - </th> + <th width="60%" align="center">Chapter 15. Programmer Notes </th> <td width="20%" align="right"> <a accesskey="n" href="program_cache.html">Next</a></td> </tr> </table> @@ -34,191 +33,227 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="program_ram"></a>Memory-only or Flash configurations</h2> + <h2 class="title" style="clear: both"><a id="program_ram"></a>Memory-only or Flash + configurations</h2> </div> </div> </div> - <p>Berkeley DB supports a variety of memory-based configurations for systems -where filesystem space is either limited in availability or entirely -replaced by some combination of memory and Flash. In addition, Berkeley DB -can be configured to minimize writes to the filesystem when the -filesystem is backed by Flash memory.</p> - <p>There are four parts of the Berkeley DB database environment normally written -to the filesystem: the database environment region files, the database -files, the database environment log files and the replication internal -files. Each of these items can -be configured to live in memory rather than in the filesystem:</p> + <p> + Berkeley DB supports a variety of memory-based + configurations for systems where filesystem space is either + limited in availability or entirely replaced by some + combination of memory and Flash. In addition, Berkeley DB can + be configured to minimize writes to the filesystem when the + filesystem is backed by Flash memory. + </p> + <p> + There are four parts of the Berkeley DB database environment + normally written to the filesystem: the database environment + region files, the database files, the database environment log + files and the replication internal files. Each of these items + can be configured to live in memory rather than in the + filesystem: + </p> <div class="variablelist"> <dl> <dt> <span class="term">The database environment region files:</span> </dt> <dd> + <p> + Each of the Berkeley DB subsystems in a + database environment is described by one or more + regions, or chunks of memory. The regions contain + all of the per-process and per-thread shared + information (including mutexes), that comprise a + Berkeley DB environment. By default, these regions + are backed by the filesystem. In situations where + filesystem backed regions aren't optimal, + applications can create memory-only database + environments in two different types of memory: + either in the application's heap memory or in + system shared memory. + </p> <p> - Each of the Berkeley DB subsystems in a database environment is - described by one or more regions, or chunks of memory. The regions - contain all of the per-process and per-thread shared information - (including mutexes), that comprise a Berkeley DB environment. By - default, these regions are backed by the filesystem. In situations - where filesystem backed regions aren't optimal, applications can - create memory-only database environments in two different types of - memory: either in the application's heap memory or in system shared - memory. - </p> - <p> - To create the database environment in heap memory, specify the - <a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a> flag to the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method. Note that database - environments created in heap memory are only accessible to the - threads of a single process, however. - </p> - <p> - To create the database environment in system shared memory, specify - the <a href="../api_reference/C/envopen.html#envopen_DB_SYSTEM_MEM" class="olink">DB_SYSTEM_MEM</a> flag to the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method. Database - environments created in system memory are accessible to multiple - processes, but note that database environments created in system - shared memory do create a small (roughly 8 byte) file in the - filesystem, read by the processes to identify which system shared - memory segments to use. - </p> + To create the database environment in heap + memory, specify the <a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a> flag to the + <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method. Note that database environments + created in heap memory are only accessible to the + threads of a single process, however. + </p> + <p> + To create the database environment in system + shared memory, specify the <a href="../api_reference/C/envopen.html#envopen_DB_SYSTEM_MEM" class="olink">DB_SYSTEM_MEM</a> flag to + the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method. Database environments + created in system memory are accessible to + multiple processes, but note that database + environments created in system shared memory do + create a small (roughly 8 byte) file in the + filesystem, read by the processes to identify + which system shared memory segments to use. + </p> <p> - For more information, see <a class="xref" href="env_region.html" title="Shared memory regions">Shared memory regions</a>. - </p> + For more information, see <a class="xref" href="env_region.html" title="Shared memory regions">Shared memory regions</a>. + </p> </dd> <dt> <span class="term">The database files:</span> </dt> <dd> - <p> - By default, databases are periodically flushed from the - Berkeley DB memory cache to backing physical files in the - filesystem. To keep databases from being written to backing - physical files, pass the <a href="../api_reference/C/mempset_flags.html#mpool_set_flags_DB_MPOOL_NOFILE" class="olink">DB_MPOOL_NOFILE</a> flag to the - <a href="../api_reference/C/mempset_flags.html" class="olink">DB_MPOOLFILE->set_flags()</a> method. This flag - implies the application's databases must fit entirely in the - Berkeley DB cache, of course. To avoid a database file growing - to consume the entire cache, applications can limit the size of - individual databases in the cache by calling the - <a href="../api_reference/C/mempset_maxsize.html" class="olink">DB_MPOOLFILE->set_maxsize()</a> method. - </p> + <p> + By default, databases are periodically flushed + from the Berkeley DB memory cache to backing + physical files in the filesystem. To keep + databases from being written to backing physical + files, pass the <a href="../api_reference/C/mempset_flags.html#mpool_set_flags_DB_MPOOL_NOFILE" class="olink">DB_MPOOL_NOFILE</a> flag to the + <a href="../api_reference/C/mempset_flags.html" class="olink">DB_MPOOLFILE->set_flags()</a> method. This flag implies the + application's databases must fit entirely in the + Berkeley DB cache, of course. To avoid a database + file growing to consume the entire cache, + applications can limit the size of individual + databases in the cache by calling the + <a href="../api_reference/C/mempset_maxsize.html" class="olink">DB_MPOOLFILE->set_maxsize()</a> method. + </p> </dd> <dt> <span class="term">The database environment log files:</span> </dt> <dd> + <p> + If a database environment is not intended to be + transactionally recoverable after application or + system failure (that is, if it will not exhibit + the transactional attribute of "durability"), + applications should not configure the database + environment for logging or transactions, in which + case no log files will be created. If the database + environment is intended to be durable, log files + must either be written to stable storage and + recovered after application or system failure, or + they must be replicated to other systems. + </p> <p> - If a database environment is not intended to be transactionally - recoverable after application or system failure (that is, if it - will not exhibit the transactional attribute of "durability"), - applications should not configure the database environment for - logging or transactions, in which case no log files will be - created. If the database environment is intended to be - durable, log files must either be written to stable storage and - recovered after application or system failure, or they must be - replicated to other systems. - </p> - <p> - In applications running on systems without any form of stable - storage, durability must be accomplished through replication. - In this case, database environments should be configured to - maintain database logs in memory, rather than in the - filesystem, by specifying the <a href="../api_reference/C/envlog_set_config.html#log_set_config_DB_LOG_IN_MEMORY" class="olink">DB_LOG_IN_MEMORY</a> flag to the - <a href="../api_reference/C/envlog_set_config.html" class="olink">DB_ENV->log_set_config()</a> method. - </p> + In applications running on systems without any + form of stable storage, durability must be + accomplished through replication. In this case, + database environments should be configured to + maintain database logs in memory, rather than in + the filesystem, by specifying the + <a href="../api_reference/C/envlog_set_config.html#log_set_config_DB_LOG_IN_MEMORY" class="olink">DB_LOG_IN_MEMORY</a> flag to the <a href="../api_reference/C/envlog_set_config.html" class="olink">DB_ENV->log_set_config()</a> + method. + </p> </dd> <dt> <span class="term">The replication internal files:</span> </dt> <dd> <p> - By default, Berkeley DB replication stores a small amount of - internal data in the filesystem. To store this replication - internal data in memory only and not in the filesystem, specify - the <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_INMEM" class="olink">DB_REP_CONF_INMEM</a> flag to the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method before - opening the database environment. - </p> + By default, Berkeley DB replication stores a + small amount of internal data in the filesystem. + To store this replication internal data in memory + only and not in the filesystem, specify the + <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_INMEM" class="olink">DB_REP_CONF_INMEM</a> flag to the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method + before opening the database environment. + </p> </dd> </dl> </div> - <p>In systems where the filesystem is backed by Flash memory, the number -of times the Flash memory is written may be a concern. Each of the -four parts of the Berkeley DB database environment normally written to the -filesystem can be configured to minimize the number of times the -filesystem is written:</p> + <p> + In systems where the filesystem is backed by Flash memory, + the number of times the Flash memory is written may be a + concern. Each of the four parts of the Berkeley DB database + environment normally written to the filesystem can be + configured to minimize the number of times the filesystem is + written: + </p> <div class="variablelist"> <dl> <dt> <span class="term">The database environment region files:</span> </dt> <dd> - <p> - On a Flash-based filesystem, the database environment should be placed - in heap or system memory, as described previously. - </p> + <p> + On a Flash-based filesystem, the database + environment should be placed in heap or system + memory, as described previously. + </p> </dd> <dt> <span class="term">The database files:</span> </dt> <dd> + <p> + The Berkeley DB library maintains a cache of + database pages. The database pages are only + written to backing physical files when the + application checkpoints the database environment + with the <a href="../api_reference/C/txncheckpoint.html" class="olink">DB_ENV->txn_checkpoint()</a> method, when database + handles are closed with the <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> method, or + when the application explicitly flushes the cache + with the <a href="../api_reference/C/dbsync.html" class="olink">DB->sync()</a> or <a href="../api_reference/C/mempsync.html" class="olink">DB_ENV->memp_sync()</a> methods. + </p> <p> - The Berkeley DB library maintains a cache of database pages. - The database pages are only written to backing physical files - when the application checkpoints the database environment with - the <a href="../api_reference/C/txncheckpoint.html" class="olink">DB_ENV->txn_checkpoint()</a> method, when database handles are closed - with the <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> method, or when the application explicitly - flushes the cache with the <a href="../api_reference/C/dbsync.html" class="olink">DB->sync()</a> or <a href="../api_reference/C/mempsync.html" class="olink">DB_ENV->memp_sync()</a> methods. - </p> + To avoid unnecessary writes of Flash memory due + to checkpoints, applications should decrease the + frequency of their checkpoints. This is especially + important in applications which repeatedly modify + a specific database page, as repeatedly writing a + database page to the backing physical file will + repeatedly update the same blocks of the + filesystem. + </p> <p> - To avoid unnecessary writes of Flash memory due to checkpoints, - applications should decrease the frequency of their - checkpoints. This is especially important in applications - which repeatedly modify a specific database page, as repeatedly - writing a database page to the backing physical file will - repeatedly update the same blocks of the filesystem. - </p> - <p> - To avoid unnecessary writes of the filesystem due to closing a - database handle, applications should specify the <a href="../api_reference/C/dbclose.html#dbclose_DB_NOSYNC" class="olink">DB_NOSYNC</a> flag - to the <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> method. - </p> - <p> - To avoid unnecessary writes of the filesystem due to flushing - the cache, applications should not explicitly flush the cache - under normal conditions – flushing the cache is rarely if - ever needed in a normally-running application. - </p> + To avoid unnecessary writes of the filesystem + due to closing a database handle, applications + should specify the <a href="../api_reference/C/dbclose.html#dbclose_DB_NOSYNC" class="olink">DB_NOSYNC</a> flag to the + <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> method. + </p> + <p> + To avoid unnecessary writes of the filesystem + due to flushing the cache, applications should not + explicitly flush the cache under normal conditions + – flushing the cache is rarely if ever + needed in a normally-running application. + </p> </dd> <dt> <span class="term">The database environment log files:</span> </dt> <dd> <p> - The Berkeley DB log files do not repeatedly overwrite the same - blocks of the filesystem as the Berkeley DB log files are not - implemented as a circular buffer and log files are not re-used. - For this reason, the Berkeley DB log files should not cause any - difficulties for Flash memory configurations. - </p> + The Berkeley DB log files do not repeatedly + overwrite the same blocks of the filesystem as the + Berkeley DB log files are not implemented as a + circular buffer and log files are not re-used. For + this reason, the Berkeley DB log files should not + cause any difficulties for Flash memory + configurations. + </p> <p> - However, as Berkeley DB does not write log records in - filesystem block sized pieces, it is probable that sequential - transaction commits (each of which flush the log file to the - backing filesystem), will write a block of Flash memory twice, - as the last log record from the first commit will write the - same block of Flash memory as the first log record from the - second commit. Applications not requiring absolute durability - should specify the <a href="../api_reference/C/envset_flags.html#set_flags_DB_TXN_WRITE_NOSYNC" class="olink">DB_TXN_WRITE_NOSYNC</a> or - <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> flags to the <a href="../api_reference/C/envset_flags.html" class="olink">DB_ENV->set_flags()</a> - method to avoid this overwrite of a block of Flash memory. - </p> + However, as Berkeley DB does not write log + records in filesystem block sized pieces, it is + probable that sequential transaction commits (each + of which flush the log file to the backing + filesystem), will write a block of Flash memory + twice, as the last log record from the first + commit will write the same block of Flash memory + as the first log record from the second commit. + Applications not requiring absolute durability + should specify the <a href="../api_reference/C/envset_flags.html#set_flags_DB_TXN_WRITE_NOSYNC" class="olink">DB_TXN_WRITE_NOSYNC</a> or + <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> flags to the <a href="../api_reference/C/envset_flags.html" class="olink">DB_ENV->set_flags()</a> method + to avoid this overwrite of a block of Flash + memory. + </p> </dd> <dt> <span class="term">The replication internal files:</span> </dt> <dd> - <p> - On a Flash-based filesystem, the replication internal data - should be stored in memory only, as described previously. - </p> + <p> + On a Flash-based filesystem, the replication + internal data should be stored in memory only, as + described previously. + </p> </dd> </dl> </div> diff --git a/docs/programmer_reference/program_runtime.html b/docs/programmer_reference/program_runtime.html index b3cd0f07..ee3376b8 100644 --- a/docs/programmer_reference/program_runtime.html +++ b/docs/programmer_reference/program_runtime.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="program_compatible.html">Prev</a> </td> - <th width="60%" align="center">Chapter 15. - Programmer Notes - </th> + <th width="60%" align="center">Chapter 15. Programmer Notes </th> <td width="20%" align="right"> <a accesskey="n" href="program_perfmon.html">Next</a></td> </tr> </table> @@ -38,130 +36,95 @@ </div> </div> </div> - <p>It is possible for applications to configure Berkeley DB at run-time to -redirect Berkeley DB library and system calls to alternate interfaces. For -example, an application might want Berkeley DB to call debugging memory -allocation routines rather than the standard C library interfaces. The -following interfaces support this functionality:</p> + <p> + It is possible for applications to configure Berkeley DB at + run-time to redirect Berkeley DB library and system calls to + alternate interfaces. For example, an application might want + Berkeley DB to call debugging memory allocation routines + rather than the standard C library interfaces. The following + interfaces support this functionality: + </p> <div class="itemizedlist"> <ul type="disc"> <li> - <p> - <a href="../api_reference/C/db_env_set_func_close.html" class="olink">db_env_set_func_close</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_close.html" class="olink">db_env_set_func_close</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_dirfree.html" class="olink">db_env_set_func_dirfree</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_dirfree.html" class="olink">db_env_set_func_dirfree</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_dirlist.html" class="olink">db_env_set_func_dirlist</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_dirlist.html" class="olink">db_env_set_func_dirlist</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_exists.html" class="olink">db_env_set_func_exists</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_exists.html" class="olink">db_env_set_func_exists</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_file_map.html" class="olink">db_env_set_func_file_map</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_file_map.html" class="olink">db_env_set_func_file_map</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_free.html" class="olink">db_env_set_func_free</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_free.html" class="olink">db_env_set_func_free</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_fsync.html" class="olink">db_env_set_func_fsync</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_fsync.html" class="olink">db_env_set_func_fsync</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_ftruncate.html" class="olink">db_env_set_func_ftruncate</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_ftruncate.html" class="olink">db_env_set_func_ftruncate</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_ioinfo.html" class="olink">db_env_set_func_ioinfo</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_ioinfo.html" class="olink">db_env_set_func_ioinfo</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_malloc.html" class="olink">db_env_set_func_malloc</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_malloc.html" class="olink">db_env_set_func_malloc</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_open.html" class="olink">db_env_set_func_open</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_open.html" class="olink">db_env_set_func_open</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_pread.html" class="olink">db_env_set_func_pread</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_pread.html" class="olink">db_env_set_func_pread</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_pwrite.html" class="olink">db_env_set_func_pwrite</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_pwrite.html" class="olink">db_env_set_func_pwrite</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_read.html" class="olink">db_env_set_func_read</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_read.html" class="olink">db_env_set_func_read</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_realloc.html" class="olink">db_env_set_func_realloc</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_realloc.html" class="olink">db_env_set_func_realloc</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_region_map.html" class="olink">db_env_set_func_region_map</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_region_map.html" class="olink">db_env_set_func_region_map</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_rename.html" class="olink">db_env_set_func_rename</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_rename.html" class="olink">db_env_set_func_rename</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_seek.html" class="olink">db_env_set_func_seek</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_seek.html" class="olink">db_env_set_func_seek</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_unlink.html" class="olink">db_env_set_func_unlink</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_unlink.html" class="olink">db_env_set_func_unlink</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_write.html" class="olink">db_env_set_func_write</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_write.html" class="olink">db_env_set_func_write</a> </p> </li> <li> - <p> - <a href="../api_reference/C/db_env_set_func_yield.html" class="olink">db_env_set_func_yield</a> - </p> + <p> <a href="../api_reference/C/db_env_set_func_yield.html" class="olink">db_env_set_func_yield</a> </p> </li> </ul> </div> - <p>These interfaces are available only on POSIX platforms and from the -Berkeley DB C language API.</p> - <p>A not-uncommon problem for applications is the new API in Solaris 2.6 -for manipulating large files. Because this API was not part of Solaris -2.5, it is difficult to create a single binary that takes advantage of -the large file functionality in Solaris 2.6, but still runs on Solaris -2.5. <a class="ulink" href="solaris.txt" target="_top">Example code</a> that supports this is -included in the Berkeley DB distribution, however, the example code was -written using previous versions of the Berkeley DB APIs, and is only useful -as an example.</p> + <p> + These interfaces are available only on POSIX platforms and + from the Berkeley DB C language API. + </p> + <p> + A not-uncommon problem for applications is the new API in + Solaris 2.6 for manipulating large files. Because this API was + not part of Solaris 2.5, it is difficult to create a single + binary that takes advantage of the large file functionality in + Solaris 2.6, but still runs on Solaris 2.5. <a class="ulink" href="solaris.txt" target="_top">Example code</a> that supports this + is included in the Berkeley DB distribution, however, the + example code was written using previous versions of the + Berkeley DB APIs, and is only useful as an example. + </p> </div> <div class="navfooter"> <hr /> @@ -178,7 +141,8 @@ as an example.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Performance Event Monitoring</td> + <td width="40%" align="right" valign="top"> Performance Event + Monitoring</td> </tr> </table> </div> diff --git a/docs/programmer_reference/program_scope.html b/docs/programmer_reference/program_scope.html index bbf10934..2550b9ff 100644 --- a/docs/programmer_reference/program_scope.html +++ b/docs/programmer_reference/program_scope.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="program_mt.html">Prev</a> </td> - <th width="60%" align="center">Chapter 15. - Programmer Notes - </th> + <th width="60%" align="center">Chapter 15. Programmer Notes </th> <td width="20%" align="right"> <a accesskey="n" href="program_namespace.html">Next</a></td> </tr> </table> @@ -38,9 +36,12 @@ </div> </div> </div> - <p>The Berkeley DB library has a number of object handles. The following table -lists those handles, their scope, and whether they are free-threaded -(that is, whether multiple threads within a process can share them).</p> + <p> + The Berkeley DB library has a number of object handles. The + following table lists those handles, their scope, and whether + they are free-threaded (that is, whether multiple threads + within a process can share them). + </p> <div class="variablelist"> <dl> <dt> @@ -49,19 +50,20 @@ lists those handles, their scope, and whether they are free-threaded </span> </dt> <dd> - <p> - The <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handle, created by the <a href="../api_reference/C/envcreate.html" class="olink">db_env_create()</a> method, refers - to a Berkeley DB database environment — a collection - of Berkeley DB subsystems, log files and databases. <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> - handles are free-threaded if the <a href="../api_reference/C/envopen.html#envopen_DB_THREAD" class="olink">DB_THREAD</a> flag is specified - to the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method when the - environment is opened. The handle should not be closed - while any other handle remains open that is using it as a - reference (for example, <a href="../api_reference/C/db.html" class="olink">DB</a>, <a href="../api_reference/C/txn.html" class="olink">TXN</a>). Once either the - <a href="../api_reference/C/envclose.html" class="olink">DB_ENV->close()</a> or <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> methods - are called, the handle may not be accessed again, - regardless of the method's return. - </p> + <p> + The <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handle, created by the <a href="../api_reference/C/envcreate.html" class="olink">db_env_create()</a> + method, refers to a Berkeley DB database + environment — a collection of Berkeley DB + subsystems, log files and databases. <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handles + are free-threaded if the <a href="../api_reference/C/envopen.html#envopen_DB_THREAD" class="olink">DB_THREAD</a> flag is + specified to the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method when the + environment is opened. The handle should not be + closed while any other handle remains open that is + using it as a reference (for example, <a href="../api_reference/C/db.html" class="olink">DB</a>, + <a href="../api_reference/C/txn.html" class="olink">TXN</a>). Once either the <a href="../api_reference/C/envclose.html" class="olink">DB_ENV->close()</a> or <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> + methods are called, the handle may not be accessed + again, regardless of the method's return. + </p> </dd> <dt> <span class="term"> @@ -69,25 +71,29 @@ lists those handles, their scope, and whether they are free-threaded </span> </dt> <dd> - <p> - The <a href="../api_reference/C/txn.html" class="olink">TXN</a> handle, created by the <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a> method, refers to a - single transaction. The handle is not free-threaded. - Transactions may span threads, but only serially, that is, the - application must serialize access to the <a href="../api_reference/C/txn.html" class="olink">TXN</a> handles. In the - case of nested transactions, since all child transactions are - part of the same parent transaction, they must observe the same - constraints. That is, children may execute in different threads - only if each child executes serially. - </p> - <p> - Once the <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> or <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a> methods are called, the - handle may not be accessed again, regardless of the method's - return. In addition, parent transactions may not issue any - Berkeley DB operations while they have active child - transactions (child transactions that have not yet been - committed or aborted) except for <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a>, <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> and - <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a>. - </p> + <p> + The <a href="../api_reference/C/txn.html" class="olink">TXN</a> handle, created by the <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a> + method, refers to a single transaction. The handle + is not free-threaded. Transactions may span + threads, but only serially, that is, the + application must serialize access to the <a href="../api_reference/C/txn.html" class="olink">TXN</a> + handles. In the case of nested transactions, since + all child transactions are part of the same parent + transaction, they must observe the same + constraints. That is, children may execute in + different threads only if each child executes + serially. + </p> + <p> + Once the <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> or <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a> methods are + called, the handle may not be accessed again, + regardless of the method's return. In addition, + parent transactions may not issue any Berkeley DB + operations while they have active child + transactions (child transactions that have not yet + been committed or aborted) except for <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a>, + <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> and <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a>. + </p> </dd> <dt> <span class="term"> @@ -95,12 +101,13 @@ lists those handles, their scope, and whether they are free-threaded </span> </dt> <dd> - <p> - The <a href="../api_reference/C/logc.html" class="olink">DB_LOGC</a> handle refers to a cursor into the log files. The - handle is not free-threaded. Once the <a href="../api_reference/C/logcclose.html" class="olink">DB_LOGC->close()</a> method is - called, the handle may not be accessed again, regardless of the - method's return. - </p> + <p> + The <a href="../api_reference/C/logc.html" class="olink">DB_LOGC</a> handle refers to a cursor into the + log files. The handle is not free-threaded. Once + the <a href="../api_reference/C/logcclose.html" class="olink">DB_LOGC->close()</a> method is called, the handle may + not be accessed again, regardless of the method's + return. + </p> </dd> <dt> <span class="term"> @@ -109,12 +116,13 @@ lists those handles, their scope, and whether they are free-threaded </dt> <dd> <p> - The <a href="../api_reference/C/memp.html" class="olink">DB_MPOOLFILE</a> handle refers to an open file in the shared memory - buffer pool of the database environment. The handle is not - free-threaded. Once the <a href="../api_reference/C/mempfclose.html" class="olink">DB_MPOOLFILE->close()</a> method is called, the - handle may not be accessed again, regardless of the method's - return. - </p> + The <a href="../api_reference/C/memp.html" class="olink">DB_MPOOLFILE</a> handle refers to an open file in the + shared memory buffer pool of the database + environment. The handle is not free-threaded. Once + the <a href="../api_reference/C/mempfclose.html" class="olink">DB_MPOOLFILE->close()</a> method is called, the handle may + not be accessed again, regardless of the method's + return. + </p> </dd> <dt> <span class="term"> @@ -122,21 +130,25 @@ lists those handles, their scope, and whether they are free-threaded </span> </dt> <dd> - <p> - The <a href="../api_reference/C/db.html" class="olink">DB</a> handle, created by the <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a> method, refers to a - single Berkeley DB database, which may or may not be part of a - database environment. <a href="../api_reference/C/db.html" class="olink">DB</a> handles are free-threaded if the - <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flag is specified to the <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> method when the - database is opened or if the database environment in which the - database is opened is free-threaded. The handle should not be - closed while any other handle that refers to the database is in - use; for example, database handles should be left open while - cursor handles into the database remain open, or transactions - that include operations on the database have not yet been - committed or aborted. Once the <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a>, <a href="../api_reference/C/dbremove.html" class="olink">DB->remove()</a> or - <a href="../api_reference/C/dbrename.html" class="olink">DB->rename()</a> methods are called, the handle may not be accessed - again, regardless of the method's return. - </p> + <p> + The <a href="../api_reference/C/db.html" class="olink">DB</a> handle, created by the <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a> + method, refers to a single Berkeley DB database, + which may or may not be part of a database + environment. <a href="../api_reference/C/db.html" class="olink">DB</a> handles are free-threaded if the + <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flag is specified to the <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> + method when the database is opened or if the + database environment in which the database is + opened is free-threaded. The handle should not be + closed while any other handle that refers to the + database is in use; for example, database handles + should be left open while cursor handles into the + database remain open, or transactions that include + operations on the database have not yet been + committed or aborted. Once the <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a>, + <a href="../api_reference/C/dbremove.html" class="olink">DB->remove()</a> or <a href="../api_reference/C/dbrename.html" class="olink">DB->rename()</a> methods are called, the + handle may not be accessed again, regardless of + the method's return. + </p> </dd> <dt> <span class="term"> @@ -144,17 +156,19 @@ lists those handles, their scope, and whether they are free-threaded </span> </dt> <dd> - <p> - The <a href="../api_reference/C/dbc.html" class="olink">DBC</a> handle refers to a cursor into a Berkeley DB - database. The handle is not free-threaded. Cursors may span - threads, but only serially, that is, the application must - serialize access to the <a href="../api_reference/C/dbc.html" class="olink">DBC</a> handles. If the cursor is to be - used to perform operations on behalf of a transaction, the - cursor must be opened and closed within the context of that - single transaction. Once <a href="../api_reference/C/dbcclose.html" class="olink">DBC->close()</a> has been called, the - handle may not be accessed again, regardless of the method's - return. - </p> + <p> + The <a href="../api_reference/C/dbc.html" class="olink">DBC</a> handle refers to a cursor into a + Berkeley DB database. The handle is not + free-threaded. Cursors may span threads, but only + serially, that is, the application must serialize + access to the <a href="../api_reference/C/dbc.html" class="olink">DBC</a> handles. If the cursor is to + be used to perform operations on behalf of a + transaction, the cursor must be opened and closed + within the context of that single transaction. + Once <a href="../api_reference/C/dbcclose.html" class="olink">DBC->close()</a> has been called, the handle may + not be accessed again, regardless of the method's + return. + </p> </dd> </dl> </div> diff --git a/docs/programmer_reference/refs.html b/docs/programmer_reference/refs.html index de579977..818a4f09 100644 --- a/docs/programmer_reference/refs.html +++ b/docs/programmer_reference/refs.html @@ -3,7 +3,7 @@ <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> - <title>Chapter 24. Additional References</title> + <title>Chapter 24. Additional References</title> <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> @@ -13,13 +13,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 24. - Additional References - </th> + <th colspan="3" align="center">Chapter 24. Additional References</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="dumpload_text.html">Prev</a> </td> @@ -33,9 +31,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="refs"></a>Chapter 24. - Additional References - </h2> + <h2 class="title"><a id="refs"></a>Chapter 24. Additional References</h2> </div> </div> </div> @@ -53,17 +49,17 @@ <dl> <dt> <span class="sect2"> - <a href="refs.html#idp3221064">Technical Papers on Berkeley DB</a> + <a href="refs.html#idp2980680">Technical Papers on Berkeley DB</a> </span> </dt> <dt> <span class="sect2"> - <a href="refs.html#idp3270728">Background on Berkeley DB Features</a> + <a href="refs.html#idp3039152">Background on Berkeley DB Features</a> </span> </dt> <dt> <span class="sect2"> - <a href="refs.html#idp3264824">Database Systems Theory</a> + <a href="refs.html#idp3033336">Database Systems Theory</a> </span> </dt> </dl> @@ -82,17 +78,17 @@ <dl> <dt> <span class="sect2"> - <a href="refs.html#idp3221064">Technical Papers on Berkeley DB</a> + <a href="refs.html#idp2980680">Technical Papers on Berkeley DB</a> </span> </dt> <dt> <span class="sect2"> - <a href="refs.html#idp3270728">Background on Berkeley DB Features</a> + <a href="refs.html#idp3039152">Background on Berkeley DB Features</a> </span> </dt> <dt> <span class="sect2"> - <a href="refs.html#idp3264824">Database Systems Theory</a> + <a href="refs.html#idp3033336">Database Systems Theory</a> </span> </dt> </dl> @@ -105,7 +101,7 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp3221064"></a>Technical Papers on Berkeley DB</h3> + <h3 class="title"><a id="idp2980680"></a>Technical Papers on Berkeley DB</h3> </div> </div> </div> @@ -170,7 +166,7 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp3270728"></a>Background on Berkeley DB Features</h3> + <h3 class="title"><a id="idp3039152"></a>Background on Berkeley DB Features</h3> </div> </div> </div> @@ -246,7 +242,7 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp3264824"></a>Database Systems Theory</h3> + <h3 class="title"><a id="idp3033336"></a>Database Systems Theory</h3> </div> </div> </div> diff --git a/docs/programmer_reference/rep.html b/docs/programmer_reference/rep.html index 15a7e5a4..6692e1fb 100644 --- a/docs/programmer_reference/rep.html +++ b/docs/programmer_reference/rep.html @@ -14,13 +14,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th colspan="3" align="center">Chapter 12. Berkeley DB Replication </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_faq.html">Prev</a> </td> @@ -34,9 +32,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="rep"></a>Chapter 12. - Berkeley DB Replication - </h2> + <h2 class="title"><a id="rep"></a>Chapter 12. Berkeley DB Replication </h2> </div> </div> </div> @@ -72,7 +68,7 @@ </dt> <dt> <span class="sect1"> - <a href="rep_base_meth.html">Base API Methods</a> + <a href="rep_base_meth.html">Base API methods</a> </span> </dt> <dt> @@ -87,104 +83,132 @@ </dt> <dt> <span class="sect1"> - <a href="group_membership.html">Managing Replication Manager Group Membership</a> + <a href="group_membership.html">Managing Replication Manager + group membership</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="group_membership.html#group_mem_add">Adding Sites to a Replication Group</a> + <a href="group_membership.html#group_mem_add">Adding sites to a replication + group</a> </span> </dt> <dt> <span class="sect2"> - <a href="group_membership.html#group_mem_remove">Removing Sites from a Replication Group</a> + <a href="group_membership.html#group_mem_remove">Removing sites from a + replication group</a> </span> </dt> <dt> <span class="sect2"> - <a href="group_membership.html#group_mem_primordialstartup">Primordial Startups</a> + <a href="group_membership.html#group_mem_primordialstartup">Primordial startups</a> </span> </dt> <dt> <span class="sect2"> - <a href="group_membership.html#group_mem_upgrade">Upgrading Groups</a> + <a href="group_membership.html#group_mem_upgrade">Upgrading groups</a> </span> </dt> </dl> </dd> <dt> <span class="sect1"> - <a href="rep_filename.html">Managing Replication Files</a> + <a href="rep_partview.html">Replication views</a> </span> </dt> <dt> <span class="sect1"> - <a href="rep_mgrmulti.html">Running Replication Manager in multiple processes</a> + <a href="rep_filename.html">Managing replication directories and files</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="rep_mgrmulti.html#idp2239216">One replication process and multiple subordinate processes</a> + <a href="rep_filename.html#rep_dir">Replication database directory + considerations</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_mgrmulti.html#idp2202400">Persistence of local site network address configuration</a> + <a href="rep_filename.html#rep_files">Managing replication internal + files</a> + </span> + </dt> + </dl> + </dd> + <dt> + <span class="sect1"> + <a href="rep_mgrmulti.html">Running Replication Manager in + multiple processes</a> + </span> + </dt> + <dd> + <dl> + <dt> + <span class="sect2"> + <a href="rep_mgrmulti.html#idp1913552">One replication process and multiple subordinate + processes</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="rep_mgrmulti.html#idp1910720">Persistence of local site network address + configuration</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_mgrmulti.html#idp2221464">Programming considerations</a> + <a href="rep_mgrmulti.html#idp1905672">Programming considerations</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_mgrmulti.html#idp2233184">Handling failure</a> + <a href="rep_mgrmulti.html#idp1902040">Handling failure</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_mgrmulti.html#idp2233360">Other miscellaneous rules</a> + <a href="rep_mgrmulti.html#idp1877896">Other miscellaneous rules</a> </span> </dt> </dl> </dd> <dt> <span class="sect1"> - <a href="rep_replicate.html">Running Replication using the db_replicate Utility</a> + <a href="rep_replicate.html">Running replication using the + db_replicate utility</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="rep_replicate.html#idp2251336">One Replication Process and Multiple Subordinate Processes</a> + <a href="rep_replicate.html#idp1926936">One replication process and multiple subordinate processes</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_replicate.html#idp2268704">Common Use Case</a> + <a href="rep_replicate.html#idp1906152">Common use case</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_replicate.html#idp2278848">Avoiding Rollback</a> + <a href="rep_replicate.html#idp1939624">Avoiding rollback</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_replicate.html#idp2283896">When to Consider an Integrated HA Application</a> + <a href="rep_replicate.html#idp1937856">When to consider an integrated replication application</a> </span> </dt> </dl> </dd> <dt> <span class="sect1"> - <a href="rep_mgr_ack.html">Choosing a Replication Manager Ack Policy</a> + <a href="rep_mgr_ack.html">Choosing a Replication Manager acknowledgement policy</a> </span> </dt> <dt> @@ -194,7 +218,8 @@ </dt> <dt> <span class="sect1"> - <a href="rep_mastersync.html">Synchronizing with a master</a> + <a href="rep_mastersync.html">Synchronizing with a + master</a> </span> </dt> <dd> @@ -211,12 +236,12 @@ </dt> <dt> <span class="sect2"> - <a href="rep_mastersync.html#idp2309616">Blocked client operations</a> + <a href="rep_mastersync.html#idp1985136">Blocked client operations</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_mastersync.html#idp2331664">Clients too far out-of-date to synchronize</a> + <a href="rep_mastersync.html#idp2008240">Clients too far out-of-date to synchronize</a> </span> </dt> </dl> @@ -238,14 +263,15 @@ </dt> <dt> <span class="sect1"> - <a href="rep_lease.html">Master Leases</a> + <a href="rep_lease.html">Master leases</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="rep_lease.html#masterlease_change_groupsize">Changing Group Size</a> + <a href="rep_lease.html#masterlease_change_groupsize">Changing group + size</a> </span> </dt> </dl> @@ -276,7 +302,7 @@ </dd> <dt> <span class="sect1"> - <a href="rep_clock_skew.html">Clock Skew</a> + <a href="rep_clock_skew.html">Clock skew</a> </span> </dt> <dt> @@ -308,6 +334,25 @@ <a href="rep_twosite.html">Special considerations for two-site replication groups</a> </span> </dt> + <dd> + <dl> + <dt> + <span class="sect2"> + <a href="rep_twosite.html#twosite_strict">Two-site strict configuration</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="rep_twosite.html#twosite_prefmas">Preferred master mode</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="rep_twosite.html#twosite_other">Other alternatives</a> + </span> + </dt> + </dl> + </dd> <dt> <span class="sect1"> <a href="rep_partition.html">Network partitions</a> @@ -330,13 +375,13 @@ </dt> <dt> <span class="sect1"> - <a href="rep_ex_rq.html">Ex_rep_base: putting it all together</a> + <a href="rep_ex_rq.html">Ex_rep_base: putting it all + together</a> </span> </dt> <dt> <span class="sect1"> - <a href="rep_ex_chan.html">Ex_rep_chan: a Replication Manager -channel example</a> + <a href="rep_ex_chan.html">Ex_rep_chan: a Replication Manager channel example</a> </span> </dt> </dl> @@ -349,71 +394,109 @@ channel example</a> </div> </div> </div> - <p>Berkeley DB includes support for building highly available applications based -on replication. Berkeley DB replication groups consist of some number of -independently configured database environments. There is a single -<span class="emphasis"><em>master</em></span> database environment and one or more <span class="emphasis"><em>client</em></span> -database environments. Master environments support both database reads -and writes; client environments support only database reads. If the -master environment fails, applications may upgrade a client to be the -new master. The database environments might be on separate computers, -on separate hardware partitions in a non-uniform memory access (NUMA) -system, or on separate disks in a single server. -As always with Berkeley DB environments, any number of -concurrent processes or threads may access a database environment. In -the case of a master environment, any number of threads of control may -read and write the environment, and in the case of a client environment, -any number of threads of control may read the environment.</p> - <p>Applications may be written to provide various degrees of consistency -between the master and clients. The system can be run synchronously -such that replicas are guaranteed to be up-to-date with all committed -transactions, but doing so may incur a significant performance penalty. -Higher performance solutions sacrifice total consistency, allowing the -clients to be out of date for an application-controlled amount of -time.</p> <p> - There are two ways to build replicated applications. The simpler way - is to use the Berkeley DB Replication Manager. The Replication Manager - provides a standard communications infrastructure, and it creates and - manages the background threads needed for processing replication - messages. -</p> - <p>The Replication Manager implementation is based on TCP/IP sockets, and -uses POSIX 1003.1 style networking and thread support. (On Windows -systems, it uses standard Windows thread support.) As a result, it is -not as portable as the rest of the Berkeley DB library itself.</p> - <p>The alternative is to use the lower-level replication "Base APIs". This -approach affords more flexibility, but requires the application to -provide some critical components:</p> + Berkeley DB includes support for building highly available + applications based on replication. Berkeley DB replication + groups consist of some number of independently configured + database environments. There is a single + <span class="emphasis"><em>master</em></span> database environment and one + or more <span class="emphasis"><em>client</em></span> database environments. + Master environments support both database reads and writes; + client environments support only database reads. If the master + environment fails, applications may upgrade a client to be the + new master. The database environments might be on separate + computers, on separate hardware partitions in a non-uniform + memory access (NUMA) system, or on separate disks in a single + server. As always with Berkeley DB environments, any number of + concurrent processes or threads may access a database + environment. In the case of a master environment, any number + of threads of control may read and write the environment, and + in the case of a client environment, any number of threads of + control may read the environment. + </p> + <p> + Applications may be written to provide various degrees of + consistency between the master and clients. The system can be + run synchronously such that replicas are guaranteed to be + up-to-date with all committed transactions, but doing so may + incur a significant performance penalty. Higher performance + solutions sacrifice total consistency, allowing the clients to + be out of date for an application-controlled amount of + time. + </p> + <p> + There are two ways to build replicated applications. The + simpler way is to use the Berkeley DB Replication Manager. The + Replication Manager provides a standard communications + infrastructure, and it creates and manages the background + threads needed for processing replication messages. + </p> + <p> + The Replication Manager implementation is based on TCP/IP + sockets, and uses POSIX 1003.1 style networking and thread + support. (On Windows systems, it uses standard Windows thread + support.) As a result, it is not as portable as the rest of + the Berkeley DB library itself. + </p> + <p> + The alternative is to use the lower-level replication "Base + APIs". This approach affords more flexibility, but requires + the application to provide some critical components: + </p> <div class="orderedlist"> <ol type="1"> - <li>A communication infrastructure. Applications may use whatever wire -protocol is appropriate for their application (for example, RPC, TCP/IP, -UDP, VI or message-passing over the backplane).</li> - <li>The application is responsible for naming. Berkeley DB refers to the members -of a replication group using an application-provided ID, and -applications must map that ID to a particular database environment or -communication channel.</li> - <li>The application is responsible for monitoring the status of the master -and clients, and identifying any unavailable database environments.</li> - <li>The application must provide whatever security policies are needed. -For example, the application may choose to encrypt data, use a secure -socket layer, or do nothing at all. The level of security is left to -the sole discretion of the application.</li> + <li> + A communication infrastructure. Applications may use + whatever wire protocol is appropriate for their + application (for example, RPC, TCP/IP, UDP, VI or + message-passing over the backplane). + </li> + <li> + The application is responsible for naming. Berkeley + DB refers to the members of a replication group using an + application-provided ID, and applications must map that ID + to a particular database environment or communication + channel. + </li> + <li> + The application is responsible for monitoring the + status of the master and clients, and identifying any + unavailable database environments. + </li> + <li> + The application must provide whatever security + policies are needed. For example, the application may + choose to encrypt data, use a secure socket layer, or do + nothing at all. The level of security is left to the sole + discretion of the application. + </li> </ol> </div> - <p>(Note that Replication Manager does not provide wire security for -replication messages.)</p> - <p>The following pages present various programming considerations, many of -which are directly relevant only for Base API applications. However, even when using Replication Manager it is -important to understand the concepts.</p> - <p>Finally, the Berkeley DB replication implementation has one other additional -feature to increase application reliability. Replication in Berkeley DB is -implemented to perform database updates using a different code path than -the standard ones. This means operations that manage to crash the -replication master due to a software bug will not necessarily also crash -replication clients.</p> - <p>For more information on the replication manager operations, see the <a href="../api_reference/C/rep.html#replist" class="olink">Replication and Related Methods</a> section in the <em class="citetitle">Berkeley DB C API Reference Guide.</em> </p> + <p> + (Note that Replication Manager does not provide wire + security for replication messages.) + </p> + <p> + The following pages present various programming + considerations, many of which are directly relevant only for + Base API applications. However, even when using Replication + Manager it is important to understand the concepts. + </p> + <p> + Finally, the Berkeley DB replication implementation has one + other additional feature to increase application reliability. + Replication in Berkeley DB is implemented to perform database + updates using a different code path than the standard ones. + This means operations that manage to crash the replication + master due to a software bug will not necessarily also crash + replication clients. + </p> + <p> + For more information on the Replication Manager operations, + see the <a href="../api_reference/C/rep.html#replist" class="olink">Replication + and Related Methods</a> section in the + <em class="citetitle">Berkeley DB C API Reference Guide.</em> + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/rep_app.html b/docs/programmer_reference/rep_app.html index 71578840..a8dba30e 100644 --- a/docs/programmer_reference/rep_app.html +++ b/docs/programmer_reference/rep_app.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_pri.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_mgr_meth.html">Next</a></td> </tr> </table> @@ -38,98 +36,134 @@ </div> </div> </div> - <p>The simplest way to build a replicated Berkeley DB application is to first -build (and debug!) the transactional version of the same application. -Then, add a thin replication layer: application initialization must be -changed and the application's communication infrastructure must be -added.</p> - <p>The application initialization changes are relatively simple. -Replication Manager provides a communication infrastructure, but -in order to use the replication Base APIs you must provide your own.</p> - <p>For implementation reasons, all replicated databases must reside in -the data directories set from <a href="../api_reference/C/envset_data_dir.html" class="olink">DB_ENV->set_data_dir()</a> (or in the -default environment home directory, if not using -<a href="../api_reference/C/envset_data_dir.html" class="olink">DB_ENV->set_data_dir()</a>), rather than in a subdirectory below the -specified directory. Care must be taken in applications using -relative pathnames and changing working directories after opening the -environment. In such applications the replication initialization code -may not be able to locate the databases, and applications that change -their working directories may need to use absolute pathnames.</p> - <p>During application initialization, the application performs -three additional tasks: first, it must specify the <a href="../api_reference/C/envopen.html#envopen_DB_INIT_REP" class="olink">DB_INIT_REP</a> -flag when opening its database environment and additionally, a -Replication Manager application must also specify the <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flag; -second, it must provide Berkeley DB information about its communications -infrastructure; and third, it must start the Berkeley DB replication system. -Generally, a replicated application will do normal Berkeley DB recovery and -configuration, exactly like any other transactional application.</p> <p> - Replication Manager applications configure the built-in communications - infrastructure by calling obtaining a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle, and then using - it to configure the local site. It can optionally obtain one or more - <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handles to configure remote sites. Once the environment has - been opened, the application starts the replication system by calling - the <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a> method. -</p> - <p>A Base API application calls the -<a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a> method to configure the entry point to its own -communications infrastructure, and then calls the -<a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> method to join or create the replication group.</p> - <p>When starting the replication system, an application has two choices: -it may choose the group master site explicitly, or alternatively it -may configure all group members as clients and then call for an -election, letting the clients select the master from among -themselves. Either is correct, and the choice is entirely up to the -application.</p> - <p>Replication Manager applications make this choice simply by setting -the flags parameter to the <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a> method.</p> - <p>For a Base API application, the result of -calling <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> is usually the discovery of a master, or the -declaration of the local environment as the master. If a master has -not been discovered after a reasonable amount of time, the application -should call <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> to call for an election.</p> - <p>Consider a Base API application with multiple processes or multiple -environment handles -that modify databases in the replicated environment. All modifications -must be done on the master environment. The first process to join or -create the master environment must call both the -<a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a> and the <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> method. Subsequent -replication processes must at least call the <a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a> method. -Those processes may call the <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> method (as long as they use the -same master or client argument). If multiple processes are modifying -the master environment there must be a unified communication -infrastructure such that messages arriving at clients have a single -master ID. Additionally the application must be structured so that all -incoming messages are able to be processed by a single <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> -handle.</p> - <p>Note that not all processes running in replicated environments need to -call <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a>, <a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a> or <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a>. Read-only -processes running in a master environment do not need to be configured -for replication in any way. Processes running in a client environment -are read-only by definition, and so do not need to be configured for -replication either (although, in the case of clients that may become -masters, it is usually simplest to configure for replication on process -startup rather than trying to reconfigure when the client becomes a -master). Obviously, at least one thread of control on each client must -be configured for replication as messages must be passed between the -master and the client.</p> - <p>Any site in a replication group may have its own private -transactional databases in the environment as well. A site may -create a local database by specifying the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_TXN_NOT_DURABLE" class="olink">DB_TXN_NOT_DURABLE</a> -flag to the <a href="../api_reference/C/dbset_flags.html" class="olink">DB->set_flags()</a> method. The application -must never create a private database with the same name -as a database replicated across the entire environment -as data corruption can result.</p> - <p>For implementation reasons, Base API applications must process -all incoming replication messages -using the same <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handle. It is not required that -a single thread of control process all messages, only that all threads -of control processing messages use the same handle.</p> - <p>No additional calls are required to shut down a database environment -participating in a replication group. The application should shut down -the environment in the usual manner, by calling the <a href="../api_reference/C/envclose.html" class="olink">DB_ENV->close()</a> method. -For Replication Manager applications, this also terminates all network -connections and background processing threads.</p> + The simplest way to build a replicated Berkeley DB + application is to first build (and debug!) the transactional + version of the same application. Then, add a thin replication + layer: application initialization must be changed and the + application's communication infrastructure must be + added. + </p> + <p> + The application initialization changes are relatively + simple. Replication Manager provides a communication + infrastructure, but in order to use the replication Base APIs + you must provide your own. + </p> + <p> + For implementation reasons, all replicated databases must + reside in the data directories set from <a href="../api_reference/C/envadd_data_dir.html" class="olink">DB_ENV->add_data_dir()</a> (or + in the default environment home directory, if not using + <a href="../api_reference/C/envadd_data_dir.html" class="olink">DB_ENV->add_data_dir()</a>), rather than in a subdirectory below the + specified directory. Care must be taken in applications using + relative pathnames and changing working directories after + opening the environment. In such applications the replication + initialization code may not be able to locate the databases, + and applications that change their working directories may + need to use absolute pathnames. + </p> + <p> + During application initialization, the application performs + three additional tasks: first, it must specify the + <a href="../api_reference/C/envopen.html#envopen_DB_INIT_REP" class="olink">DB_INIT_REP</a> flag when opening its database environment and + additionally, a Replication Manager application must also + specify the <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flag; second, it must provide Berkeley + DB information about its communications infrastructure; and + third, it must start the Berkeley DB replication system. + Generally, a replicated application will do normal Berkeley DB + recovery and configuration, exactly like any other + transactional application. + </p> + <p> + Replication Manager applications configure the built-in + communications infrastructure by calling obtaining a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> + handle, and then using it to configure the local site. It can + optionally obtain one or more <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handles to configure + remote sites. Once the environment has been opened, the + application starts the replication system by calling the + <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a> method. + </p> + <p> + A Base API application calls the <a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a> method to + configure the entry point to its own communications + infrastructure, and then calls the <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> method to join + or create the replication group. + </p> + <p> + When starting the replication system, an application has two + choices: it may choose the group master site explicitly, or + alternatively it may configure all group members as clients + and then call for an election, letting the clients select the + master from among themselves. Either is correct, and the + choice is entirely up to the application. + </p> + <p> + Replication Manager applications make this choice simply by + setting the flags parameter to the <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a> + method. + </p> + <p> + For a Base API application, the result of calling <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> + is usually the discovery of a master, or the declaration of + the local environment as the master. If a master has not been + discovered after a reasonable amount of time, the application + should call <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> to call for an election. + </p> + <p> + Consider a Base API application with multiple processes or + multiple environment handles that modify databases in the + replicated environment. All modifications must be done on the + master environment. The first process to join or create the + master environment must call both the <a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a> and the + <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> method. Subsequent replication processes must at + least call the <a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a> method. Those processes may call + the <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> method (as long as they use the same master or + client argument). If multiple processes are modifying the + master environment there must be a unified communication + infrastructure such that messages arriving at clients have a + single master ID. Additionally the application must be + structured so that all incoming messages are able to be + processed by a single <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handle. + </p> + <p> + Note that not all processes running in replicated + environments need to call <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a>, <a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a> or + <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a>. Read-only processes running in a master + environment do not need to be configured for replication in + any way. Processes running in a client environment are + read-only by definition, and so do not need to be configured + for replication either (although, in the case of clients that + may become masters, it is usually simplest to configure for + replication on process startup rather than trying to + reconfigure when the client becomes a master). Obviously, at + least one thread of control on each client must be configured + for replication as messages must be passed between the master + and the client. + </p> + <p> + Any site in a replication group may have its own private + transactional databases in the environment as well. A site may + create a local database by specifying the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_TXN_NOT_DURABLE" class="olink">DB_TXN_NOT_DURABLE</a> + flag to the <a href="../api_reference/C/dbset_flags.html" class="olink">DB->set_flags()</a> method. The application must never + create a private database with the same name as a database + replicated across the entire environment as data corruption + can result. + </p> + <p> + For implementation reasons, Base API applications must + process all incoming replication messages using the same <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> + handle. It is not required that a single thread of control + process all messages, only that all threads of control + processing messages use the same handle. + </p> + <p> + No additional calls are required to shut down a database + environment participating in a replication group. The + application should shut down the environment in the usual + manner, by calling the <a href="../api_reference/C/envclose.html" class="olink">DB_ENV->close()</a> method. For Replication + Manager applications, this also terminates all network + connections and background processing threads. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/rep_base_meth.html b/docs/programmer_reference/rep_base_meth.html index 83e6d7db..adfed96b 100644 --- a/docs/programmer_reference/rep_base_meth.html +++ b/docs/programmer_reference/rep_base_meth.html @@ -3,7 +3,7 @@ <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> - <title>Base API Methods</title> + <title>Base API methods</title> <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> @@ -14,17 +14,15 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Base API Methods</th> + <th colspan="3" align="center">Base API methods</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_mgr_meth.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_comm.html">Next</a></td> </tr> </table> @@ -34,12 +32,14 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="rep_base_meth"></a>Base API Methods</h2> + <h2 class="title" style="clear: both"><a id="rep_base_meth"></a>Base API methods</h2> </div> </div> </div> - <p>Base API applications use the following -Berkeley DB methods.</p> + <p> + Base API applications use the following Berkeley DB + methods. + </p> <div class="variablelist"> <dl> <dt> @@ -49,9 +49,10 @@ Berkeley DB methods.</p> </dt> <dd> <p> - The <a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a> method configures the replication system's - communications infrastructure. - </p> + The <a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a> method configures the + replication system's communications + infrastructure. + </p> </dd> <dt> <span class="term"> @@ -59,10 +60,11 @@ Berkeley DB methods.</p> </span> </dt> <dd> - <p> - The <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> method configures (or reconfigures) an existing database - environment to be a replication master or client. - </p> + <p> + The <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> method configures (or + reconfigures) an existing database environment to + be a replication master or client. + </p> </dd> <dt> <span class="term"> @@ -70,18 +72,21 @@ Berkeley DB methods.</p> </span> </dt> <dd> - <p> - The <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method is used to process incoming messages - from other environments in the replication group. For clients, - it is responsible for accepting log records and updating the - local databases based on messages from the master. For both - the master and the clients, it is responsible for handling - administrative functions (for example, the protocol for dealing - with lost messages), and permitting new clients to join an - active replication group. This method should only be called - after the replication system's communications infrastructure - has been configured via <a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a>. - </p> + <p> + The <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method is used to process + incoming messages from other environments in the + replication group. For clients, it is responsible + for accepting log records and updating the local + databases based on messages from the master. For + both the master and the clients, it is responsible + for handling administrative functions (for + example, the protocol for dealing with lost + messages), and permitting new clients to join an + active replication group. This method should only + be called after the replication system's + communications infrastructure has been configured + via <a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a>. + </p> </dd> <dt> <span class="term"> @@ -89,12 +94,13 @@ Berkeley DB methods.</p> </span> </dt> <dd> - <p> - The <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method causes the replication group to elect a - new master; it is called whenever contact with the master is - lost and the application wants the remaining sites to select a - new master. - </p> + <p> + The <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method causes the replication + group to elect a new master; it is called whenever + contact with the master is lost and the + application wants the remaining sites to select a + new master. + </p> </dd> <dt> <span class="term"> @@ -102,11 +108,12 @@ Berkeley DB methods.</p> </span> </dt> <dd> - <p> - The <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> method is needed for applications to - discover important replication-related events, such as the - result of an election and appointment of a new master. - </p> + <p> + The <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> method is needed for + applications to discover important + replication-related events, such as the result of + an election and appointment of a new master. + </p> </dd> <dt> <span class="term"> @@ -115,9 +122,9 @@ Berkeley DB methods.</p> </dt> <dd> <p> - The <a href="../api_reference/C/reppriority.html" class="olink">DB_ENV->rep_set_priority()</a> method configures the local site's priority for - the purpose of elections. - </p> + The <a href="../api_reference/C/reppriority.html" class="olink">DB_ENV->rep_set_priority()</a> method configures the local + site's priority for the purpose of elections. + </p> </dd> <dt> <span class="term"> @@ -125,11 +132,11 @@ Berkeley DB methods.</p> </span> </dt> <dd> - <p> - This method optionally configures various timeout values. Otherwise - default timeout values as specified in <a href="../api_reference/C/repset_timeout.html" class="olink">DB_ENV->rep_set_timeout()</a> are - used. - </p> + <p> + This method optionally configures various + timeout values. Otherwise default timeout values + as specified in <a href="../api_reference/C/repset_timeout.html" class="olink">DB_ENV->rep_set_timeout()</a> are used. + </p> </dd> <dt> <span class="term"> @@ -137,18 +144,21 @@ Berkeley DB methods.</p> </span> </dt> <dd> - <p> - The <a href="../api_reference/C/repset_limit.html" class="olink">DB_ENV->rep_set_limit()</a> method imposes an upper bound on the amount of - data that will be sent in response to a single call to - <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a>. During client recovery, that is, when a replica - site is trying to synchronize with the master, clients may ask - the master for a large number of log records. If it is going - to harm an application for the master message loop to remain - busy for an extended period transmitting records to the - replica, then the application will want to use <a href="../api_reference/C/repset_limit.html" class="olink">DB_ENV->rep_set_limit()</a> to - limit the amount of data the master will send before - relinquishing control and accepting other messages. - </p> + <p> + The <a href="../api_reference/C/repset_limit.html" class="olink">DB_ENV->rep_set_limit()</a> method imposes an upper bound on + the amount of data that will be sent in response + to a single call to <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a>. During client + recovery, that is, when a replica site is trying + to synchronize with the master, clients may ask + the master for a large number of log records. If + it is going to harm an application for the master + message loop to remain busy for an extended period + transmitting records to the replica, then the + application will want to use <a href="../api_reference/C/repset_limit.html" class="olink">DB_ENV->rep_set_limit()</a> to limit + the amount of data the master will send before + relinquishing control and accepting other + messages. + </p> </dd> <dt> <span class="term"> @@ -157,16 +167,24 @@ Berkeley DB methods.</p> </dt> <dd> <p> - This method sets a threshold for the minimum and maximum time that - a client waits before requesting retransmission of a missing - message. - </p> + This method sets a threshold for the minimum + and maximum time that a client waits before + requesting retransmission of a missing message. + </p> </dd> </dl> </div> - <p>In addition to the methods previously described, Base API applications -may also call the following methods, as needed: -<a href="../api_reference/C/repstat.html" class="olink">DB_ENV->rep_stat()</a>, <a href="../api_reference/C/repsync.html" class="olink">DB_ENV->rep_sync()</a> and <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a>.</p> + <p> + Base API applications may configure one or more view sites. + A view is a full or partial copy of the replicated data that + does not otherwise participate in the replication group. For + more information, see <a class="xref" href="rep_partview.html" title="Replication views">Replication views</a>. + </p> + <p> + In addition to the methods previously described, Base API + applications may also call the following methods, as needed: + <a href="../api_reference/C/repstat.html" class="olink">DB_ENV->rep_stat()</a>, <a href="../api_reference/C/repsync.html" class="olink">DB_ENV->rep_sync()</a> and <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/rep_bulk.html b/docs/programmer_reference/rep_bulk.html index 476666e6..630e91bf 100644 --- a/docs/programmer_reference/rep_bulk.html +++ b/docs/programmer_reference/rep_bulk.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_init.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_trans.html">Next</a></td> </tr> </table> @@ -38,27 +36,38 @@ </div> </div> </div> - <p>Sites in a replication group may be configured to use bulk transfer by -calling the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method with the <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_BULK" class="olink">DB_REP_CONF_BULK</a> -flag. When configured for bulk transfer, sites will accumulate records -in a buffer and transfer them to another site in a single network -transfer. Configuring bulk transfer makes sense for master sites, of -course. Additionally, applications using client-to-client -synchronization may find it helpful to configure bulk transfer for -client sites as well.</p> - <p>When a master is generating new log records, or any information request -is made of a master, and bulk transfer has been configured, records will -accumulate in a bulk buffer. The bulk buffer will be sent to the client -if either the buffer is full or if a permanent record (for example, a -transaction commit or checkpoint record) is queued for the client.</p> - <p>When a client is responding to another client's request for information, -and bulk transfer has been configured, records will accumulate in a bulk -buffer. The bulk buffer will be sent to the client when the buffer is -full or when the client's request has been satisfied; no particular type -of record will cause the buffer to be sent.</p> - <p>The size of the bulk buffer itself is internally determined and cannot -be configured. However, the overall size of a transfer may be limited -using the <a href="../api_reference/C/repset_limit.html" class="olink">DB_ENV->rep_set_limit()</a> method.</p> + <p> + Sites in a replication group may be configured to use bulk + transfer by calling the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method with the + <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_BULK" class="olink">DB_REP_CONF_BULK</a> flag. When configured for bulk transfer, + sites will accumulate records in a buffer and transfer them to + another site in a single network transfer. Configuring bulk + transfer makes sense for master sites, of course. + Additionally, applications using client-to-client + synchronization may find it helpful to configure bulk transfer + for client sites as well. + </p> + <p> + When a master is generating new log records, or any + information request is made of a master, and bulk transfer has + been configured, records will accumulate in a bulk buffer. The + bulk buffer will be sent to the client if either the buffer is + full or if a permanent record (for example, a transaction + commit or checkpoint record) is queued for the client. + </p> + <p> + When a client is responding to another client's request for + information, and bulk transfer has been configured, records + will accumulate in a bulk buffer. The bulk buffer will be sent + to the client when the buffer is full or when the client's + request has been satisfied; no particular type of record will + cause the buffer to be sent. + </p> + <p> + The size of the bulk buffer itself is internally determined + and cannot be configured. However, the overall size of a + transfer may be limited using the <a href="../api_reference/C/repset_limit.html" class="olink">DB_ENV->rep_set_limit()</a> method. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/rep_clock_skew.html b/docs/programmer_reference/rep_clock_skew.html index 515ae468..f0980de4 100644 --- a/docs/programmer_reference/rep_clock_skew.html +++ b/docs/programmer_reference/rep_clock_skew.html @@ -3,7 +3,7 @@ <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> - <title>Clock Skew</title> + <title>Clock skew</title> <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> @@ -14,17 +14,15 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Clock Skew</th> + <th colspan="3" align="center">Clock skew</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_ryw.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="repmgr_channels.html">Next</a></td> </tr> </table> @@ -34,56 +32,72 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="rep_clock_skew"></a>Clock Skew</h2> + <h2 class="title" style="clear: both"><a id="rep_clock_skew"></a>Clock skew</h2> </div> </div> </div> - <p>Since master leases take into account a timeout that is used across -all sites in a replication group, leases must also take into account -any known skew (or drift) between the clocks on different machines -in the group. The guarantees provided by master leases take clock -skew into account. Consider a replication group where a client's -clock is running faster than the master's clock and the group has -a lease timeout of 5 seconds. If clock skew is not taken into -account, eventually, the client will believe that 5 seconds have -passed faster than the master and that client may then grant its -lease to another site. Meanwhile, the master site does not believe -5 seconds have passed because its clock is slower, and it believes -it still holds a valid lease grant. For this reason, Berkeley DB compensates -for clock skew.</p> - <p>The master of a group using leases must account for skew -in case that site has the slowest clock in the group. This computation -avoids the problem of a master site believing a lease grant is valid -too long. Clients in a group must account for skew in case they -have the fastest clock in the group. This computation avoids -the problem of a client site expiring its grant too soon and -potentially granting a lease to a different site. Berkeley DB uses -a conservative computation and accounts for clock skew on both -sides, yielding a double compensation.</p> - <p>The <a href="../api_reference/C/repclockskew.html" class="olink">DB_ENV->rep_set_clockskew()</a> method takes the values for both the fastest -and slowest clocks in the entire replication group as parameters. -The values passed in must be the same for all sites in the group. -If the user knows the maximum clock drift of their sites, then those -values can be expressed as a relative percentage. Or, if the user -runs an experiment then the actual values can be used.</p> - <p>For example, suppose the user knows that there is a maximum drift -rate of 2% among sites in the group. The user should pass in -102 and 100 for the fast and slow clock values respectively. -That is an unusually large value, so suppose, for example, the -rate is 0.03% among sites in the group. The user should pass in -10003 and 10000 for the fast and slow clock values. Those values -can be used to express the level of precision the user needs.</p> - <p>An example of an experiment a user can run to help determine skew -would be to write a program that started simultaneously on all -sites in the group. Suppose, after 1 day (86400 seconds), one -site shows 86400 seconds and the other site shows it ran faster -and it indicates 86460 seconds has passed. -The user can use 86460 and 86400 for their parameters for the -fast and slow clock values.</p> - <p>Since Berkeley DB is using those fast and slow clock values to compute -a ratio internally, if the user cannot detect or measure any -clock skew, then the same value should be passed in for both -parameters, such as 1 and 1.</p> + <p> + Since master leases take into account a timeout that is used + across all sites in a replication group, leases must also take + into account any known skew (or drift) between the clocks on + different machines in the group. The guarantees provided by + master leases take clock skew into account. Consider a + replication group where a client's clock is running faster + than the master's clock and the group has a lease timeout of 5 + seconds. If clock skew is not taken into account, eventually, + the client will believe that 5 seconds have passed faster than + the master and that client may then grant its lease to another + site. Meanwhile, the master site does not believe 5 seconds + have passed because its clock is slower, and it believes it + still holds a valid lease grant. For this reason, Berkeley DB + compensates for clock skew. + </p> + <p> + The master of a group using leases must account for skew in + case that site has the slowest clock in the group. This + computation avoids the problem of a master site believing a + lease grant is valid too long. Clients in a group must account + for skew in case they have the fastest clock in the group. + This computation avoids the problem of a client site expiring + its grant too soon and potentially granting a lease to a + different site. Berkeley DB uses a conservative computation + and accounts for clock skew on both sides, yielding a double + compensation. + </p> + <p> + The <a href="../api_reference/C/repclockskew.html" class="olink">DB_ENV->rep_set_clockskew()</a> method takes the values for both the + fastest and slowest clocks in the entire replication group as + parameters. The values passed in must be the same for all + sites in the group. If the user knows the maximum clock drift + of their sites, then those values can be expressed as a + relative percentage. Or, if the user runs an experiment then + the actual values can be used. + </p> + <p> + For example, suppose the user knows that there is a maximum + drift rate of 2% among sites in the group. The user should + pass in 102 and 100 for the fast and slow clock values + respectively. That is an unusually large value, so suppose, + for example, the rate is 0.03% among sites in the group. The + user should pass in 10003 and 10000 for the fast and slow + clock values. Those values can be used to express the level of + precision the user needs. + </p> + <p> + An example of an experiment a user can run to help determine + skew would be to write a program that started simultaneously + on all sites in the group. Suppose, after 1 day (86400 + seconds), one site shows 86400 seconds and the other site + shows it ran faster and it indicates 86460 seconds has passed. + The user can use 86460 and 86400 for their parameters for the + fast and slow clock values. + </p> + <p> + Since Berkeley DB is using those fast and slow clock values + to compute a ratio internally, if the user cannot detect or + measure any clock skew, then the same value should be passed + in for both parameters, such as 1 and 1. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/rep_comm.html b/docs/programmer_reference/rep_comm.html index 0adad6dc..aaf5df34 100644 --- a/docs/programmer_reference/rep_comm.html +++ b/docs/programmer_reference/rep_comm.html @@ -8,13 +8,13 @@ <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> <link rel="up" href="rep.html" title="Chapter 12. Berkeley DB Replication" /> - <link rel="prev" href="rep_base_meth.html" title="Base API Methods" /> + <link rel="prev" href="rep_base_meth.html" title="Base API methods" /> <link rel="next" href="rep_newsite.html" title="Connecting to a new site" /> </head> <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_base_meth.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_newsite.html">Next</a></td> </tr> </table> @@ -38,47 +36,61 @@ </div> </div> </div> - <p>Replication Manager provides a built-in communications -infrastructure.</p> - <p>Base API applications must provide -their own communications infrastructure, which is typically written with one -or more threads of control looping on one or more communication -channels, receiving and sending messages. These threads accept messages -from remote environments for the local database environment, and accept -messages from the local environment for remote environments. Messages -from remote environments are passed to the local database environment -using the <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method. Messages from the local environment are -passed to the application for transmission using the callback function -specified to the <a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a> method.</p> - <p>Processes establish communication channels by calling the -<a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a> method, regardless of whether they are running in -client or server environments. This method specifies the <span class="bold"><strong>send</strong></span> -function, a callback function used by Berkeley DB for sending messages to -other database environments in the replication group. The <span class="bold"><strong>send</strong></span> -function takes an environment ID and two opaque data objects. It is the -responsibility of the <span class="bold"><strong>send</strong></span> function to transmit the information -in the two data objects to the database environment corresponding to the -ID, with the receiving application then calling the <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method -to process the message.</p> - <p>The details of the transport mechanism are left entirely to the -application; the only requirement is that the data buffer and size of -each of the control and rec <a href="../api_reference/C/dbt.html" class="olink">DBT</a>s passed to the <span class="bold"><strong>send</strong></span> -function on the sending site be faithfully copied and delivered to the -receiving site by means of a call to <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> with -corresponding arguments. Messages that are broadcast (whether by -broadcast media or when directed by setting the -<a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a> method's envid parameter DB_EID_BROADCAST), should -not be processed by the message sender. In all cases, the application's -transport media or software must ensure that <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> is -never called with a message intended for a different database -environment or a broadcast message sent from the same environment on -which <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> will be called. -The <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method is -free-threaded; it is safe to deliver any number of messages -simultaneously, and from any arbitrary thread or process in the Berkeley DB -environment.</p> - <p>There are a number of informational returns from the -<a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method:</p> + <p> + Replication Manager provides a built-in communications + infrastructure. + </p> + <p> + Base API applications must provide their own communications + infrastructure, which is typically written with one or more + threads of control looping on one or more communication + channels, receiving and sending messages. These threads accept + messages from remote environments for the local database + environment, and accept messages from the local environment + for remote environments. Messages from remote environments are + passed to the local database environment using the + <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method. Messages from the local environment are + passed to the application for transmission using the callback + function specified to the <a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a> method. + </p> + <p> + Processes establish communication channels by calling the + <a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a> method, regardless of whether they are running + in client or server environments. This method specifies the + <span class="bold"><strong>send</strong></span> function, a callback + function used by Berkeley DB for sending messages to other + database environments in the replication group. The <span class="bold"><strong>send</strong></span> function takes an environment + ID and two opaque data objects. It is the responsibility of + the <span class="bold"><strong>send</strong></span> function to transmit + the information in the two data objects to the database + environment corresponding to the ID, with the receiving + application then calling the <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method to process + the message. + </p> + <p> + The details of the transport mechanism are left entirely to + the application; the only requirement is that the data buffer + and size of each of the control and rec <a href="../api_reference/C/dbt.html" class="olink">DBT</a>s passed to the + <span class="bold"><strong>send</strong></span> function on the + sending site be faithfully copied and delivered to the + receiving site by means of a call to <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> with + corresponding arguments. Messages that are broadcast (whether + by broadcast media or when directed by setting the + <a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a> method's envid parameter DB_EID_BROADCAST), + should not be processed by the message sender. In all cases, + the application's transport media or software must ensure that + <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> is never called with a message intended for a + different database environment or a broadcast message sent + from the same environment on which <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> will be + called. The <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method is free-threaded; it is safe + to deliver any number of messages simultaneously, and from any + arbitrary thread or process in the Berkeley DB + environment. + </p> + <p> + There are a number of informational returns from the + <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method: + </p> <div class="variablelist"> <dl> <dt> @@ -87,15 +99,16 @@ environment.</p> </span> </dt> <dd> - <p> - When <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> returns <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_DUPMASTER" class="olink">DB_REP_DUPMASTER</a>, it means that - another database environment in the replication group also - believes itself to be the master. The application should - complete all active transactions, close all open database - handles, reconfigure itself as a client using the - <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> method, and then call for an election by calling - the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method. - </p> + <p> + When <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> returns <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_DUPMASTER" class="olink">DB_REP_DUPMASTER</a>, + it means that another database environment in the + replication group also believes itself to be the + master. The application should complete all active + transactions, close all open database handles, + reconfigure itself as a client using the + <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> method, and then call for an election + by calling the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method. + </p> </dd> <dt> <span class="term"> @@ -103,12 +116,13 @@ environment.</p> </span> </dt> <dd> - <p> - When <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> returns <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_HOLDELECTION" class="olink">DB_REP_HOLDELECTION</a>, it means that - another database environment in the replication group has - called for an election. The application should call the - <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method. - </p> + <p> + When <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> returns + <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_HOLDELECTION" class="olink">DB_REP_HOLDELECTION</a>, it means that another + database environment in the replication group has + called for an election. The application should + call the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method. + </p> </dd> <dt> <span class="term"> @@ -116,13 +130,13 @@ environment.</p> </span> </dt> <dd> - <p> - When <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> returns <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_IGNORE" class="olink">DB_REP_IGNORE</a>, it means that this - message cannot be processed. This is normally an indication - that this message is irrelevant to the current replication - state, such as a message from an old generation that arrived - late. - </p> + <p> + When <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> returns <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_IGNORE" class="olink">DB_REP_IGNORE</a>, it + means that this message cannot be processed. This + is normally an indication that this message is + irrelevant to the current replication state, such + as a message from an old master that arrived late. + </p> </dd> <dt> <span class="term"> @@ -131,13 +145,14 @@ environment.</p> </dt> <dd> <p> - When <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> returns <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_ISPERM" class="olink">DB_REP_ISPERM</a>, it means a permanent - record, perhaps a message previously returned as - <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_NOTPERM" class="olink">DB_REP_NOTPERM</a>, was successfully written to disk. This - record may have filled a gap in the log record that allowed - additional records to be written. The <span class="bold"><strong>ret_lsnp</strong></span> contains the maximum LSN of - the permanent records written. - </p> + When <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> returns <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_ISPERM" class="olink">DB_REP_ISPERM</a>, it + means a permanent record, perhaps a message + previously returned as <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_NOTPERM" class="olink">DB_REP_NOTPERM</a>, was + successfully written to disk. This record may have + filled a gap in the log record that allowed + additional records to be written. The <span class="bold"><strong>ret_lsnp</strong></span> contains the + maximum LSN of the permanent records written. + </p> </dd> <dt> <span class="term"> @@ -146,12 +161,13 @@ environment.</p> </dt> <dd> <p> - When <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> returns <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_NEWSITE" class="olink">DB_REP_NEWSITE</a>, it means that a - message from a previously unknown member of the replication - group has been received. The application should reconfigure - itself as necessary so it is able to send messages to this - site. - </p> + When <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> returns <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_NEWSITE" class="olink">DB_REP_NEWSITE</a>, it + means that a message from a previously unknown + member of the replication group has been received. + The application should reconfigure itself as + necessary so it is able to send messages to this + site. + </p> </dd> <dt> <span class="term"> @@ -160,16 +176,19 @@ environment.</p> </dt> <dd> <p> - When <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> returns <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_NOTPERM" class="olink">DB_REP_NOTPERM</a>, it means a message - marked as <a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a> was processed successfully but was not - written to disk. This is normally an indication that one or - more messages, which should have arrived before this message, - have not yet arrived. This operation will be written to disk - when the missing messages arrive. The <span class="bold"><strong>ret_lsnp</strong></span> argument will contain the - LSN of this record. The application should take whatever - action is deemed necessary to retain its recoverability - characteristics. - </p> + When <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> returns <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_NOTPERM" class="olink">DB_REP_NOTPERM</a>, it + means a message marked as <a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a> was + processed successfully but was not written to + disk. This is normally an indication that one or + more messages, which should have arrived before + this message, have not yet arrived. This operation + will be written to disk when the missing messages + arrive. The <span class="bold"><strong>ret_lsnp</strong></span> argument + will contain the + LSN of this record. The application should take + whatever action is deemed necessary to retain its + recoverability characteristics. + </p> </dd> </dl> </div> @@ -185,7 +204,7 @@ environment.</p> <td width="40%" align="right"> <a accesskey="n" href="rep_newsite.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Base API Methods </td> + <td width="40%" align="left" valign="top">Base API methods </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/rep_elect.html b/docs/programmer_reference/rep_elect.html index 33427c6a..bf3bd062 100644 --- a/docs/programmer_reference/rep_elect.html +++ b/docs/programmer_reference/rep_elect.html @@ -8,13 +8,13 @@ <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> <link rel="up" href="rep.html" title="Chapter 12. Berkeley DB Replication" /> - <link rel="prev" href="rep_mgr_ack.html" title="Choosing a Replication Manager Ack Policy" /> + <link rel="prev" href="rep_mgr_ack.html" title="Choosing a Replication Manager acknowledgement policy" /> <link rel="next" href="rep_mastersync.html" title="Synchronizing with a master" /> </head> <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_mgr_ack.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_mastersync.html">Next</a></td> </tr> </table> @@ -38,152 +36,197 @@ </div> </div> </div> - <p>Replication Manager automatically conducts elections when necessary, -based on configuration information supplied to the -<a href="../api_reference/C/reppriority.html" class="olink">DB_ENV->rep_set_priority()</a> method, unless the application turns off automatic -elections using the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method.</p> - <p>It is the responsibility of a Base API application -to initiate elections if desired. It is never dangerous -to hold an election, as the Berkeley DB election process ensures there is -never more than a single master database environment. Clients should -initiate an election whenever they lose contact with the master -environment, whenever they see a return of <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_HOLDELECTION" class="olink">DB_REP_HOLDELECTION</a> -from the <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method, or when, for whatever reason, they do -not know who the master is. It is not necessary for applications to -immediately hold elections when they start, as any existing master -will be discovered after calling <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a>. If no master has -been found after a short wait period, then the application should call -for an election.</p> - <p>For a client to win an election, the replication group must currently -have no master, and the client must have the most recent log records. -In the case of clients having equivalent log records, the priority of -the database environments participating in the election will determine -the winner. The application specifies the minimum number of replication -group members that must participate in an election for a winner to be -declared. We recommend at least ((N/2) + 1) members. If fewer than the -simple majority are specified, a warning will be given.</p> - <p>If an application's policy for what site should win an election can be -parameterized in terms of the database environment's information (that -is, the number of sites, available log records and a relative priority -are all that matter), then Berkeley DB can handle all elections transparently. -However, there are cases where the application has more complete -knowledge and needs to affect the outcome of elections. For example, -applications may choose to handle master selection, explicitly -designating master and client sites. Applications in these cases may -never need to call for an election. Alternatively, applications may -choose to use <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a>'s arguments to force the correct outcome -to an election. That is, if an application has three sites, A, B, and -C, and after a failure of C determines that A must become the winner, -the application can guarantee an election's outcome by specifying -priorities appropriately after an election:</p> + <p> + Replication Manager automatically conducts elections when + necessary, based on configuration information supplied to the + <a href="../api_reference/C/reppriority.html" class="olink">DB_ENV->rep_set_priority()</a> method, unless the application turns off + automatic elections using the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method. + </p> + <p> + It is the responsibility of a Base API application to + initiate elections if desired. It is never dangerous to hold + an election, as the Berkeley DB election process ensures there + is never more than a single master database environment. + Clients should initiate an election whenever they lose contact + with the master environment, whenever they see a return of + <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_HOLDELECTION" class="olink">DB_REP_HOLDELECTION</a> from the <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method, or when, + for whatever reason, they do not know who the master is. It is + not necessary for applications to immediately hold elections + when they start, as any existing master will be discovered + after calling <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a>. If no master has been found after a + short wait period, then the application should call for an + election. + </p> + <p> + For a client to win an election, the replication group must + currently have no master, and the client must have the most + recent log records. In the case of clients having equivalent + log records, the priority of the database environments + participating in the election will determine the winner. The + application specifies the minimum number of replication group + members that must participate in an election for a winner to + be declared. We recommend at least ((N/2) + 1) members. If + fewer than the simple majority are specified, a warning will + be given. + </p> + <p> + If an application's policy for what site should win an + election can be parameterized in terms of the database + environment's information (that is, the number of sites, + available log records and a relative priority are all that + matter), then Berkeley DB can handle all elections + transparently. However, there are cases where the application + has more complete knowledge and needs to affect the outcome of + elections. For example, applications may choose to handle + master selection, explicitly designating master and client + sites. Applications in these cases may never need to call for + an election. Alternatively, applications may choose to use + <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a>'s arguments to force the correct outcome to an + election. That is, if an application has three sites, A, B, + and C, and after a failure of C determines that A must become + the winner, the application can guarantee an election's + outcome by specifying priorities appropriately after an + election: + </p> <pre class="programlisting">on A: priority 100, nsites 2 on B: priority 0, nsites 2</pre> - <p>It is dangerous to configure more than one master environment using the -<a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> method, and applications should be careful not to do so. -Applications should only configure themselves as the master environment -if they are the only possible master, or if they have won an election. -An application knows it has won an election when it receives the -<a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_ELECTED" class="olink">DB_EVENT_REP_ELECTED</a> event.</p> <p> - Normally, when a master failure is detected it is desired that an - election finish quickly so the application can continue to service - updates. Also, participating sites are already up and can participate. - However, in the case of restarting a whole group after an - administrative shutdown, it is possible that a slower booting site had - later logs than any other site. To cover that case, an application - would like to give the election more time to ensure all sites have a - chance to participate. Since it is intractable for a starting site to - determine which case the whole group is in, the use of a long timeout - gives all sites a reasonable chance to participate. If an application - wanting full participation sets the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method's - <span class="bold"><strong>nvotes</strong></span> argument to the number of sites - in the group and one site does not reboot, a master can never be elected - without manual intervention. -</p> + It is dangerous to configure more than one master + environment using the <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> method, and applications + should be careful not to do so. Applications should only + configure themselves as the master environment if they are the + only possible master, or if they have won an election. An + application knows it has won an election when it receives the + <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_ELECTED" class="olink">DB_EVENT_REP_ELECTED</a> event. + </p> + <p> + Normally, when a master failure is detected it is desired + that an election finish quickly so the application can + continue to service updates. Also, participating sites are + already up and can participate. However, in the case of + restarting a whole group after an administrative shutdown, it + is possible that a slower booting site had later logs than any + other site. To cover that case, an application would like to + give the election more time to ensure all sites have a chance + to participate. Since it is intractable for a starting site to + determine which case the whole group is in, the use of a long + timeout gives all sites a reasonable chance to participate. If + an application wanting full participation sets the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> + method's <span class="bold"><strong>nvotes</strong></span> argument to + the number of sites in the group and one site does not reboot, + a master can never be elected without manual intervention. + </p> + <p> + In those cases, the desired action at a group level is to + hold a full election if all sites crashed and a majority + election if a subset of sites crashed or rebooted. Since an + individual site cannot know which number of votes to require, + a mechanism is available to accomplish this using timeouts. By + setting a long timeout (perhaps on the order of minutes) using + the <span class="bold"><strong>DB_REP_FULL_ELECTION_TIMEOUT</strong></span> flag to the + <a href="../api_reference/C/repset_timeout.html" class="olink">DB_ENV->rep_set_timeout()</a> method, an application can allow Berkeley DB to + elect a master even without full participation. Sites may also + want to set a normal election timeout for majority based + elections using the <span class="bold"><strong>DB_REP_ELECTION_TIMEOUT</strong></span> + flag to the <a href="../api_reference/C/repset_timeout.html" class="olink">DB_ENV->rep_set_timeout()</a> method. + </p> + <p> + Consider 3 sites, A, B, and C where A is the master. In the + case where all three sites crash and all reboot, all sites + will set a timeout for a full election, say 10 minutes, but + only require a majority for <span class="bold"><strong>nvotes</strong></span> to + the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method. Once all + three sites are booted the election will complete immediately + if they reboot within 10 minutes of each other. Consider if + all three sites crash and only two reboot. The two sites will + enter the election, but after the 10 minute timeout they will + elect with the majority of two sites. Using the full election + timeout sets a threshold for allowing a site to reboot and + rejoin the group. + </p> + <p> + To add a database environment to the replication group with + the intent of it becoming the master, first add it as a + client. Since it may be out-of-date with respect to the + current master, allow it to update itself from the current + master. Then, shut the current master down. Presumably, the + added client will win the subsequent election. If the client + does not win the election, it is likely that it was not given + sufficient time to update itself with respect to the current + master. + </p> + <p> + If a client is unable to find a master or win an election, + it means that the network has been partitioned and there are + not enough environments participating in the election for one + of the participants to win. In this case, the application + should repeatedly call <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> and <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a>, alternating + between attempting to discover an existing master, and holding + an election to declare a new one. In desperate circumstances, + an application could simply declare itself the master by + calling <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a>, or by reducing the number of participants + required to win an election until the election is won. Neither + of these solutions is recommended: in the case of a network + partition, either of these choices can result in there being + two masters in one replication group, and the databases in the + environment might irretrievably diverge as they are modified + in different ways by the masters. + </p> + <p> + Note that this presents a special problem for a replication + group consisting of only two environments. If a master site + fails, the remaining client can never comprise a majority of + sites in the group. If the client application can reach a + remote network site, or some other external tie-breaker, it + may be able to determine whether it is safe to declare itself + master. Otherwise it must choose between providing + availability of a writable master (at the risk of duplicate + masters), or strict protection against duplicate masters (but + no master when a failure occurs). Replication Manager offers + this choice via the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method + <a href="../api_reference/C/repconfig.html#config_DB_REPMGR_CONF_2SITE_STRICT" class="olink">DB_REPMGR_CONF_2SITE_STRICT</a> flag. Base API applications can + accomplish this by judicious setting of the <span class="bold"><strong>nvotes</strong></span> and <span class="bold"><strong>nsites</strong></span> + parameters to the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method. + </p> <p> -In those cases, the desired action at a group level is to hold -a full election if all sites crashed and a majority election if -a subset of sites crashed or rebooted. Since an individual site cannot know -which number of votes to require, a mechanism is available to -accomplish this using timeouts. By setting a long timeout (perhaps -on the order of minutes) using the <span class="bold"><strong>DB_REP_FULL_ELECTION_TIMEOUT</strong></span> -flag to the <a href="../api_reference/C/repset_timeout.html" class="olink">DB_ENV->rep_set_timeout()</a> method, an application can -allow Berkeley DB to elect a master even without full participation. -Sites may also want to set a normal election timeout for majority -based elections using the <span class="bold"><strong>DB_REP_ELECTION_TIMEOUT</strong></span> flag -to the <a href="../api_reference/C/repset_timeout.html" class="olink">DB_ENV->rep_set_timeout()</a> method.</p> + It is possible for a less-preferred database environment to + win an election if a number of systems crash at the same time. + Because an election winner is declared as soon as enough + environments participate in the election, the environment on a + slow booting but well-connected machine might lose to an + environment on a badly connected but faster booting machine. + In the case of a number of environments crashing at the same + time (for example, a set of replicated servers in a single + machine room), applications should bring the database + environments on line as clients initially (which will allow + them to process read queries immediately), and then hold an + election after sufficient time has passed for the slower + booting machines to catch up. + </p> <p> -Consider 3 sites, A, B, and C where A is the master. In the -case where all three sites crash and all reboot, all sites -will set a timeout for a full election, say 10 minutes, but only -require a majority for <span class="bold"><strong>nvotes</strong></span> to the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method. -Once all three sites are booted the election will complete -immediately if they reboot within 10 minutes of each other. Consider -if all three sites crash and only two reboot. The two sites will -enter the election, but after the 10 minute timeout they will -elect with the majority of two sites. Using the full election -timeout sets a threshold for allowing a site to reboot and rejoin -the group.</p> - <p>To add a database environment to the replication group with the intent -of it becoming the master, first add it as a client. Since it may be -out-of-date with respect to the current master, allow it to update -itself from the current master. Then, shut the current master down. -Presumably, the added client will win the subsequent election. If the -client does not win the election, it is likely that it was not given -sufficient time to update itself with respect to the current master.</p> - <p>If a client is unable to find a master or win an election, it means that -the network has been partitioned and there are not enough environments -participating in the election for one of the participants to win. -In this case, the application should repeatedly call <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> -and <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a>, alternating between attempting to discover an -existing master, and holding an election to declare a new one. In -desperate circumstances, an application could simply declare itself the -master by calling <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a>, or by reducing the number of -participants required to win an election until the election is won. -Neither of these solutions is recommended: in the case of a network -partition, either of these choices can result in there being two masters -in one replication group, and the databases in the environment might -irretrievably diverge as they are modified in different ways by the -masters.</p> - <p>Note that this presents a special problem for a replication group -consisting of only two environments. If a master site fails, the -remaining client can never comprise a majority of sites in the group. -If the client application can reach a remote network site, or some other -external tie-breaker, it may be able to determine whether it is safe -to declare itself master. Otherwise it must choose between providing -availability of a writable master (at the risk of duplicate masters), -or strict protection against duplicate masters (but no master when a -failure occurs). Replication Manager offers this choice via the -<a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method <a href="../api_reference/C/repconfig.html#config_DB_REPMGR_CONF_2SITE_STRICT" class="olink">DB_REPMGR_CONF_2SITE_STRICT</a> flag. Base API -applications can accomplish this by judicious setting of the -<span class="bold"><strong>nvotes</strong></span> and -<span class="bold"><strong>nsites</strong></span> parameters to the -<a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method. </p> - <p>It is possible for a less-preferred database environment to win an -election if a number of systems crash at the same time. Because an -election winner is declared as soon as enough environments participate -in the election, the environment on a slow booting but well-connected -machine might lose to an environment on a badly connected but faster -booting machine. In the case of a number of environments crashing at -the same time (for example, a set of replicated servers in a single -machine room), applications should bring the database environments on -line as clients initially (which will allow them to process read queries -immediately), and then hold an election after sufficient time has passed -for the slower booting machines to catch up.</p> - <p>If, for any reason, a less-preferred database environment becomes the -master, it is possible to switch masters in a replicated environment. -For example, the preferred master crashes, and one of the replication -group clients becomes the group master. In order to restore the -preferred master to master status, take the following steps:</p> + If, for any reason, a less-preferred database environment + becomes the master, it is possible to switch masters in a + replicated environment. For example, the preferred master + crashes, and one of the replication group clients becomes the + group master. In order to restore the preferred master to + master status, take the following steps: + </p> <div class="orderedlist"> <ol type="1"> - <li>The preferred master should reboot and re-join the replication group -as a client.</li> - <li>Once the preferred master has caught up with the replication group, the -application on the current master should complete all active transactions -and reconfigure itself as a client using the <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> method.</li> - <li>Then, the current or preferred master should call for an election using -the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method.</li> + <li> + The preferred master should reboot and re-join the + replication group as a client. + </li> + <li> + Once the preferred master has caught up with the + replication group, the application on the current master + should complete all active transactions and reconfigure + itself as a client using the <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> method. + </li> + <li> + Then, the current or preferred master should call + for an election using the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method. + </li> </ol> </div> </div> @@ -198,11 +241,12 @@ the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elec <td width="40%" align="right"> <a accesskey="n" href="rep_mastersync.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Choosing a Replication Manager Ack Policy </td> + <td width="40%" align="left" valign="top">Choosing a Replication Manager acknowledgement policy </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Synchronizing with a master</td> + <td width="40%" align="right" valign="top"> Synchronizing with a + master</td> </tr> </table> </div> diff --git a/docs/programmer_reference/rep_ex.html b/docs/programmer_reference/rep_ex.html index 3e5f9e62..f458dbc1 100644 --- a/docs/programmer_reference/rep_ex.html +++ b/docs/programmer_reference/rep_ex.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_faq.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_ex_comm.html">Next</a></td> </tr> </table> @@ -38,180 +36,196 @@ </div> </div> </div> - <p>Ex_rep, found in the <code class="filename">examples_c/ex_rep</code> subdirectory -of the Berkeley DB distribution, is a simple but complete demonstration of a -replicated application. The application is a mock stock ticker. The -master accepts a stock symbol and a numerical value as input, and -stores this information into a replicated database; either master or -clients can display the contents of the database, given an empty input -line.</p> - <p>There are two versions of the application: ex_rep_mgr uses Replication -Manager, while ex_rep_base uses the replication Base API. This is -intended to demonstrate that, while the basic function of the -application is the same in either case, the replication support -infrastructure differs markedly.</p> - <p>The communication infrastructure demonstrated with ex_rep_base has the -same dependencies on system networking and threading support as does -the Replication Manager (see the <a class="xref" href="rep.html#rep_intro" title="Replication introduction">Replication introduction</a>). The Makefile created by the standard UNIX -configuration will build the ex_rep examples on most platforms. Enter -"make ex_rep_mgr" and/or "make ex_rep_base" to build them.</p> - <p>The synopsis for both programs is as follows:</p> - <code class="literal">ex_rep_xxx <span class="bold"><strong>-h home</strong></span> <span class="bold"><strong>-l host:port</strong></span> [<span class="bold"><strong>-MC</strong></span>] [<span class="bold"><strong>-r host:port</strong></span>] [<span class="bold"><strong>-R host:port</strong></span>] [<span class="bold"><strong>-a all|quorum</strong></span>] [<span class="bold"><strong>-b</strong></span>] [<span class="bold"><strong>-n sites</strong></span>] [<span class="bold"><strong>-p priority</strong></span>] [<span class="bold"><strong>-v</strong></span>]</code> <p> - where "ex_rep_xxx" is either "ex_rep_mgr" or "ex_rep_base". The only - difference is that: -</p> + Ex_rep, found in the <code class="filename">examples/c/ex_rep</code> + subdirectory of the Berkeley DB distribution, is a simple but + complete demonstration of a replicated application. The + application is a mock stock ticker. The master accepts a stock + symbol and a numerical value as input, and stores this + information into a replicated database; either master or + clients can display the contents of the database, given an + empty input line. + </p> + <p> + There are two versions of the application: ex_rep_mgr uses + Replication Manager, while ex_rep_base uses the replication + Base API. This is intended to demonstrate that, while the + basic function of the application is the same in either case, + the replication support infrastructure differs + markedly. + </p> + <p> + The communication infrastructure demonstrated with + ex_rep_base has the same dependencies on system networking and + threading support as does the Replication Manager (see the + <a class="xref" href="rep.html#rep_intro" title="Replication introduction">Replication introduction</a>). + The Makefile created by the standard UNIX configuration will + build the ex_rep examples on most platforms. Enter "make + ex_rep_mgr" and/or "make ex_rep_base" to build them. + </p> + <p> + The synopsis for both programs is as follows: + </p> + <code class="literal">ex_rep_xxx <span class="bold"><strong>-h home</strong></span> + <span class="bold"><strong>-l host:port</strong></span> [<span class="bold"><strong>-MC</strong></span>] [<span class="bold"><strong>-r + host:port</strong></span>] [<span class="bold"><strong>-R + host:port</strong></span>] [<span class="bold"><strong>-a + all|quorum</strong></span>] [<span class="bold"><strong>-b</strong></span>] + [<span class="bold"><strong>-n + sites</strong></span>] [<span class="bold"><strong>-p + priority</strong></span>] [<span class="bold"><strong>-v</strong></span>]</code> + <p> + where "ex_rep_xxx" is either "ex_rep_mgr" or "ex_rep_base". + The only difference is that: + </p> <div class="itemizedlist"> <ul type="disc"> <li> <p> - specifying <span class="bold"><strong>-M</strong></span> or - <span class="bold"><strong>-C</strong></span> is optional for ex_rep_mgr, - but one of these options must be specified for ex_rep_base. - </p> + specifying <span class="bold"><strong>-M</strong></span> or + <span class="bold"><strong>-C</strong></span> is optional + for ex_rep_mgr, but one of these options must be + specified for ex_rep_base. + </p> </li> <li> - <p> - The <span class="bold"><strong>-n</strong></span> option is not supported - supported by ex_rep_mgr. That option specifies the number of - nodes in the replication group. When you use the Replication - Manager, this number is automatically determined for you. - </p> + <p> + The <span class="bold"><strong>-n</strong></span> option is + not supported supported by ex_rep_mgr. That option + specifies the number of nodes in the replication + group. When you use the Replication Manager, this + number is automatically determined for you. + </p> </li> </ul> </div> - <p> - The options apply to either version of the program except where noted. - They are as follows: -</p> + <p> + The options apply to either version of the program except + where noted. They are as follows: + </p> <div class="variablelist"> <dl> <dt> <span class="term"> - <span class="bold"> - <strong>-h</strong> - </span> + <span class="bold"><strong>-h</strong></span> </span> </dt> - <dd>Specify a home directory for the database environment.</dd> + <dd> + Specify a home directory for the database + environment. + </dd> <dt> <span class="term"> - <span class="bold"> - <strong>-l</strong> - </span> + <span class="bold"><strong>-l</strong></span> </span> </dt> - <dd>Listen on local host "host" at port "port" for incoming connections.</dd> + <dd> + Listen on local host "host" at port "port" for + incoming connections. + </dd> <dt> <span class="term"> - <span class="bold"> - <strong>-M</strong> - </span> + <span class="bold"><strong>-M</strong></span> </span> </dt> - <dd>Configure this process as a master.</dd> + <dd> + Configure this process as a master. + </dd> <dt> <span class="term"> - <span class="bold"> - <strong>-C</strong> - </span> + <span class="bold"><strong>-C</strong></span> </span> </dt> - <dd>Configure this process as a client.</dd> + <dd> + Configure this process as a client. + </dd> <dt> <span class="term"> - <span class="bold"> - <strong>-r</strong> - </span> + <span class="bold"><strong>-r</strong></span> </span> </dt> <dd> - Identifies the helper site used for joining the group. - </dd> + Identifies the helper site used for joining the + group. + </dd> <dt> <span class="term"> - <span class="bold"> - <strong>-R</strong> - </span> + <span class="bold"><strong>-R</strong></span> </span> </dt> - <dd> - Identifies a remote peer to be used for joining the group. This - peer is used for syncing purposes. See - <a class="xref" href="rep_mastersync.html#rep_c2c_sync" title="Client-to-client synchronization">Client-to-client synchronization</a> - for more information. - </dd> + <dd> + Identifies a remote peer to be used for joining + the group. This peer is used for syncing purposes. See + <a class="xref" href="rep_mastersync.html#rep_c2c_sync" title="Client-to-client synchronization">Client-to-client synchronization</a> for more + information. + </dd> <dt> <span class="term"> - <span class="bold"> - <strong>-a</strong> - </span> + <span class="bold"><strong>-a</strong></span> </span> </dt> <dd> - Specify repmgr acknowledgement policy of all or quorum. See - <a href="../api_reference/C/repmgrset_ack_policy.html" class="olink">DB_ENV->repmgr_set_ack_policy()</a> for more information (ex_rep_mgr - only.) - </dd> + Specify repmgr acknowledgement policy of all or + quorum. See <a href="../api_reference/C/repmgrset_ack_policy.html" class="olink">DB_ENV->repmgr_set_ack_policy()</a> for more + information (ex_rep_mgr only.) + </dd> <dt> <span class="term"> - <span class="bold"> - <strong>-b</strong> - </span> + <span class="bold"><strong>-b</strong></span> </span> </dt> <dd> - Indicates that bulk transfer should be used. See <a class="xref" href="rep_bulk.html" title="Bulk transfer">Bulk transfer</a> for more - information. - </dd> + Indicates that bulk transfer should be used. + See <a class="xref" href="rep_bulk.html" title="Bulk transfer">Bulk transfer</a> for more information. + </dd> <dt> <span class="term"> - <span class="bold"> - <strong>-n</strong> - </span> + <span class="bold"><strong>-n</strong></span> </span> </dt> - <dd> - Specify the total number of sites in the replication group - (ex_rep_base only). - </dd> + <dd> + Specify the total number of sites in the + replication group (ex_rep_base only). + </dd> <dt> <span class="term"> - <span class="bold"> - <strong>-p</strong> - </span> + <span class="bold"><strong>-p</strong></span> </span> </dt> - <dd>Set the - election priority. See - <a class="xref" href="rep_elect.html" title="Elections">Elections</a> for more - information. - </dd> + <dd> + Set the election priority. See <a class="xref" href="rep_elect.html" title="Elections">Elections</a> + for more information. + </dd> <dt> <span class="term"> - <span class="bold"> - <strong>-v</strong> - </span> + <span class="bold"><strong>-v</strong></span> </span> </dt> - <dd>Indicates that additional informational and debugging output should be enabled.</dd> + <dd> + Indicates that additional informational and + debugging output should be enabled. + </dd> </dl> </div> - <p>A typical ex_rep_mgr session begins with a command such as the -following, to start a master:</p> + <p> + A typical ex_rep_mgr session begins with a command such as + the following, to start a master: + </p> <pre class="programlisting">ex_rep_mgr -M -p 100 -h DIR1 -l localhost:30100</pre> - <p>and several clients:</p> + <p> + and several clients: + </p> <pre class="programlisting">ex_rep_mgr -C -p 50 -h DIR2 -l localhost:30101 -r localhost:30100 ex_rep_mgr -C -p 10 -h DIR3 -l localhost:30102 -r localhost:30100 ex_rep_mgr -C -p 0 -h DIR4 -l localhost:30103 -r localhost:30100</pre> <p> - In this example, the client with home directory DIR4 can never become a - master (its priority is 0). Both of the other clients can become - masters, but the one with home directory DIR2 is preferred. Priorities - are assigned by the application and should reflect the desirability of - having particular clients take over as master in the case that the - master fails. -</p> + In this example, the client with home directory DIR4 can + never become a master (its priority is 0). Both of the other + clients can become masters, but the one with home directory + DIR2 is preferred. Priorities are assigned by the application + and should reflect the desirability of having particular + clients take over as master in the case that the master fails. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/rep_ex_chan.html b/docs/programmer_reference/rep_ex_chan.html index f1736033..066d1dc2 100644 --- a/docs/programmer_reference/rep_ex_chan.html +++ b/docs/programmer_reference/rep_ex_chan.html @@ -14,18 +14,15 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Ex_rep_chan: a Replication Manager -channel example</th> + <th colspan="3" align="center">Ex_rep_chan: a Replication Manager channel example</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_ex_rq.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="xa.html">Next</a></td> </tr> </table> @@ -35,156 +32,157 @@ channel example</th> <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="rep_ex_chan"></a>Ex_rep_chan: a Replication Manager -channel example</h2> + <h2 class="title" style="clear: both"><a id="rep_ex_chan"></a>Ex_rep_chan: a Replication Manager channel example</h2> </div> </div> </div> <p> - Ex_rep_chan, found in the <code class="filename">examples/c/ex_rep_chan</code> - subdirectory of the Berkeley DB distribution, is a simple but complete - demonstration of a replicated application that uses the Replication - Manager feature of channels to perform write forwarding. The - application is a mock stock ticker. Although similar to the ex_rep_mgr - example program, this example differs in that it provides an example of - using Replication Manager message channels. Any site accepts a command - to write data to the database. If the site is a client, then, using - the channels feature, the application forwards the request to the - master site. If the site is a master then the request is automatically - handled locally. You can read and write stock values at any site - without needing to know what site is currently the master. -</p> + Ex_rep_chan, found in the + <code class="filename">examples/c/ex_rep_chan</code> subdirectory + of the Berkeley DB distribution, is a simple but complete + demonstration of a replicated application that uses the + Replication Manager feature of channels to perform write + forwarding. The application is a mock stock ticker. Although + similar to the ex_rep_mgr example program, this example + differs in that it provides an example of using Replication + Manager message channels. Any site accepts a command to write + data to the database. If the site is a client, then, using the + channels feature, the application forwards the request to the + master site. If the site is a master then the request is + automatically handled locally. You can read and write stock + values at any site without needing to know what site is + currently the master. + </p> <p> - The set of supported commands can be viewed with either the - <span class="bold"><strong>help</strong></span> or the - <span class="bold"><strong>?</strong></span> command. Several commands work with - key/data pairs where the key is a stock symbol and the data is its - value. -</p> + The set of supported commands can be viewed with either the + <span class="bold"><strong>help</strong></span> or the <span class="bold"><strong>?</strong></span> command. Several commands work + with key/data pairs where the key is a stock symbol and the + data is its value. + </p> <p> - The command to retrieve and print the current site's database contents - is <span class="bold"><strong>print</strong></span> or simply an empty input - line. To read the contents of the master's database from any site use - the <span class="bold"><strong>get key key ...</strong></span> command. That - command will forward the read request to the master if necessary and - return the key/data pairs for all given keys. -</p> + The command to retrieve and print the current site's + database contents is <span class="bold"><strong>print</strong></span> or + simply an empty input line. To read the contents of the + master's database from any site use the <span class="bold"><strong>get key key ...</strong></span> + command. That command will + forward the read request to the master if necessary and return + the key/data pairs for all given keys. + </p> + <p> + There are two commands to put data into the database. Both + commands take one or more key/data pairs, all of which are + written into the database in a single transaction at the + master site. The <span class="bold"><strong>put</strong></span> command + sends the data to the master site, and simply waits for a + status response. The <span class="bold"><strong>put_sync</strong></span> + command sends the data to the master site, and uses a + transaction token returned by the master to wait for the + contents of that put to be available on the local site. This + serves as a demonstration of the <a class="link" href="rep_ryw.html" title="Read your writes consistency">read + your writes</a> consistency feature. + </p> + <p> + The Makefile created by the standard UNIX configuration + will build the ex_rep_chan example on most platforms. Enter + "make ex_rep_chan" to build it. + </p> <p> - There are two commands to put data into the database. Both commands - take one or more key/data pairs, all of which are written into the - database in a single transaction at the master site. The - <span class="bold"><strong>put</strong></span> command sends the data to the - master site, and simply waits for a status response. The - <span class="bold"><strong>put_sync</strong></span> command sends the data to the - master site, and uses a transaction token returned by the master to - wait for the contents of that put to be available on the local - site. This serves as a demonstration of the - <a class="link" href="rep_ryw.html" title="Read your writes consistency">read your writes</a> consistency feature. -</p> + The synopsis for the program is as follows: + </p> <p> - The Makefile created by the standard UNIX configuration will build the - ex_rep_chan example on most platforms. Enter "make ex_rep_chan" to - build it. -</p> + <code class="literal">ex_rep_chan <span class="bold"><strong>-h home</strong></span> + <span class="bold"><strong>-l host:port</strong></span> [<span class="bold"><strong>-MC</strong></span>] [<span class="bold"><strong>-r + host:port</strong></span>] [<span class="bold"><strong>-R + host:port</strong></span>] [<span class="bold"><strong>-p + priority</strong></span>] [<span class="bold"><strong>-v</strong></span>]</code> + </p> <p> - The synopsis for the program is as follows: -</p> - <p> -<code class="literal">ex_rep_chan <span class="bold"><strong>-h home</strong></span> <span class="bold"><strong>-l host:port</strong></span> [<span class="bold"><strong>-MC</strong></span>] [<span class="bold"><strong>-r host:port</strong></span>] [<span class="bold"><strong>-R host:port</strong></span>] [<span class="bold"><strong>-p priority</strong></span>] [<span class="bold"><strong>-v</strong></span>]</code> -</p> - <p> - The options are as follows: -</p> + The options are as follows: + </p> <div class="variablelist"> <dl> <dt> <span class="term"> - <span class="bold"> - <strong>-h</strong> - </span> + <span class="bold"><strong>-h</strong></span> </span> </dt> - <dd>Specify a home directory for the database environment.</dd> + <dd> + Specify a home directory for the database + environment. + </dd> <dt> <span class="term"> - <span class="bold"> - <strong>-l</strong> - </span> + <span class="bold"><strong>-l</strong></span> </span> </dt> - <dd>Listen on local host "host" at port "port" for incoming connections.</dd> + <dd> + Listen on local host "host" at port "port" for + incoming connections. + </dd> <dt> <span class="term"> - <span class="bold"> - <strong>-M</strong> - </span> + <span class="bold"><strong>-M</strong></span> </span> </dt> - <dd>Configure this process as a master.</dd> + <dd> + Configure this process as a master. + </dd> <dt> <span class="term"> - <span class="bold"> - <strong>-C</strong> - </span> + <span class="bold"><strong>-C</strong></span> </span> </dt> - <dd>Configure this process as a client.</dd> + <dd> + Configure this process as a client. + </dd> <dt> <span class="term"> - <span class="bold"> - <strong>-r</strong> - </span> + <span class="bold"><strong>-r</strong></span> </span> </dt> <dd> - Identifies the helper site used for joining the group. - </dd> + Identifies the helper site used for joining the + group. + </dd> <dt> <span class="term"> - <span class="bold"> - <strong>-R</strong> - </span> + <span class="bold"><strong>-R</strong></span> </span> </dt> - <dd> - Identifies a remote peer to be used for joining the group. This - peer is used for syncing purposes. See - <a class="xref" href="rep_mastersync.html#rep_c2c_sync" title="Client-to-client synchronization">Client-to-client synchronization</a> - for more information. - </dd> + <dd> + Identifies a remote peer to be used for joining + the group. This peer is used for syncing purposes. See + <a class="xref" href="rep_mastersync.html#rep_c2c_sync" title="Client-to-client synchronization">Client-to-client synchronization</a> for more + information. + </dd> <dt> <span class="term"> - <span class="bold"> - <strong>-p</strong> - </span> + <span class="bold"><strong>-p</strong></span> </span> </dt> - <dd> - Set the election priority. See - <a class="xref" href="rep_elect.html" title="Elections">Elections</a> for more - information. - </dd> + <dd> + Set the election priority. See <a class="xref" href="rep_elect.html" title="Elections">Elections</a> + for more information. + </dd> <dt> <span class="term"> - <span class="bold"> - <strong>-v</strong> - </span> + <span class="bold"><strong>-v</strong></span> </span> </dt> <dd> - Indicates that additional informational and debugging output - should be enabled. - </dd> + Indicates that additional informational and + debugging output should be enabled. + </dd> </dl> </div> - <p> - A typical ex_rep_chan session begins with a command such as the - following, to start a master: -</p> + <p> + A typical ex_rep_chan session begins with a command such as + the following, to start a master: + </p> <pre class="programlisting">ex_rep_chan -M -h DIR1 -l localhost:30100</pre> <p> - and several clients: -</p> + and several clients: + </p> <pre class="programlisting">ex_rep_chan -C -h DIR2 -l localhost:30101 -r localhost:30100 ex_rep_chan -C -h DIR3 -l localhost:30102 -r localhost:30100 ex_rep_chan -C -h DIR4 -l localhost:30103 -r localhost:30100</pre> @@ -200,13 +198,12 @@ ex_rep_chan -C -h DIR4 -l localhost:30103 -r localhost:30100</pre> <td width="40%" align="right"> <a accesskey="n" href="xa.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Ex_rep_base: putting it all together </td> + <td width="40%" align="left" valign="top">Ex_rep_base: putting it all + together </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 13. - Distributed Transactions - </td> + <td width="40%" align="right" valign="top"> Chapter 13. Distributed Transactions </td> </tr> </table> </div> diff --git a/docs/programmer_reference/rep_ex_comm.html b/docs/programmer_reference/rep_ex_comm.html index ac219a2f..3be89b65 100644 --- a/docs/programmer_reference/rep_ex_comm.html +++ b/docs/programmer_reference/rep_ex_comm.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_ex.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_ex_rq.html">Next</a></td> </tr> </table> @@ -39,74 +37,99 @@ </div> </div> <p> -Base API applications must implement a -communication infrastructure. The communication infrastructure -consists of three parts: a way to map environment IDs to particular -sites, the functions to send and receive messages, and the application -architecture that supports the particular communication infrastructure -used (for example, individual threads per communicating site, a shared -message handler for all sites, a hybrid solution). The communication -infrastructure for ex_rep_base is implemented in the file -<code class="filename">ex_rep/base/rep_net.c</code>, and each part of that infrastructure -is described as follows.</p> - <p>Ex_rep_base maintains a table of environment ID to TCP/IP port -mappings. A pointer to this table is stored in a structure pointed to -by the app_private field of the <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> object so it can be -accessed by any function that has the database environment handle. -The table is represented by a machtab_t structure which contains a -reference to a linked list of member_t's, both of which are defined in -<code class="filename">ex_rep/base/rep_net.c</code>. Each member_t contains the host and -port identification, the environment ID, and a file descriptor.</p> - <p>This design is particular to this application and communication -infrastructure, but provides an indication of the sort of functionality -that is needed to maintain the application-specific state for a -TCP/IP-based infrastructure. The goal of the table and its interfaces -is threefold: First, it must guarantee that given an environment ID, -the send function can send a message to the appropriate place. Second, -when given the special environment ID <a href="../api_reference/C/reptransport.html#transport_DB_EID_BROADCAST" class="olink">DB_EID_BROADCAST</a>, the send -function can send messages to all the machines in the group. Third, -upon receipt of an incoming message, the receive function can correctly -identify the sender and pass the appropriate environment ID to the -<a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method.</p> - <p>Mapping a particular environment ID to a specific port is accomplished -by looping through the linked list until the desired environment ID is -found. Broadcast communication is implemented by looping through the -linked list and sending to each member found. Since each port -communicates with only a single other environment, receipt of a message -on a particular port precisely identifies the sender.</p> - <p>This is implemented in the quote_send, quote_send_broadcast and -quote_send_one functions, which can be found in -<code class="filename">ex_rep/base/rep_net.c</code>.</p> - <p>The example provided is merely one way to satisfy these requirements, -and there are alternative implementations as well. For instance, -instead of associating separate socket connections with each remote -environment, an application might instead label each message with a -sender identifier; instead of looping through a table and sending a -copy of a message to each member of the replication group, the -application could send a single message using a broadcast protocol.</p> - <p>The quote_send function is passed as the callback to -<a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a>; Berkeley DB automatically sends messages as needed -for replication. The receive function is a mirror to the quote_send_one -function. It is not a callback function (the application is responsible -for collecting messages and calling <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> on them as is -convenient). In the sample application, all messages transmitted are -Berkeley DB messages that get handled by <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a>, however, this -is not always going to be the case. The application may want to pass -its own messages across the same channels, distinguish between its own -messages and those of Berkeley DB, and then pass only the Berkeley DB ones to -<a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a>.</p> - <p>The final component of the communication infrastructure is the process -model used to communicate with all the sites in the replication group. -Each site creates a thread of control that listens on its designated -socket (as specified by the <span class="bold"><strong>-l</strong></span> command line argument) and -then creates a new channel for each site that contacts it. In addition, -each site explicitly connects to the sites specified in the -<span class="bold"><strong>-r</strong></span> and <span class="bold"><strong>-R</strong></span> -command line arguments. This is a fairly standard TCP/IP -process architecture and is implemented by the connect_thread, -connect_all and connect_site functions -in <code class="filename">ex_rep/base/rep_msg.c</code> and supporting functions -in <code class="filename">ex_rep/base/rep_net.c</code>.</p> + Base API applications must implement a communication + infrastructure. The communication infrastructure consists of + three parts: a way to map environment IDs to particular sites, + the functions to send and receive messages, and the + application architecture that supports the particular + communication infrastructure used (for example, individual + threads per communicating site, a shared message handler for + all sites, a hybrid solution). The communication + infrastructure for ex_rep_base is implemented in the file + <code class="filename">ex_rep/base/rep_net.c</code>, and each part + of that infrastructure is described as follows. + </p> + <p> + Ex_rep_base maintains a table of environment ID to TCP/IP + port mappings. A pointer to this table is stored in a + structure pointed to by the app_private field of the <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> + object so it can be accessed by any function that has the + database environment handle. The table is represented by a + machtab_t structure which contains a reference to a linked + list of member_t's, both of which are defined in + <code class="filename">ex_rep/base/rep_net.c</code>. Each member_t + contains the host and port identification, the environment ID, + and a file descriptor. + </p> + <p> + This design is particular to this application and + communication infrastructure, but provides an indication of + the sort of functionality that is needed to maintain the + application-specific state for a TCP/IP-based infrastructure. + The goal of the table and its interfaces is threefold: First, + it must guarantee that given an environment ID, the send + function can send a message to the appropriate place. Second, + when given the special environment ID <a href="../api_reference/C/reptransport.html#transport_DB_EID_BROADCAST" class="olink">DB_EID_BROADCAST</a>, the + send function can send messages to all the machines in the + group. Third, upon receipt of an incoming message, the receive + function can correctly identify the sender and pass the + appropriate environment ID to the <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method. + </p> + <p> + Mapping a particular environment ID to a specific port is + accomplished by looping through the linked list until the + desired environment ID is found. Broadcast communication is + implemented by looping through the linked list and sending to + each member found. Since each port communicates with only a + single other environment, receipt of a message on a particular + port precisely identifies the sender. + </p> + <p> + This is implemented in the quote_send, quote_send_broadcast + and quote_send_one functions, which can be found in + <code class="filename">ex_rep/base/rep_net.c</code>. + </p> + <p> + The example provided is merely one way to satisfy these + requirements, and there are alternative implementations as + well. For instance, instead of associating separate socket + connections with each remote environment, an application might + instead label each message with a sender identifier; instead + of looping through a table and sending a copy of a message to + each member of the replication group, the application could + send a single message using a broadcast protocol. + </p> + <p> + The quote_send function is passed as the callback to + <a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a>; Berkeley DB automatically sends messages as + needed for replication. The receive function is a mirror to + the quote_send_one function. It is not a callback function + (the application is responsible for collecting messages and + calling <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> on them as is convenient). In the sample + application, all messages transmitted are Berkeley DB messages + that get handled by <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a>, however, this is not always + going to be the case. The application may want to pass its own + messages across the same channels, distinguish between its own + messages and those of Berkeley DB, and then pass only the + Berkeley DB ones to <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a>. + </p> + <p> + The final component of the communication infrastructure is + the process model used to communicate with all the sites in + the replication group. Each site creates a thread of control + that listens on its designated socket (as specified by the + <span class="bold"><strong>-l</strong></span> command line argument) + and then creates a new channel for each site that contacts it. + In addition, each site explicitly connects to the sites + specified in the <span class="bold"><strong>-r</strong></span> and + <span class="bold"><strong>-R</strong></span> command line + arguments. This is a fairly standard TCP/IP process + architecture and is implemented by the connect_thread, + connect_all and connect_site functions in + <code class="filename">ex_rep/base/rep_msg.c</code> and supporting + functions in + <code class="filename">ex_rep/base/rep_net.c</code>. + </p> </div> <div class="navfooter"> <hr /> @@ -123,7 +146,8 @@ in <code class="filename">ex_rep/base/rep_net.c</code>.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Ex_rep_base: putting it all together</td> + <td width="40%" align="right" valign="top"> Ex_rep_base: putting it all + together</td> </tr> </table> </div> diff --git a/docs/programmer_reference/rep_ex_rq.html b/docs/programmer_reference/rep_ex_rq.html index 16fc9cb5..e59f30a7 100644 --- a/docs/programmer_reference/rep_ex_rq.html +++ b/docs/programmer_reference/rep_ex_rq.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Ex_rep_base: putting it all together</th> + <th colspan="3" align="center">Ex_rep_base: putting it all + together</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_ex_comm.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_ex_chan.html">Next</a></td> </tr> </table> @@ -34,51 +33,69 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="rep_ex_rq"></a>Ex_rep_base: putting it all together</h2> + <h2 class="title" style="clear: both"><a id="rep_ex_rq"></a>Ex_rep_base: putting it all + together</h2> </div> </div> </div> - <p>Beyond simply initializing a replicated environment, a Base API application -must set up its communication -infrastructure, and then make sure that incoming messages are received -and processed.</p> - <p>To initialize replication, ex_rep_base creates a Berkeley DB environment and -calls <a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a> to establish a send function. (See -the main function in <code class="filename">ex_rep/base/rep_base.c</code>, including its -calls to the create_env and env_init functions in -<code class="filename">ex_rep/common/rep_common.c</code>.)</p> - <p>ex_rep_base opens a listening socket for incoming connections and opens -an outgoing connection to every machine that it knows about (that is, -all the sites listed in the <span class="bold"><strong>-r</strong></span> and -<span class="bold"><strong>-R</strong></span> command line arguments). -Applications can structure the details of this in different ways, but -ex_rep_base creates a user-level thread to listen on its socket, plus -a thread to loop and handle messages on each socket, in addition to the -threads needed to manage the user interface, update the database on the -master, and read from the database on the client (in other words, in -addition to the normal functionality of any database application).</p> - <p>Once the initial threads have all been started and the communications -infrastructure is initialized, the application signals that it is ready -for replication and joins a replication group by calling -<a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a>. (Again, see the main function in -<code class="filename">ex_rep/base/rep_base.c</code>.)</p> - <p>Note the use of the optional second argument to <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> in -the client initialization code. The argument "local" is a piece of -data, opaque to Berkeley DB, that will be broadcast to each member of a -replication group; it allows new clients to join a replication group, -without knowing the location of all its members; the new client will -be contacted by the members it does not know about, who will receive -the new client's contact information that was specified in "myaddr." -See <a class="xref" href="rep_newsite.html" title="Connecting to a new site">Connecting to a new site</a> for more -information.</p> - <p>The final piece of a replicated application is the code that loops, -receives, and processes messages from a given remote environment. -ex_rep_base runs one of these loops in a parallel thread for each -socket connection (see the hm_loop function in -<code class="filename">ex_rep/base/rep_msg.c</code>). Other applications may want to queue -messages somehow and process them asynchronously, or select() on a -number of sockets and either look up the correct environment ID for each -or encapsulate the ID in the communications protocol.</p> + <p> + Beyond simply initializing a replicated environment, a Base + API application must set up its communication infrastructure, + and then make sure that incoming messages are received and + processed. + </p> + <p> + To initialize replication, ex_rep_base creates a Berkeley DB + environment and calls <a href="../api_reference/C/reptransport.html" class="olink">DB_ENV->rep_set_transport()</a> to establish a send + function. (See the main function in + <code class="filename">ex_rep/base/rep_base.c</code>, including its + calls to the create_env and env_init functions in + <code class="filename">ex_rep/common/rep_common.c</code>.) + </p> + <p> + ex_rep_base opens a listening socket for incoming + connections and opens an outgoing connection to every machine + that it knows about (that is, all the sites listed in the + <span class="bold"><strong>-r</strong></span> and <span class="bold"><strong>-R</strong></span> command line arguments). + Applications can structure the details of this in different + ways, but ex_rep_base creates a user-level thread to listen on + its socket, plus a thread to loop and handle messages on each + socket, in addition to the threads needed to manage the user + interface, update the database on the master, and read from + the database on the client (in other words, in addition to the + normal functionality of any database application). + </p> + <p> + Once the initial threads have all been started and the + communications infrastructure is initialized, the application + signals that it is ready for replication and joins a + replication group by calling <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a>. (Again, see the main + function in + <code class="filename">ex_rep/base/rep_base.c</code>.) + </p> + <p> + Note the use of the optional second argument to <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> + in the client initialization code. The argument "local" is a + piece of data, opaque to Berkeley DB, that will be broadcast + to each member of a replication group; it allows new clients + to join a replication group, without knowing the location of + all its members; the new client will be contacted by the + members it does not know about, who will receive the new + client's contact information that was specified in "myaddr." + See <a class="xref" href="rep_newsite.html" title="Connecting to a new site">Connecting to a new site</a> + for more information. + </p> + <p> + The final piece of a replicated application is the code that + loops, receives, and processes messages from a given remote + environment. ex_rep_base runs one of these loops in a parallel + thread for each socket connection (see the hm_loop function in + <code class="filename">ex_rep/base/rep_msg.c</code>). Other + applications may want to queue messages somehow and process + them asynchronously, or select() on a number of sockets and + either look up the correct environment ID for each or + encapsulate the ID in the communications protocol. + </p> </div> <div class="navfooter"> <hr /> @@ -95,8 +112,7 @@ or encapsulate the ID in the communications protocol.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Ex_rep_chan: a Replication Manager -channel example</td> + <td width="40%" align="right" valign="top"> Ex_rep_chan: a Replication Manager channel example</td> </tr> </table> </div> diff --git a/docs/programmer_reference/rep_faq.html b/docs/programmer_reference/rep_faq.html index 480a352b..898b0621 100644 --- a/docs/programmer_reference/rep_faq.html +++ b/docs/programmer_reference/rep_faq.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_partition.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_ex.html">Next</a></td> </tr> </table> @@ -42,136 +40,164 @@ <ol type="1"> <li> <p> - <span class="bold"><strong>Does Berkeley DB provide support for - forwarding write queries from clients to - masters?</strong></span> - </p> - <p> - No, it does not. - - - - In general this protocol is left entirely to the application. - Note, there is no reason not to use the communications channels - a Base API application establishes for replication support to forward - database update messages to the master, since Berkeley DB does - not require those channels to be used exclusively for - replication messages. Replication Manager does not currently - offer this service to the application. - </p> + <span class="bold"><strong>Does Berkeley DB provide support + for forwarding write queries from clients to + masters?</strong></span> + </p> + <p> + No, it does not. In + general this protocol is left entirely to the + application. A Replication Manager application can use + message channels to implement this capability (see + <a class="xref" href="rep_ex_chan.html" title="Ex_rep_chan: a Replication Manager channel example">Ex_rep_chan: a Replication Manager channel example</a> for information + about a sample program that demonstrates this use of + message channels). For a Base API application, it is + possible to use the communications channels it + establishes for replication support to forward + database update messages to the master, since Berkeley + DB does not require those channels to be used + exclusively for replication messages. + </p> </li> <li> <p> - <span class="bold"><strong>Can I use replication to partition my - environment across multiple sites?</strong></span> - </p> - <p> - No, this is not possible. All replicated databases must be equally - shared by all environments in the replication group. - </p> + <span class="bold"><strong>Can I use replication to + partition my environment across multiple + sites?</strong></span> + </p> + <p> + Yes, you can create partial views to accomplish + this. See <a class="xref" href="rep_partview.html" title="Replication views">Replication views</a> for more + information. + </p> </li> <li> <p> - <span class="bold"><strong>I'm running with replication but I don't see my databases - on the client.</strong></span> - </p> - <p> - This problem may be the result of the application using absolute - path names for its databases, and the pathnames are not valid on - the client system. - </p> + <span class="bold"><strong>I'm running with replication but + I don't see my databases on the client.</strong></span> + </p> + <p> + This problem may be the result of the application + using absolute path names for its databases, and the + pathnames are not valid on the client system. + </p> </li> <li> <p> - <span class="bold"><strong>How can I distinguish Berkeley DB messages from application messages?</strong></span> - </p> + <span class="bold"><strong>How can I distinguish Berkeley + DB messages from application messages?</strong></span> + </p> <p> - There is no way to distinguish Berkeley DB messages from - application-specific messages, nor does Berkeley DB offer any way - to wrap application messages inside of Berkeley DB messages. - Distributed applications exchanging their own messages should - either enclose Berkeley DB messages in their own wrappers, or use - separate network connections to send and receive Berkeley DB - messages. The one exception to this rule is connection information - for new sites; Berkeley DB offers a simple method for sites joining - replication groups to send connection information to the other - database environments in the group (see <a class="xref" href="rep_newsite.html" title="Connecting to a new site">Connecting to a new site</a> for more information). - </p> + Replication Manager provides its own communications + infrastructure for replication messages. You can + create message channels to pass application-specific + messages using this infrastructure (see <a class="xref" href="repmgr_channels.html" title="Using Replication Manager message channels">Using Replication Manager message channels</a> for more + information). + </p> + <p> + In a Base API application, there is no way to + distinguish Berkeley DB messages from + application-specific messages, nor does Berkeley DB + offer any way to wrap application messages inside of + Berkeley DB messages. Distributed applications + exchanging their own messages should either enclose + Berkeley DB messages in their own wrappers, or use + separate network connections to send and receive + Berkeley DB messages. The one exception to this rule + is connection information for new sites; Berkeley DB + offers a simple method for sites joining replication + groups to send connection information to the other + database environments in the group (see <a class="xref" href="rep_newsite.html" title="Connecting to a new site">Connecting to a new site</a> for more information). + </p> </li> <li> <p> - <span class="bold"><strong>How should I build my <span class="bold"><strong>send</strong></span> function?</strong></span> - </p> - <p> - This depends on the specifics of the application. One common way - is to write the <span class="bold"><strong>rec</strong></span> and <span class="bold"><strong>control</strong></span> arguments' sizes and data to a - socket connected to each remote site. On a fast, local area net, - the simplest method is likely to be to construct broadcast - messages. Each Berkeley DB message would be encapsulated inside an - application specific message, with header information specifying - the intended recipient(s) for the message. This will likely - require a global numbering scheme, however, as the Berkeley DB - library has to be able to send specific log records to clients - apart from the general broadcast of new log records intended for - all members of a replication group. - </p> + <span class="bold"><strong>How should I build my <span class="bold"><strong>send</strong></span> + function?</strong></span> + </p> + <p> + This depends on the specifics of the application. + One common way is to write the <span class="bold"><strong>rec</strong></span> + and <span class="bold"><strong>control</strong></span> arguments' sizes and data to a + socket connected to each remote site. On a fast, local + area net, the simplest method is likely to be to + construct broadcast messages. Each Berkeley DB message + would be encapsulated inside an application specific + message, with header information specifying the + intended recipient(s) for the message. This will + likely require a global numbering scheme, however, as + the Berkeley DB library has to be able to send + specific log records to clients apart from the general + broadcast of new log records intended for all members + of a replication group. + </p> </li> <li> <p> - <span class="bold"><strong>Does every one of my threads of control on - the master have to set up its own connection to every client? - And, does every one of my threads of control on the client have - to set up its own connection to every master?</strong></span> - </p> - <p> - This is not always necessary. In the Berkeley DB replication - model, any thread of control which modifies a database in the - master environment must be prepared to send a message to the client - environments, and any thread of control which delivers a message to - a client environment must be prepared to send a message to the - master. There are many ways in which these requirements can be - satisfied. - </p> - <p> - The simplest case is probably a single, multithreaded process - running on the master and clients. The process running on the - master would require a single write connection to each client and a - single read connection from each client. A process running on each - client would require a single read connection from the master and a - single write connection to the master. Threads running in these - processes on the master and clients would use the same network - connections to pass messages back and forth. - </p> - <p> - A common complication is when there are multiple processes running - on the master and clients. A straight-forward solution is to - increase the numbers of connections on the master — each - process running on the master has its own write connection to each - client. However, this requires only one additional connection for - each possible client in the master process. The master environment - still requires only a single read connection from each client (this - can be done by allocating a separate thread of control which does - nothing other than receive client messages and forward them into - the database). Similarly, each client still only requires a single - thread of control that receives master messages and forwards them - into the database, and which also takes database messages and - forwards them back to the master. This model requires the - networking infrastructure support many-to-one writers-to-readers, - of course. - </p> + <span class="bold"><strong>Does every one of my threads of + control on the master have to set up its own + connection to every client? And, does every one of + my threads of control on the client have to set up + its own connection to every master?</strong></span> + </p> + <p> + This is not always necessary. In the Berkeley DB + replication model, any thread of control which + modifies a database in the master environment must be + prepared to send a message to the client environments, + and any thread of control which delivers a message to + a client environment must be prepared to send a + message to the master. There are many ways in which + these requirements can be satisfied. + </p> + <p> + The simplest case is probably a single, + multithreaded process running on the master and + clients. The process running on the master would + require a single write connection to each client and a + single read connection from each client. A process + running on each client would require a single read + connection from the master and a single write + connection to the master. Threads running in these + processes on the master and clients would use the same + network connections to pass messages back and forth. + </p> + <p> + A common complication is when there are multiple + processes running on the master and clients. A + straight-forward solution is to increase the numbers + of connections on the master — each process + running on the master has its own write connection to + each client. However, this requires only one + additional connection for each possible client in the + master process. The master environment still requires + only a single read connection from each client (this + can be done by allocating a separate thread of control + which does nothing other than receive client messages + and forward them into the database). Similarly, each + client still only requires a single thread of control + that receives master messages and forwards them into + the database, and which also takes database messages + and forwards them back to the master. This model + requires the networking infrastructure support + many-to-one writers-to-readers, of course. + </p> <p> - If the number of network connections is a problem in the - multiprocess model, and inter-process communication on the system - is inexpensive enough, an alternative is have a single process - which communicates between the master and each client, and whenever - a process' <span class="bold"><strong>send</strong></span> function is - called, the process passes the message to the communications - process which is responsible for forwarding the message to the - appropriate client. Alternatively, a broadcast mechanism will - simplify the entire networking infrastructure, as processes will - likely no longer have to maintain their own specific network - connections. - </p> + If the number of network connections is a problem + in the multiprocess model, and inter-process + communication on the system is inexpensive enough, an + alternative is have a single process which + communicates between the master and each client, and + whenever a process' <span class="bold"><strong>send</strong></span> + function is called, the process + passes the message to the communications process which + is responsible for forwarding the message to the + appropriate client. Alternatively, a broadcast + mechanism will simplify the entire networking + infrastructure, as processes will likely no longer + have to maintain their own specific network + connections. + </p> </li> </ol> </div> diff --git a/docs/programmer_reference/rep_filename.html b/docs/programmer_reference/rep_filename.html index b528ee6d..8b4e07ae 100644 --- a/docs/programmer_reference/rep_filename.html +++ b/docs/programmer_reference/rep_filename.html @@ -3,28 +3,26 @@ <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> - <title>Managing Replication Files</title> + <title>Managing replication directories and files</title> <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> <link rel="up" href="rep.html" title="Chapter 12. Berkeley DB Replication" /> - <link rel="prev" href="group_membership.html" title="Managing Replication Manager Group Membership" /> + <link rel="prev" href="rep_partview.html" title="Replication views" /> <link rel="next" href="rep_mgrmulti.html" title="Running Replication Manager in multiple processes" /> </head> <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Managing Replication Files</th> + <th colspan="3" align="center">Managing replication directories and files</th> </tr> <tr> - <td width="20%" align="left"><a accesskey="p" href="group_membership.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <td width="20%" align="left"><a accesskey="p" href="rep_partview.html">Prev</a> </td> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_mgrmulti.html">Next</a></td> </tr> </table> @@ -34,125 +32,199 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="rep_filename"></a>Managing Replication Files</h2> + <h2 class="title" style="clear: both"><a id="rep_filename"></a>Managing replication directories and files</h2> </div> </div> </div> - <p> - Whether you use the Base API or the Replication Manager, - replication creates a set of internal files that are normally stored - on-disk in your environment home directory. These files contain - metadata which is necessary for replication operations, and so you - should never delete these files. - </p> - <p> - You can cause these files to not be stored on disk, but instead to - be held entirely in-memory, by specifying the <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_INMEM" class="olink">DB_REP_CONF_INMEM</a> - flag to the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method. Doing this can improve your - application's data throughput by avoiding the disk I/O associated - with these metadata files. However, in the event that your - application is shut down, the contents of these files are lost. This - results in some loss of functionality, including an increased - chance that elections will fail, or that the wrong site will win an - election. See the <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_INMEM" class="olink">DB_REP_CONF_INMEM</a> flag description for more - information. - </p> - <p> - Note that turning on <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_INMEM" class="olink">DB_REP_CONF_INMEM</a> means that Replication - Manager cannot store group membership changes persistently. This is - because Replication Manager stores group membership information in - an internal database, which is held in memory when - <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_INMEM" class="olink">DB_REP_CONF_INMEM</a> is turned on. For this reason, if your - Replication Manager application requires replication metadata to be - stored in memory, then you must manually identify all the sites in - your replication group using the <code class="literal">DB_LEGACY</code> site - configuration attribute. Be aware that this configuration needs to - be made permanent. (Normally, <code class="literal">DB_LEGACY</code> is used - only on a temporary basis for the purpose of upgrading old - Replication Manager applications.) - </p> - <p> - Do the following: - </p> - <div class="orderedlist"> - <ol type="1"> - <li> - <p> - Shut down all the sites in your replication group. - </p> - </li> - <li> - <p> - For every site in your replication group: - </p> - <div class="orderedlist"> - <ol type="a"> - <li> - <p> - Configure a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle for the local site. - Use <a href="../api_reference/C/dbsite_set_config.html" class="olink">DB_SITE->set_config()</a> to indicate that this - is a legacy site by setting the - <code class="literal">DB_LEGACY</code> parameter. - </p> - </li> - <li> - <p> - Configure a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle for <span class="emphasis"><em>every - other site</em></span> in the replication - group. Set the <code class="literal">DB_LEGACY</code> - parameter for each of these handles. - </p> - <p> - Please pay careful attention to this step. To - repeat: a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle MUST be configured - for EVERY site in the replication group. - </p> - </li> - </ol> + <div class="toc"> + <dl> + <dt> + <span class="sect2"> + <a href="rep_filename.html#rep_dir">Replication database directory + considerations</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="rep_filename.html#rep_files">Managing replication internal + files</a> + </span> + </dt> + </dl> + </div> + <div class="sect2" lang="en" xml:lang="en"> + <div class="titlepage"> + <div> + <div> + <h3 class="title"><a id="rep_dir"></a>Replication database directory + considerations</h3> + </div> + </div> + </div> + <p> + If your application is going to locate databases in any + directory other than the environment home directory, you + need to consider the directory structure for all sites. + There are several recommendations to make in this area. + </p> + <p> + The use of absolute pathnames is strongly discouraged + when replication is in use. Absolute pathnames will not + work if there is more than one site on a single machine. + Replication with absolute pathnames is unlikely to work + across different machines unless great care is taken to + make sure the entire path is exactly the same on every + machine. + </p> + <p> + If the master uses a data directory, as specified via + <a href="../api_reference/C/envadd_data_dir.html" class="olink">DB_ENV->add_data_dir()</a> or <a href="../api_reference/C/envset_create_dir.html" class="olink">DB_ENV->set_create_dir()</a>, it is + recommended that you create the same directory structure + on all client sites. When the same directory structure + appears on a master and the client, replication creates + the client databases in the same directory as the master + regardless of the local client directory settings. If a + master directory is missing on a client, replication + decides where to create the client databases by using the + client's local directory settings and the Berkeley DB file + naming rules as described in <a class="xref" href="env_naming.html" title="File naming">File naming</a>. + </p> + </div> + <div class="sect2" lang="en" xml:lang="en"> + <div class="titlepage"> + <div> + <div> + <h3 class="title"><a id="rep_files"></a>Managing replication internal + files</h3> </div> - </li> - <li> - <p> - Restart all the sites in the replication group. - </p> - </li> - </ol> + </div> + </div> + <p> + Whether you use the Base API or the Replication + Manager, replication creates a set of internal files that + are normally stored on-disk in your environment home + directory. These files contain metadata which is necessary + for replication operations, and so you should never delete + these files. + </p> + <p> + You can cause these files to not be stored on disk, but + instead to be held entirely in-memory, by specifying the + <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_INMEM" class="olink">DB_REP_CONF_INMEM</a> flag to the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method. Doing + this can improve your application's data throughput by + avoiding the disk I/O associated with these metadata + files. However, in the event that your application is shut + down, the contents of these files are lost. This results + in some loss of functionality, including an increased + chance that elections will fail, or that the wrong site + will win an election. See the <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_INMEM" class="olink">DB_REP_CONF_INMEM</a> flag + description for more information. + </p> + <p> + Note that turning on <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_INMEM" class="olink">DB_REP_CONF_INMEM</a> means that + Replication Manager cannot store group membership changes + persistently. This is because Replication Manager stores + group membership information in an internal database, + which is held in memory when <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_INMEM" class="olink">DB_REP_CONF_INMEM</a> is turned + on. For this reason, if your Replication Manager + application requires replication metadata to be stored in + memory, then you must manually identify all the sites in + your replication group using the + <code class="literal">DB_LEGACY</code> site configuration + attribute. Be aware that this configuration needs to be + made permanent. (Normally, <code class="literal">DB_LEGACY</code> is + used only on a temporary basis for the purpose of + upgrading old Replication Manager applications.) + </p> + <p> Do the following: + </p> + <div class="orderedlist"> + <ol type="1"> + <li> + <p> + Shut down all the sites in your replication + group. + </p> + </li> + <li> + <p> + For every site in your replication group: + </p> + <div class="orderedlist"> + <ol type="a"> + <li> + <p> + Configure a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle for the + local site. Use <a href="../api_reference/C/dbsite_set_config.html" class="olink">DB_SITE->set_config()</a> to + indicate that this is a legacy site by + setting the <code class="literal">DB_LEGACY</code> + parameter. + </p> + </li> + <li> + <p> + Configure a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle for + <span class="emphasis"><em>every other site</em></span> + in the replication group. Set the + <code class="literal">DB_LEGACY</code> parameter + for each of these handles. + </p> + <p> + Please pay careful attention to this + step. To repeat: a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle MUST be + configured for EVERY site in the + replication group. + </p> + </li> + </ol> + </div> + </li> + <li> + <p> + Restart all the sites in the replication group. + </p> + </li> + </ol> + </div> + <p> + Alternatively, you can store persistent environment + metadata files, including those required by replication, + in a location other than your environment home directory. + This is necessary if your environment home directory is on + a device that is unstable, because the persistent metadata + files cannot be lost or deleted. You do this using the + <a href="../api_reference/C/envset_metadata_dir.html" class="olink">DB_ENV->set_metadata_dir()</a> method. + </p> + <p> + Note that you must configure the handling of your + environment metadata consistently across your entire + replication group. That is, if you place your replication + metadata in-memory on one site, then it must be placed + in-memory on all the sites in the group. Similarly, if you + place your replication metadata files in a non-standard + directory location on one site, then they must be placed + in the exact same directory location on all the sites in + your group. + </p> </div> - <p> - Alternatively, you can store persistent environment metadata files, - including those required by replication, in a location other than - your environment home directory. Doing so can help improve I/O - throughput by placing these files on a spindle that is not being - used for other environment data I/O. You do this using the - <a href="../api_reference/C/envset_metadata_dir.html" class="olink">DB_ENV->set_metadata_dir()</a> method. - </p> - <p> - Note that you must configure the handling of your environment - metadata consistently across your entire replication group. That - is, if you place your replication metadata in-memory on one - site, then it must be placed in-memory on all the sites in the - group. Similarly, if you place your replication metadata files in a - non-standard directory location on one site, then they must be - placed in the exact same directory location on all the sites in - your group. - </p> </div> <div class="navfooter"> <hr /> <table width="100%" summary="Navigation footer"> <tr> - <td width="40%" align="left"><a accesskey="p" href="group_membership.html">Prev</a> </td> + <td width="40%" align="left"><a accesskey="p" href="rep_partview.html">Prev</a> </td> <td width="20%" align="center"> <a accesskey="u" href="rep.html">Up</a> </td> <td width="40%" align="right"> <a accesskey="n" href="rep_mgrmulti.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Managing Replication Manager Group Membership </td> + <td width="40%" align="left" valign="top">Replication views </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Running Replication Manager in multiple processes</td> + <td width="40%" align="right" valign="top"> Running Replication Manager in + multiple processes</td> </tr> </table> </div> diff --git a/docs/programmer_reference/rep_id.html b/docs/programmer_reference/rep_id.html index cf9b0fdc..14824fe8 100644 --- a/docs/programmer_reference/rep_id.html +++ b/docs/programmer_reference/rep_id.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_pri.html">Next</a></td> </tr> </table> @@ -38,26 +36,37 @@ </div> </div> </div> - <p>Each database environment included in a replication group must have a -unique identifier for itself and for the other members of the -replication group. The identifiers do not need to be global, that is, -each database environment can assign local identifiers to members of -the replication group as it encounters them. For example, given three -sites: A, B and C, site A might assign the identifiers 1 and 2 to sites -B and C respectively, while site B might assign the identifiers 301 and -302 to sites A and C respectively. Note that it is not wrong to have -global identifiers, it is just not a requirement.</p> <p> - Replication Manager assigns and manages environment IDs on behalf of - the application. -</p> - <p>It is the responsibility of a Base API application to label each incoming -replication message passed to <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method with the appropriate -identifier. Subsequently, Berkeley DB will label outgoing messages to the -<span class="bold"><strong>send</strong></span> function with those same identifiers.</p> - <p>Negative identifiers are reserved for use by Berkeley DB, and should never be -assigned to environments by the application. Two of these reserved -identifiers are intended for application use, as follows:</p> + Each database environment included in a replication group + must have a unique identifier for itself and for the other + members of the replication group. The identifiers do not need + to be global, that is, each database environment can assign + local identifiers to members of the replication group as it + encounters them. For example, given three sites: A, B and C, + site A might assign the identifiers 1 and 2 to sites B and C + respectively, while site B might assign the identifiers 301 + and 302 to sites A and C respectively. Note that it is not + wrong to have global identifiers, it is just not a + requirement. + </p> + <p> + Replication Manager assigns and manages environment IDs on + behalf of the application. + </p> + <p> + It is the responsibility of a Base API application to label + each incoming replication message passed to <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> + method with the appropriate identifier. Subsequently, Berkeley + DB will label outgoing messages to the <span class="bold"><strong>send</strong></span> + function with those same + identifiers. + </p> + <p> + Negative identifiers are reserved for use by Berkeley DB, + and should never be assigned to environments by the + application. Two of these reserved identifiers are intended + for application use, as follows: + </p> <div class="variablelist"> <dl> <dt> @@ -66,10 +75,11 @@ identifiers are intended for application use, as follows:</p> </span> </dt> <dd> - <p> - The <a href="../api_reference/C/reptransport.html#transport_DB_EID_BROADCAST" class="olink">DB_EID_BROADCAST</a> identifier indicates a message should be - broadcast to all members of a replication group. - </p> + <p> + The <a href="../api_reference/C/reptransport.html#transport_DB_EID_BROADCAST" class="olink">DB_EID_BROADCAST</a> identifier indicates a + message should be broadcast to all members of a + replication group. + </p> </dd> <dt> <a id="rep_id.DB_EID_INVALID"></a> @@ -77,10 +87,11 @@ identifiers are intended for application use, as follows:</p> </dt> <dd> <p> - The DB_EID_INVALID identifier is an invalid environment ID, - and may be used to initialize environment ID variables that - are subsequently checked for validity. - </p> + The DB_EID_INVALID identifier is an invalid + environment ID, and may be used to initialize + environment ID variables that are subsequently + checked for validity. + </p> </dd> </dl> </div> @@ -96,9 +107,7 @@ identifiers are intended for application use, as follows:</p> <td width="40%" align="right"> <a accesskey="n" href="rep_pri.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 12. - Berkeley DB Replication - </td> + <td width="40%" align="left" valign="top">Chapter 12. Berkeley DB Replication </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/rep_init.html b/docs/programmer_reference/rep_init.html index 51865945..8678124f 100644 --- a/docs/programmer_reference/rep_init.html +++ b/docs/programmer_reference/rep_init.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_mastersync.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_bulk.html">Next</a></td> </tr> </table> @@ -38,47 +36,66 @@ </div> </div> </div> - <p>By default, adding a new site to a replication group only requires the -client to join. Berkeley DB will automatically perform internal -initialization from the master to the client, bringing the client into -sync with the master.</p> <p> - However, depending on the network and infrastructure, it can be - advantageous in a few instances to use a "hot backup" to initialize a - client into a replication group. Clients not wanting to automatically - perform internal initialization should call the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method to - turn off the <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_AUTOINIT" class="olink">DB_REP_CONF_AUTOINIT</a> flag. Turning off this - configuration flag causes Berkeley DB to - return <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_JOIN_FAILURE" class="olink">DB_REP_JOIN_FAILURE</a> to the application's <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method instead of - performing internal initialization. - -</p> - <p>To use a hot backup to initialize a client into a replication group, -perform the following steps:</p> + By default, adding a new site to a replication group only + requires the client to join. Berkeley DB will automatically + perform internal initialization from the master to the client, + bringing the client into sync with the master. + </p> + <p> + However, depending on the network and infrastructure, it + can be advantageous in a few instances to use a "hot backup" + to initialize a client into a replication group. Clients not + wanting to automatically perform internal initialization + should call the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method to turn off the + <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_AUTOINIT" class="olink">DB_REP_CONF_AUTOINIT</a> flag. Turning off this configuration + flag causes Berkeley DB to return <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_JOIN_FAILURE" class="olink">DB_REP_JOIN_FAILURE</a> to the + application's <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method instead of performing + internal initialization. + </p> + <p> + To use a hot backup to initialize a client into a + replication group, perform the following steps: + </p> <div class="orderedlist"> <ol type="1"> - <li>Do an archival backup of the master's environment, as described in -<a class="xref" href="transapp_archival.html" title="Database and log file archival">Database and log file archival</a>. The backup can either be a conventional backup or a hot -backup.</li> - <li>Copy the archival backup into a clean environment directory on the -client.</li> - <li>Run catastrophic recovery on the client's new environment, as described -in <a class="xref" href="transapp_recovery.html" title="Recovery procedures">Recovery procedures</a>.</li> - <li>Reconfigure and reopen the environment as a client member of the -replication group.</li> + <li> + Do an archival backup of the master's environment, + as described in <a class="xref" href="transapp_archival.html" title="Database and log file archival">Database and log file + archival</a>. The backup can + either be a conventional backup or a hot + backup. + </li> + <li> + Copy the archival backup into a clean environment + directory on the client. + </li> + <li> + Run catastrophic recovery on the client's new + environment, as described in <a class="xref" href="transapp_recovery.html" title="Recovery procedures">Recovery procedures</a>. + </li> + <li> + Reconfigure and reopen the environment as a client + member of the replication group. + </li> </ol> </div> - <p>If copying the backup to the client takes a long time relative to the -frequency with which log files are reclaimed using the -<a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility or the <a href="../api_reference/C/logarchive.html" class="olink">DB_ENV->log_archive()</a> method, it may be -necessary to suppress log reclamation until the newly restarted client -has "caught up" and applied all log records generated during its -downtime.</p> - <p>As with any Berkeley DB application, the database environment must be in a -consistent state at application startup. This is most easily assured -by running recovery at startup time in one thread or process; it is -harmless to do this on both clients and masters even when not strictly -necessary.</p> + <p> + If copying the backup to the client takes a long time + relative to the frequency with which log files are reclaimed + using the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility or the <a href="../api_reference/C/logarchive.html" class="olink">DB_ENV->log_archive()</a> method, it may be + necessary to suppress log reclamation until the newly + restarted client has "caught up" and applied all log records + generated during its downtime. + </p> + <p> + As with any Berkeley DB application, the database + environment must be in a consistent state at application + startup. This is most easily assured by running recovery at + startup time in one thread or process; it is harmless to do + this on both clients and masters even when not strictly + necessary. + </p> </div> <div class="navfooter"> <hr /> @@ -91,7 +108,8 @@ necessary.</p> <td width="40%" align="right"> <a accesskey="n" href="rep_bulk.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Synchronizing with a master </td> + <td width="40%" align="left" valign="top">Synchronizing with a + master </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/rep_lease.html b/docs/programmer_reference/rep_lease.html index 6db976d0..301e11af 100644 --- a/docs/programmer_reference/rep_lease.html +++ b/docs/programmer_reference/rep_lease.html @@ -3,7 +3,7 @@ <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> - <title>Master Leases</title> + <title>Master leases</title> <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> @@ -14,17 +14,15 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Master Leases</th> + <th colspan="3" align="center">Master leases</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_trans.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_ryw.html">Next</a></td> </tr> </table> @@ -34,7 +32,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="rep_lease"></a>Master Leases</h2> + <h2 class="title" style="clear: both"><a id="rep_lease"></a>Master leases</h2> </div> </div> </div> @@ -42,343 +40,388 @@ <dl> <dt> <span class="sect2"> - <a href="rep_lease.html#masterlease_change_groupsize">Changing Group Size</a> + <a href="rep_lease.html#masterlease_change_groupsize">Changing group + size</a> </span> </dt> </dl> </div> - <p> - Some applications have strict requirements about the consistency of - data read on a master site. Berkeley DB provides a mechanism called - master leases to provide such consistency. Without master leases, it - is sometimes possible for Berkeley DB to return old data to an - application when newer data is available due to unfortunate scheduling - as illustrated below: -</p> + <p> + Some applications have strict requirements about the + consistency of data read on a master site. Berkeley DB + provides a mechanism called master leases to provide such + consistency. Without master leases, it is sometimes possible + for Berkeley DB to return old data to an application when + newer data is available due to unfortunate scheduling as + illustrated below: + </p> <div class="orderedlist"> <ol type="1"> - <li><span class="bold"><strong>Application on master site</strong></span>: Read data item - <span class="emphasis"><em>foo</em></span> via Berkeley DB <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> call. - </li> - <li><span class="bold"><strong>Application on master site</strong></span>: sleep, get descheduled, etc. - </li> - <li><span class="bold"><strong>System</strong></span>: Master changes role, becomes a client. - </li> - <li><span class="bold"><strong>System</strong></span>: New site is elected master. - </li> - <li><span class="bold"><strong>System</strong></span>: New master modifies data item - <span class="emphasis"><em>foo</em></span>. - </li> - <li><span class="bold"><strong>Application</strong></span>: Berkeley DB returns old data for - <span class="emphasis"><em>foo</em></span> to application. - </li> + <li><span class="bold"><strong>Application on master + site</strong></span>: Read data item + <span class="emphasis"><em>foo</em></span> via Berkeley DB <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> or + <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> call. + </li> + <li><span class="bold"><strong>Application on master + site</strong></span>: sleep, get descheduled, etc. + </li> + <li><span class="bold"><strong>System</strong></span>: Master changes + role, becomes a client. + </li> + <li><span class="bold"><strong>System</strong></span>: New site is + elected master. + </li> + <li><span class="bold"><strong>System</strong></span>: New master + modifies data item <span class="emphasis"><em>foo</em></span>. + </li> + <li><span class="bold"><strong>Application</strong></span>: Berkeley DB + returns old data for <span class="emphasis"><em>foo</em></span> to + application. + </li> </ol> </div> - <p> - By using master leases, Berkeley DB can provide guarantees about the - consistency of data read on a master site. The master site can be - considered a recognized authority for the data and consequently can - provide authoritative reads. Clients grant master leases to a master - site. By doing so, clients acknowledge the right of that site to - retain the role of master for a period of time. During that period of - time, clients cannot elect a new master, become master, nor grant their - lease to another site. -</p> - <p> - By holding a collection of granted leases, a master site can guarantee - to the application that the data returned is the current, authoritative - value. As a master performs operations, it continually requests - updated grants from the clients. When a read operation is required, - the master guarantees that it holds a valid collection of lease grants - from clients before returning data to the application. By holding - leases, Berkeley DB provides several guarantees to the application: -</p> + <p> + By using master leases, Berkeley DB can provide guarantees + about the consistency of data read on a master site. The + master site can be considered a recognized authority for the + data and consequently can provide authoritative reads. Clients + grant master leases to a master site. By doing so, clients + acknowledge the right of that site to retain the role of + master for a period of time. During that period of time, + clients cannot elect a new master, become master, or grant + their lease to another site. + </p> + <p> + By holding a collection of granted leases, a master site + can guarantee to the application that the data returned is the + current, authoritative value. As a master performs operations, + it continually requests updated grants from the clients. When + a read operation is required, the master guarantees that it + holds a valid collection of lease grants from clients before + returning data to the application. By holding leases, Berkeley + DB provides several guarantees to the application: + </p> <div class="orderedlist"> <ol type="1"> <li> - Authoritative reads: A guarantee that the data being read by the - application is the current value. - </li> + Authoritative reads: A guarantee that the data + being read by the application is the current value. + </li> <li> <p> - Durability from rollback: A guarantee that the data being - written or read by the application is permanent across a - majority of client sites and will never be rolled back. - </p> - <p> - The rollback guarantee also depends on the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> flag. The - guarantee is effective as long as there isn't total replication group - failure while clients have granted leases but are holding the updates - in their cache. The application must weigh the performance impact of - synchronous transactions against the risk of total replication group - failure. If clients grant a lease while holding updated data in cache, - and total failure occurs, then the data is no longer present on the - clients and rollback can occur if the master also crashes. - </p> + Durability from rollback: A guarantee that the data + being written or read by the application is permanent + across a majority of client sites and will never be + rolled back. + </p> <p> - The guarantee that data will not be rolled back applies only to data - successfully committed on a master. Data read on a client, or read - while ignoring leases can be rolled back. - </p> + The rollback guarantee also depends on the + <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> flag. The guarantee is effective as + long as there isn't a failure of half of the + replication group while clients have granted leases + but are holding the updates in their cache. The + application must weigh the performance impact of + synchronous transactions against the risk of the + failure of at least half of the replication group. If + clients grant a lease while holding updated data in + cache, and failure occurs, then the data is no longer + present on the clients and rollback can occur if a + sufficient number of other sites also crash. + </p> + <p> + The guarantee that data will not be rolled back + applies only to data successfully committed on a + master. Data read on a client, or read while ignoring + leases can be rolled back. + </p> </li> <li> + <p> + Freshness: A guarantee that the data being read by + the application on the <span class="emphasis"><em>master</em></span> is + up-to-date and has not been modified or removed during + the read. + </p> <p> - Freshness: A guarantee that the data being read by the - application on the <span class="emphasis"><em>master</em></span> is up-to-date - and has not been modified or removed during the read. - </p> - <p> - The read authority is only on the master. Read operations on a - client always ignore leases and consequently, these operations - can return stale data. - </p> + The read authority is only on the master. Read + operations on a client always ignore leases and + consequently, these operations can return stale data. + </p> </li> <li> - <p> - Master viability: A guarantee that a current master with valid - leases cannot encounter a duplicate master situation. - </p> - <p> - Leases remove the possibility of a duplicate master situation that - forces the current master to downgrade to a client. However, it is - still possible that old masters with expired leases can discover a - later master and return <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_DUPMASTER" class="olink">DB_REP_DUPMASTER</a> to the application. - </p> + <p> + Master viability: A guarantee that a current master + with valid leases cannot encounter a duplicate master + situation. + </p> + <p> + Leases remove the possibility of a duplicate master + situation that forces the current master to downgrade + to a client. However, it is still possible that old + masters with expired leases can discover a later + master and return <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_DUPMASTER" class="olink">DB_REP_DUPMASTER</a> to the + application. + </p> </li> </ol> </div> <p> - There are several requirements of the application using leases: -</p> + There are several requirements of the application using + leases: + </p> <div class="orderedlist"> <ol type="1"> + <li> + Replication Manager applications must configure a + majority (or larger) acknowledgement policy via the + <a href="../api_reference/C/repmgrset_ack_policy.html" class="olink">DB_ENV->repmgr_set_ack_policy()</a> method. Base API applications must + implement and enforce such a policy on their own. + </li> + <li> + Base API applications must return an error from the + send callback function when the majority acknowledgement + policy is not met for permanent records marked with + <a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a>. Note that the Replication Manager + automatically fulfills this requirement. + </li> + <li> + Base API applications must set the number of sites + in the group using the <a href="../api_reference/C/repnsites.html" class="olink">DB_ENV->rep_set_nsites()</a> method before starting + replication and cannot change it during operation. + </li> + <li> + Using leases in a replication group is all or none. + Behavior is undefined when some sites configure leases and + others do not. Use the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method to turn on + leases. + </li> <li> - Replication Manager applications must configure a majority (or - larger) acknowledgement policy via the <a href="../api_reference/C/repmgrset_ack_policy.html" class="olink">DB_ENV->repmgr_set_ack_policy()</a> method. - Base API applications must implement and enforce such a policy on - their own. - </li> - <li> - Base API applications must return an error from the send callback - function when the majority acknowledgement policy is not met for - permanent records marked with <a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a>. Note that the - Replication Manager automatically fulfills this requirement. - </li> + The configured lease timeout value must be the same + on all sites in a replication group, set via the + <a href="../api_reference/C/repset_timeout.html" class="olink">DB_ENV->rep_set_timeout()</a> method. + </li> + <li> + The configured clock skew ratio must be the same on + all sites in a replication group. This value defaults to + no skew, but can be set via the <a href="../api_reference/C/repclockskew.html" class="olink">DB_ENV->rep_set_clockskew()</a> method. + </li> + <li> + Applications that care about read guarantees must + perform all read operations on the master. Reading on a + client does not guarantee freshness. + </li> <li> - Base API applications must set the number of sites in the group - using the <a href="../api_reference/C/repnsites.html" class="olink">DB_ENV->rep_set_nsites()</a> method before starting replication and cannot - change it during operation. - </li> + The application must use elections to choose a + master site. It must never simply declare a master without + having won an election (as is allowed without Master + Leases). + </li> <li> - Using leases in a replication group is all or none. Behavior is - undefined when some sites configure leases and others do not. Use - the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method to turn on leases. - </li> - <li> - The configured lease timeout value must be the same on all sites - in a replication group, set via the <a href="../api_reference/C/repset_timeout.html" class="olink">DB_ENV->rep_set_timeout()</a> method. - </li> - <li> - The configured clock_scale_factor value must be the same on all - sites in a replication group. This value defaults to no skew, but - can be set via the <a href="../api_reference/C/repclockskew.html" class="olink">DB_ENV->rep_set_clockskew()</a> method. - </li> - <li> - Applications that care about read guarantees must perform all read - operations on the master. Reading on a client does not guarantee - freshness. - </li> - <li> - The application must use elections to choose a master site. It - must never simply declare a master without having won an election - (as is allowed without Master Leases). - </li> + Unelectable (zero priority) sites never grant + leases and cannot be used to guarantee data durability. A + majority of sites in the replication group must be + electable in order to meet the requirement of getting + lease grants from a majority of sites. Minimizing the + number of unelectable sites improves replication group + availability. + </li> </ol> </div> + <p> + Master leases are based on timeouts. Berkeley DB assumes + that time always runs forward. Users who change the system + clock on either client or master sites when leases are in use + void all guarantees and can get undefined behavior. See the + <a href="../api_reference/C/repset_timeout.html" class="olink">DB_ENV->rep_set_timeout()</a> method for more information. + </p> <p> - Master leases are based on timeouts. Berkeley DB assumes that time - always runs forward. Users who change the system clock on either - client or master sites when leases are in use void all guarantees and - can get undefined behavior. See the <a href="../api_reference/C/repset_timeout.html" class="olink">DB_ENV->rep_set_timeout()</a> method for more - information. -</p> - <p> - Applications using master leases should be prepared to handle - <code class="literal">DB_REP_LEASE_EXPIRED</code> errors from read operations - on a master and from the <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a> method. -</p> - <p> - Read operations on a master that should not be subject to leases can - use the <a href="../api_reference/C/dbget.html#get_DB_IGNORE_LEASE" class="olink">DB_IGNORE_LEASE</a> flag to the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> method. Read operations - on a client always imply leases are ignored. -</p> - <p> - Master lease checks cannot succeed until a majority of sites have - completed client synchronization. Read operations on a master performed - before this condition is met can use the <a href="../api_reference/C/dbget.html#get_DB_IGNORE_LEASE" class="olink">DB_IGNORE_LEASE</a> flag to - avoid errors. -</p> + Applications using master leases should be prepared to + handle <code class="literal">DB_REP_LEASE_EXPIRED</code> errors from + read operations on a master and from the <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a> method. + </p> + <p> + Read operations on a master that should not be subject to + leases can use the <a href="../api_reference/C/dbget.html#get_DB_IGNORE_LEASE" class="olink">DB_IGNORE_LEASE</a> flag to the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> + method. Read operations on a client always imply leases are + ignored. + </p> + <p> + Master lease checks cannot succeed until a majority of + sites have completed client synchronization. Read operations + on a master performed before this condition is met can use the + <a href="../api_reference/C/dbget.html#get_DB_IGNORE_LEASE" class="olink">DB_IGNORE_LEASE</a> flag to avoid errors. + </p> <p> - Clients are forbidden from participating in elections while they have - an outstanding lease granted to a master. Therefore, if the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> - method is called, then Berkeley DB will block, waiting until its lease - grant expires before participating in any election. While it waits, - the client attempts to contact the current master. If the client finds - a current master, then it returns from the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method. When - leases are configured and the lease has never yet been granted (on - start-up), clients must wait a full lease timeout before participating - in an election. -</p> + Clients are forbidden from participating in elections while + they have an outstanding lease granted to a master. Therefore, + if the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method is called, then Berkeley DB will + block, waiting until its lease grant expires before + participating in any election. While it waits, the client + attempts to contact the current master. If the client finds a + current master, then it returns from the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method. + When leases are configured and the lease has never yet been + granted (on start-up), clients must wait a full lease timeout + before participating in an election. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="masterlease_change_groupsize"></a>Changing Group Size</h3> + <h3 class="title"><a id="masterlease_change_groupsize"></a>Changing group + size</h3> </div> </div> </div> <p> - If you are using master leases and you change the size of your - replication group, there is a remote possibility that you can - lose some data previously thought to be durable. This is only - true for users of the Base API. + If you are using master leases and you change the size + of your replication group, there is a remote possibility + that you can lose some data previously thought to be + durable. This is only true for users of the Base API. </p> <p> - The problem can arise if you are removing sites from your - replication group. (You might be increasing the size of your - site overall, but if you remove all of the wrong sites you can - lose data.) + The problem can arise if you are removing sites from + your replication group. (You might be increasing the size + of your group overall, but if you remove all of the wrong + sites you can lose data.) </p> - <p> - Suppose you have a replication group with five sites; A, B, C, - D and E; and you are using a quorum acknowledgement policy. Then: + <p> + Suppose you have a replication group with five sites; + A, B, C, D and E; and you are using a quorum + acknowledgement policy. Then: </p> <div class="orderedlist"> <ol type="1"> <li> - <p> - Master A replicates a transaction to replicas B and C. - Those sites acknowledge the write activity. + <p> + Master A replicates a transaction to replicas B + and C. Those sites acknowledge the write activity. </p> </li> <li> - <p> - Sites D and E do not receive the transaction. However, - B and C have acknowledged the transaction, which means the - acknowledgement policy is met and so the transaction is - considered durable. + <p> + Sites D and E do not receive the transaction. + However, B and C have acknowledged the + transaction, which means the acknowledgement + policy is met and so the transaction is considered + durable. </p> </li> <li> <p> - You shutdown sites B and C. Now only A has the transaction. + You shut down sites B and C. Now only A has the + transaction. </p> </li> <li> <p> - You increase the size of your replication group to 3 - using <a href="../api_reference/C/repnsites.html" class="olink">DB_ENV->rep_set_nsites()</a>. + You decrease the size of your replication group + to 3 using <a href="../api_reference/C/repnsites.html" class="olink">DB_ENV->rep_set_nsites()</a>. </p> </li> <li> <p> - You shutdown or otherwise lose site A. + You shut down or otherwise lose site A. </p> </li> <li> - <p> - Sites D and E hold an election. Because the size of the - replication group is 3, they have enough sites to - successfully hold an election. However, neither site - has the transaction in question. In this way, the - transaction can become lost. + <p> + Sites D and E hold an election. Because the + size of the replication group is 3, they have + enough sites to successfully hold an election. + However, neither site has the transaction in + question. In this way, the transaction can become + lost. </p> </li> </ol> </div> <p> - An alternative scenario exists where you do not change the size - of your replication group, or you actually increase the size of - your replication group, but in the process you happen to remove - the exact wrong sites: + An alternative scenario exists where you do not change + the size of your replication group, or you actually + increase the size of your replication group, but in the + process you happen to remove the exact wrong sites: </p> <div class="orderedlist"> <ol type="1"> <li> <p> - Master A replicates a transaction to replicas B and C. - Those sites acknowledge the write activity. + Master A replicates a transaction to replicas B + and C. Those sites acknowledge the write activity. </p> </li> <li> - <p> - Sites D and E do not receive the transaction. However, - B and C have acknowledged the transaction, which means the - acknowledgement policy is met and so the transaction is - considered durable. + <p> + Sites D and E do not receive the transaction. + However, B and C have acknowledged the + transaction, which means the acknowledgement + policy is met and so the transaction is considered + durable. </p> </li> <li> - <p> - You shutdown sites B and C. Now only A has the transaction. + <p> + You shut down sites B and C. Now only A has the + transaction. </p> </li> <li> <p> - You add three new sites to your replication group: F, - G and H, increasing the size of your replication group - to 6 using <a href="../api_reference/C/repnsites.html" class="olink">DB_ENV->rep_set_nsites()</a>. + You add three new sites to your replication + group: F, G and H, increasing the size of your + replication group to 6 using <a href="../api_reference/C/repnsites.html" class="olink">DB_ENV->rep_set_nsites()</a>. </p> </li> <li> <p> - You shutdown or otherwise lose site A before F, G and H - can be fully populated with data. + You shut down or otherwise lose site A before + F, G and H can be fully populated with data. </p> </li> <li> <p> - Sites D, E, F, G and H hold an election. Because the size of the - replication group is 6, they have enough sites to - successfully hold an election. However, none of these sites - has the transaction in question. In this way, the - transaction can become lost. + Sites D, E, F, G and H hold an election. + Because the size of the replication group is 6, + they have enough sites to successfully hold an + election. However, none of these sites has the + transaction in question. In this way, the + transaction can become lost. </p> </li> </ol> </div> <p> - This scenario represents a race condition that would be highly - unlikely to be seen outside of a lab environment. To minimize - the chance of this race condition occurring to the absolute - minimum, do one or more of the following when using master - leases with the Base API: + This scenario represents a race condition that would be + highly unlikely to be seen outside of a lab environment. + To minimize the chance of this race condition occurring to + the absolute minimum, do one or more of the following when + using master leases with the Base API: </p> <div class="orderedlist"> <ol type="1"> <li> <p> - Require all sites to acknowledge transaction commits. + Require all sites to acknowledge transaction + commits. </p> </li> <li> <p> - Never change the size of your replication group unless - all sites in the group are running and communicating - normally with one another. + Never change the size of your replication group + unless all sites in the group are running and + communicating normally with one another. </p> </li> <li> - <p> - Don't remove (or replace) a large percentage of your - sites from your replication group unless all sites in - the group are running and communicating normally with - one another. If you are going to remove a large - percentage of your sites from your replication group, - try removing just one site at a time, pausing in - between each removal to give the replication group a - chance to fully distribute all writes before removing - the next site. + <p> + Don't remove (or replace) a large percentage of + your sites from your replication group unless all + sites in the group are running and communicating + normally with one another. If you are going to + remove a large percentage of your sites from your + replication group, try removing just one site at a + time, pausing in between each removal to give the + replication group a chance to fully distribute all + writes before removing the next site. </p> </li> </ol> diff --git a/docs/programmer_reference/rep_mastersync.html b/docs/programmer_reference/rep_mastersync.html index 203b61aa..05506c54 100644 --- a/docs/programmer_reference/rep_mastersync.html +++ b/docs/programmer_reference/rep_mastersync.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Synchronizing with a master</th> + <th colspan="3" align="center">Synchronizing with a + master</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_elect.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_init.html">Next</a></td> </tr> </table> @@ -34,7 +33,8 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="rep_mastersync"></a>Synchronizing with a master</h2> + <h2 class="title" style="clear: both"><a id="rep_mastersync"></a>Synchronizing with a + master</h2> </div> </div> </div> @@ -52,21 +52,25 @@ </dt> <dt> <span class="sect2"> - <a href="rep_mastersync.html#idp2309616">Blocked client operations</a> + <a href="rep_mastersync.html#idp1985136">Blocked client operations</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_mastersync.html#idp2331664">Clients too far out-of-date to synchronize</a> + <a href="rep_mastersync.html#idp2008240">Clients too far out-of-date to synchronize</a> </span> </dt> </dl> </div> - <p>When a client detects a new replication group master, the client must -synchronize with the new master before the client can process new -database changes. Synchronizing is a heavyweight operation which can -place a burden on both the client and the master. There are several -controls an application can use to reduce the synchronization burden.</p> + <p> + When a client detects a new replication group master, the + client must synchronize with the new master before the client + can process new database changes. Synchronizing is a + heavyweight operation which can place a burden on both the + client and the master. There are several controls an + application can use to reduce the synchronization + burden. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> @@ -75,21 +79,30 @@ controls an application can use to reduce the synchronization burden.</p> </div> </div> </div> - <p>When a replication group has a new master, either as specified by the -application or as a result of winning an election, all clients in the -replication group must synchronize with the new master. This can -strain the resources of the new master since a large number of clients -may be attempting to communicate with and transfer records from the -master. Client applications wanting to delay client synchronization -should call the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method with the -<a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_DELAYCLIENT" class="olink">DB_REP_CONF_DELAYCLIENT</a> flag. The application will be -notified of the establishment of the new master as usual, but the -client will not proceed to synchronize with the new master.</p> - <p>Applications learn of a new master via the -<a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_NEWMASTER" class="olink">DB_EVENT_REP_NEWMASTER</a> event.</p> - <p>Client applications choosing to delay synchronization in this manner are -responsible for synchronizing the client environment at some future time -using the <a href="../api_reference/C/repsync.html" class="olink">DB_ENV->rep_sync()</a> method.</p> + <p> + When a replication group has a new master, either as + specified by the application or as a result of winning an + election, all clients in the replication group must + synchronize with the new master. This can strain the + resources of the new master since a large number of + clients may be attempting to communicate with and transfer + records from the master. Client applications wanting to + delay client synchronization should call the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> + method with the <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_DELAYCLIENT" class="olink">DB_REP_CONF_DELAYCLIENT</a> flag. The + application will be notified of the establishment of the + new master as usual, but the client will not proceed to + synchronize with the new master. + </p> + <p> + Applications learn of a new master via the + <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_NEWMASTER" class="olink">DB_EVENT_REP_NEWMASTER</a> event. + </p> + <p> + Client applications choosing to delay synchronization in + this manner are responsible for synchronizing the client + environment at some future time using the <a href="../api_reference/C/repsync.html" class="olink">DB_ENV->rep_sync()</a> + method. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> @@ -99,92 +112,125 @@ using the <a href="../api_reference/C/repsync.html" class="olink">DB_ENV->rep </div> </div> </div> - <p>Instead of synchronizing with the new master, it is sometimes possible -for a client to synchronize with another client. Berkeley DB initiates -synchronization at the client by sending a request message via the -transport call-back function of the communication infrastructure. The -message is destined for the master site, but is also marked with a -<a href="../api_reference/C/reptransport.html#transport_DB_REP_ANYWHERE" class="olink">DB_REP_ANYWHERE</a> flag. The application may choose to send such -a request to another client, or to ignore the flag, sending it to its -indicated destination.</p> - <p>Furthermore, when the other client receives such a request it may be -unable to satisfy it. In this case it will reply to the requesting -client, telling it that it is unable to provide the requested -information. The requesting client will then re-issue the request. -Additionally, if the original request never reaches the other client, -the requesting client will again re-issue the request. In either of -these cases the message will be marked with the <a href="../api_reference/C/reptransport.html#transport_DB_REP_REREQUEST" class="olink">DB_REP_REREQUEST</a> -flag. The application may continue trying to find another client to -service the request, or it may give up and simply send it to the master -(that is, the environment ID explicitly specified to the transport -function).</p> <p> - Replication Manager allows an application to designate one or more - remote sites (called its "peers") to receive client-to-client requests. - You do this by setting the <code class="literal">DB_REPMGR_PEER</code> parameter - using the <a href="../api_reference/C/dbsite_set_config.html" class="olink">DB_SITE->set_config()</a> method. Replication Manager always tries - to send requests marked with the <a href="../api_reference/C/reptransport.html#transport_DB_REP_ANYWHERE" class="olink">DB_REP_ANYWHERE</a> flag to a peer, if - available. However, it always sends a <a href="../api_reference/C/reptransport.html#transport_DB_REP_REREQUEST" class="olink">DB_REP_REREQUEST</a> to the master - site. -</p> - <p>Base API applications have complete freedom -in choosing where to send these <a href="../api_reference/C/reptransport.html#transport_DB_REP_ANYWHERE" class="olink">DB_REP_ANYWHERE</a> requests, and -in deciding how to handle <a href="../api_reference/C/reptransport.html#transport_DB_REP_REREQUEST" class="olink">DB_REP_REREQUEST</a>.</p> - <p>The delayed synchronization and client-to-client synchronization -features allow applications to do load balancing within replication -groups. For example, consider a replication group with 5 sites, A, B, -C, D and E. Site E just crashed, and site A was elected master. Sites -C and D have been configured for delayed synchronization. When site B -is notified that site A is a new master, it immediately synchronizes. -When B finishes synchronizing with the master, the application calls the -<a href="../api_reference/C/repsync.html" class="olink">DB_ENV->rep_sync()</a> method on sites C and D to cause them to synchronize as well. -Sites C and D (and E, when it has finished rebooting) can send their -requests to site B, and B then bears the brunt of the work and -network traffic for synchronization, making master site A available to -handle the normal application load and any write requests paused by -the election.</p> + Instead of synchronizing with the new master, it is + sometimes possible for a client to synchronize with + another client. Berkeley DB initiates synchronization at + the client by sending a request message via the transport + call-back function of the communication infrastructure. + The message is destined for the master site, but is also + marked with a <a href="../api_reference/C/reptransport.html#transport_DB_REP_ANYWHERE" class="olink">DB_REP_ANYWHERE</a> flag. The application may + choose to send such a request to another client, or to + ignore the flag, sending it to its indicated + destination. + </p> + <p> + Furthermore, when the other client receives such a + request it may be unable to satisfy it. In this case it + will reply to the requesting client, telling it that it is + unable to provide the requested information. The + requesting client will then re-issue the request. + Additionally, if the original request never reaches the + other client, the requesting client will again re-issue + the request. In either of these cases the message will be + marked with the <a href="../api_reference/C/reptransport.html#transport_DB_REP_REREQUEST" class="olink">DB_REP_REREQUEST</a> flag. The application + may continue trying to find another client to service the + request, or it may give up and simply send it to the + master (that is, the environment ID explicitly specified + to the transport function). + </p> + <p> + Replication Manager allows an application to designate + one or more remote sites (called its "peers") to receive + client-to-client requests. You do this by setting the + <code class="literal">DB_REPMGR_PEER</code> parameter using the + <a href="../api_reference/C/dbsite_set_config.html" class="olink">DB_SITE->set_config()</a> method. Replication Manager always + tries to send requests marked with the <a href="../api_reference/C/reptransport.html#transport_DB_REP_ANYWHERE" class="olink">DB_REP_ANYWHERE</a> + flag to a peer, if available. However, a + <a href="../api_reference/C/reptransport.html#transport_DB_REP_REREQUEST" class="olink">DB_REP_REREQUEST</a> message is always handled by requesting + the information from the master. A view can serve as a + peer, but a partial view is more likely to be missing + requested information, which will then be requested from + the master. + </p> + <p> + Base API applications have complete freedom in choosing + where to send these <a href="../api_reference/C/reptransport.html#transport_DB_REP_ANYWHERE" class="olink">DB_REP_ANYWHERE</a> requests, and in + deciding how to handle <a href="../api_reference/C/reptransport.html#transport_DB_REP_REREQUEST" class="olink">DB_REP_REREQUEST</a>. + </p> + <p> + The delayed synchronization and client-to-client + synchronization features allow applications to do load + balancing within replication groups. For example, consider + a replication group with 5 sites, A, B, C, D and E. Site E + just crashed, and site A was elected master. Sites C and D + have been configured for delayed synchronization. When + site B is notified that site A is a new master, it + immediately synchronizes. When B finishes synchronizing + with the master, the application calls the <a href="../api_reference/C/repsync.html" class="olink">DB_ENV->rep_sync()</a> + method on sites C and D to cause them to synchronize as + well. Sites C and D (and E, when it has finished + rebooting) can send their requests to site B, and B then + bears the brunt of the work and network traffic for + synchronization, making master site A available to handle + the normal application load and any write requests paused + by the election. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2309616"></a>Blocked client operations</h3> + <h3 class="title"><a id="idp1985136"></a>Blocked client operations</h3> </div> </div> </div> - <p>Clients in the process of synchronizing with the master block access to -Berkeley DB operations during some parts of that process. -By default, most Berkeley DB methods will block until -client synchronization is complete, and then the method call proceeds.</p> - <p>Client applications which cannot wait and would prefer an immediate -error return instead of blocking, should call the -<a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method with the <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_NOWAIT" class="olink">DB_REP_CONF_NOWAIT</a> flag. This -configuration causes <a href="../api_reference/C/db.html" class="olink">DB</a> method calls to immediately return a -<a href="../api_reference/C/dbput.html#dbput_DB_REP_LOCKOUT" class="olink">DB_REP_LOCKOUT</a> error instead of blocking, if the client is -currently synchronizing with the master.</p> + <p> + Clients in the process of synchronizing with the master + block access to Berkeley DB operations during some parts + of that process. By default, most Berkeley DB methods will + block until client synchronization is complete, and then + the method call proceeds. + </p> + <p> + Client applications which cannot wait and would prefer + an immediate error return instead of blocking, should call + the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method with the <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_NOWAIT" class="olink">DB_REP_CONF_NOWAIT</a> flag. + This configuration causes <a href="../api_reference/C/db.html" class="olink">DB</a> method calls to immediately + return a <a href="../api_reference/C/dbput.html#dbput_DB_REP_LOCKOUT" class="olink">DB_REP_LOCKOUT</a> error instead of blocking, if + the client is currently synchronizing with the + master. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2331664"></a>Clients too far out-of-date to synchronize</h3> + <h3 class="title"><a id="idp2008240"></a>Clients too far out-of-date to synchronize</h3> </div> </div> </div> - <p>Clients attempting to synchronize with the master may discover that -synchronization is not possible because the client no longer has any -overlapping information with the master site. By default, the master and -client automatically detect this state and perform an internal initialization -of the client. Because internal initialization requires transfer of -entire databases to the client, it can take a relatively long period of -time and may require database handles to be reopened in the client -applications.</p> - <p>Client applications which cannot wait or would prefer -to do a hot backup instead of performing internal initialization, should -call the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method to turn off the <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_AUTOINIT" class="olink">DB_REP_CONF_AUTOINIT</a> flag. -Turning off this configuration flag causes Berkeley DB to return -<a href="../api_reference/C/repmessage.html#repmsg_DB_REP_JOIN_FAILURE" class="olink">DB_REP_JOIN_FAILURE</a> to the application instead of performing -internal initialization.</p> + <p> + Clients attempting to synchronize with the master may + discover that synchronization is not possible because the + client no longer has any overlapping information with the + master site. By default, the master and client + automatically detect this state and perform an internal + initialization of the client. Because internal + initialization requires transfer of entire databases to + the client, it can take a relatively long period of time + and may require database handles to be reopened in the + client applications. + </p> + <p> + Client applications which cannot wait or would prefer to + do a hot backup instead of performing internal + initialization, should call the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method to turn + off the <a href="../api_reference/C/repconfig.html#config_DB_REP_CONF_AUTOINIT" class="olink">DB_REP_CONF_AUTOINIT</a> flag. Turning off this + configuration flag causes Berkeley DB to return + <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_JOIN_FAILURE" class="olink">DB_REP_JOIN_FAILURE</a> to the application instead of + performing internal initialization. + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/rep_mgr_ack.html b/docs/programmer_reference/rep_mgr_ack.html index 9b9a847c..41d9e935 100644 --- a/docs/programmer_reference/rep_mgr_ack.html +++ b/docs/programmer_reference/rep_mgr_ack.html @@ -3,28 +3,26 @@ <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> - <title>Choosing a Replication Manager Ack Policy</title> + <title>Choosing a Replication Manager acknowledgement policy</title> <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> <link rel="up" href="rep.html" title="Chapter 12. Berkeley DB Replication" /> - <link rel="prev" href="rep_replicate.html" title="Running Replication using the db_replicate Utility" /> + <link rel="prev" href="rep_replicate.html" title="Running replication using the db_replicate utility" /> <link rel="next" href="rep_elect.html" title="Elections" /> </head> <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Choosing a Replication Manager Ack Policy</th> + <th colspan="3" align="center">Choosing a Replication Manager acknowledgement policy</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_replicate.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_elect.html">Next</a></td> </tr> </table> @@ -34,75 +32,114 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="rep_mgr_ack"></a>Choosing a Replication Manager Ack Policy</h2> + <h2 class="title" style="clear: both"><a id="rep_mgr_ack"></a>Choosing a Replication Manager acknowledgement policy</h2> </div> </div> </div> - <p>Replication Manager allows the user to choose from a variety of -acknowledgement policies. There are two characteristics that should -be considered when choosing the policy: consistency and durability. -Consistency means making sure some number of clients have applied -all available master transactions. Durability, in this context, -means only indicating success only if enough clients have -applied a transaction. The issue of how many is enough depends -on the application's requirements and varies per acknowledgement policy. -For example, <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_QUORUM" class="olink">DB_REPMGR_ACKS_QUORUM</a> means the data will survive -a change in master or a network partition. -In most cases, the number of sites for consistency is -equal to the number of sites for durability. Replication Manager -uses the consistency value to decide whether or not to wait for -acknowledgements. Replication manager uses the durability value -to decide either the transaction was successfully processed or that -a <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_PERM_FAILED" class="olink">DB_EVENT_REP_PERM_FAILED</a> event should be generated.</p> - <p>Replication Manager also strives to give the application the -answer and return to the application as quickly as possible. Therefore, -if it knows that the number of sites connected is insufficient to meet -the consistency value, then it does not wait for any acknowledgements -and if it knows that the durability value cannot be met, it returns -<a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_PERM_FAILED" class="olink">DB_EVENT_REP_PERM_FAILED</a> immediately to the user.</p> - <p>With one exception, discussed below, all acknowledgement policies combine -the consistency and durability values. For most policies the primary -purpose is the durability of the data. For example, the <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_QUORUM" class="olink">DB_REPMGR_ACKS_QUORUM</a> -policy ensures that, if successful, the transaction's data is safe in -the event of a network partition so that a majority of the sites in -the group have the data. The <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_NONE" class="olink">DB_REPMGR_ACKS_NONE</a> policy does not -consider either consistency or durability, and it is very -fast because it does not wait for any acknowledgements and it does not -ever trigger the <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_PERM_FAILED" class="olink">DB_EVENT_REP_PERM_FAILED</a> event. Other policies, -<a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ALL" class="olink">DB_REPMGR_ACKS_ALL</a> and <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ALL_PEERS" class="olink">DB_REPMGR_ACKS_ALL_PEERS</a>, have a primary purpose of -consistency. These two policies wait for acknowledgements from all -(or all electable) sites in the group.</p> - <p>In the face of failure, however, the <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ALL" class="olink">DB_REPMGR_ACKS_ALL</a> and -<a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ALL_PEERS" class="olink">DB_REPMGR_ACKS_ALL_PEERS</a> policies can result -in a surprising lack of consistency due to the fact that Replication Manager -strives to give the answer back to the application as fast as it can. -So, for example, with <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ALL" class="olink">DB_REPMGR_ACKS_ALL</a>, and one site down, Replication -Manager knows that disconnected site can never acknowledge, so it -immediately triggers <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_PERM_FAILED" class="olink">DB_EVENT_REP_PERM_FAILED</a>. An unfortunate side -effect of this policy is that existing, running sites may fall further and -further behind the master if the master site is sending a fast, busy -stream of transactions and never waiting for any site to send an -acknowledgement. The master does not wait because the consistency -value cannot be met, and it does trigger the <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_PERM_FAILED" class="olink">DB_EVENT_REP_PERM_FAILED</a> -event because the durability value cannot be met, but those -actions now affect the consistency of the other running sites.</p> - <p>In order to counteract this unfortunate side effect, -the <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ALL_AVAILABLE" class="olink">DB_REPMGR_ACKS_ALL_AVAILABLE</a> acknowledgement policy focuses on the -consistency aspect, but also considers durability. This policy -uses all sites for consistency, -and a quorum of sites for its decision about durability. As long -as there is a non-zero number of client replicas to send to, the -master will wait for all available sites to acknowledge the -transaction. As long as any client site is connected, this policy -will prevent the master from racing ahead if one or more sites is down. -On the master, this policy will then consider the transaction durable if -the number of acknowledgements meets quorum for the group.</p> - <p>The following acknowledgement policies determine durability -using acknowledgements from electable peers only: <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_QUORUM" class="olink">DB_REPMGR_ACKS_QUORUM</a>, -<a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ONE_PEER" class="olink">DB_REPMGR_ACKS_ONE_PEER</a>, <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ALL_PEERS" class="olink">DB_REPMGR_ACKS_ALL_PEERS</a>. An electable -peer is a site where the priority value is greater than zero. In -replication groups using these policies, an unelectable site does not -send acknowledgements and cannot contribute to transaction durability.</p> + <p> + Replication Manager allows the user to choose from a variety + of acknowledgement policies. There are two characteristics + that should be considered when choosing the policy: + consistency and durability. Consistency means making sure some + number of clients have applied all available master + transactions. Durability, in this context, means only + indicating success only if enough clients have applied a + transaction. The issue of how many is enough depends on the + application's requirements and varies per acknowledgement + policy. For example, <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_QUORUM" class="olink">DB_REPMGR_ACKS_QUORUM</a> means the data + will survive a change in master or a network partition. In + most cases, the number of sites for consistency is equal to + the number of sites for durability. Replication Manager uses + the consistency value to decide whether or not to wait for + acknowledgements. Replication manager uses the durability + value to decide either the transaction was successfully + processed or that a <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_PERM_FAILED" class="olink">DB_EVENT_REP_PERM_FAILED</a> event should be + generated. + </p> + <p> + Replication Manager also strives to give the application the + answer and return to the application as quickly as possible. + Therefore, if it knows that the number of sites connected is + insufficient to meet the consistency value, then it does not + wait for any acknowledgements and if it knows that the + durability value cannot be met, it returns + <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_PERM_FAILED" class="olink">DB_EVENT_REP_PERM_FAILED</a> immediately to the user. + </p> + <p> + With one exception, discussed below, all acknowledgement + policies combine the consistency and durability values. For + most policies the primary purpose is the durability of the + data. For example, the <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_QUORUM" class="olink">DB_REPMGR_ACKS_QUORUM</a> policy ensures + that, if successful, the transaction's data is safe in the + event of a network partition so that a majority of the sites + in the group have the data. The <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_NONE" class="olink">DB_REPMGR_ACKS_NONE</a> policy + does not consider either consistency or durability, and it is + very fast because it does not wait for any acknowledgements + and it does not ever trigger the <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_PERM_FAILED" class="olink">DB_EVENT_REP_PERM_FAILED</a> + event. Other policies, <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ALL" class="olink">DB_REPMGR_ACKS_ALL</a> and + <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ALL_PEERS" class="olink">DB_REPMGR_ACKS_ALL_PEERS</a>, have a primary purpose of + consistency. These two policies wait for acknowledgements from + all (or all electable) sites in the group. + </p> + <p> + In the face of failure, however, the <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ALL" class="olink">DB_REPMGR_ACKS_ALL</a> + and <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ALL_PEERS" class="olink">DB_REPMGR_ACKS_ALL_PEERS</a> policies can result in a + surprising lack of consistency due to the fact that + Replication Manager strives to give the answer back to the + application as fast as it can. So, for example, with + <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ALL" class="olink">DB_REPMGR_ACKS_ALL</a>, and one site down, Replication Manager + knows that disconnected site can never acknowledge, so it + immediately triggers <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_PERM_FAILED" class="olink">DB_EVENT_REP_PERM_FAILED</a>. An + unfortunate side effect of this policy is that existing, + running sites may fall further and further behind the master + if the master site is sending a fast, busy stream of + transactions and never waiting for any site to send an + acknowledgement. The master does not wait because the + consistency value cannot be met, and it does trigger the + <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_PERM_FAILED" class="olink">DB_EVENT_REP_PERM_FAILED</a> event because the durability value + cannot be met, but those actions now affect the consistency of + the other running sites. + </p> + <p> + In order to counteract this unfortunate side effect, the + <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ALL_AVAILABLE" class="olink">DB_REPMGR_ACKS_ALL_AVAILABLE</a> acknowledgement policy focuses + on the consistency aspect, but also considers durability. This + policy uses all sites for consistency, and a quorum of sites + for its decision about durability. As long as there is a + non-zero number of client replicas to send to, the master will + wait for all available sites to acknowledge the transaction. + As long as any client site is connected, this policy will + prevent the master from racing ahead if one or more sites is + down. On the master, this policy will then consider the + transaction durable if the number of acknowledgements meets + quorum for the group. + </p> + <p> + The following acknowledgement policies determine durability + using acknowledgements from electable peers only: + <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_QUORUM" class="olink">DB_REPMGR_ACKS_QUORUM</a>, <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ONE_PEER" class="olink">DB_REPMGR_ACKS_ONE_PEER</a>, + <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ALL_PEERS" class="olink">DB_REPMGR_ACKS_ALL_PEERS</a>. An electable peer is a site where + the priority value is greater than zero. In replication groups + using these policies, an unelectable site does not send + acknowledgements and cannot contribute to transaction + durability. + </p> + <p> + If your application needs to associate a <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_PERM_FAILED" class="olink">DB_EVENT_REP_PERM_FAILED</a> + event with the specific transaction and thread that caused it, + this can be done by using your platform's thread-specific data + facility. You can create a thread-specific data structure that + contains a durability indicator. You would set the durability + indicator in the <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_PERM_FAILED" class="olink">DB_EVENT_REP_PERM_FAILED</a> portion of your event + notification callback function. You can then check and clear the + durability indicator after your transaction commit operation + completes. You must be careful to coordinate this approach across + all application threads performing transaction commits or checkpoints + because any of these threads can generate a <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_PERM_FAILED" class="olink">DB_EVENT_REP_PERM_FAILED</a> + event. The ex_rep_mgr sample application demonstrates one way to + implement this capability. See + <a class="xref" href="rep_ex.html" title="Ex_rep: a replication example">Ex_rep: a replication example</a> for more information. + </p> </div> <div class="navfooter"> <hr /> @@ -115,7 +152,8 @@ send acknowledgements and cannot contribute to transaction durability.</p> <td width="40%" align="right"> <a accesskey="n" href="rep_elect.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Running Replication using the db_replicate Utility </td> + <td width="40%" align="left" valign="top">Running replication using the + db_replicate utility </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/rep_mgr_meth.html b/docs/programmer_reference/rep_mgr_meth.html index 24cd6ae9..63add04e 100644 --- a/docs/programmer_reference/rep_mgr_meth.html +++ b/docs/programmer_reference/rep_mgr_meth.html @@ -9,12 +9,12 @@ <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> <link rel="up" href="rep.html" title="Chapter 12. Berkeley DB Replication" /> <link rel="prev" href="rep_app.html" title="Building replicated applications" /> - <link rel="next" href="rep_base_meth.html" title="Base API Methods" /> + <link rel="next" href="rep_base_meth.html" title="Base API methods" /> </head> <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_app.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_base_meth.html">Next</a></td> </tr> </table> @@ -39,13 +37,13 @@ </div> </div> <p> - Applications which use the Replication Manager support generally - call the following Berkeley DB methods. The general pattern is to - call various methods to configure Replication Manager, and then - start it by calling <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a>. Once this initialization is - complete, the application rarely needs to call any of these - methods. (A prime example of an exception to this rule would be - the <a href="../api_reference/C/repsync.html" class="olink">DB_ENV->rep_sync()</a> method, if the application is + Applications which use the Replication Manager support + generally call the following Berkeley DB methods. The general + pattern is to call various methods to configure Replication + Manager, and then start it by calling <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a>. Once this + initialization is complete, the application rarely needs to + call any of these methods. (A prime example of an exception to + this rule would be the <a href="../api_reference/C/repsync.html" class="olink">DB_ENV->rep_sync()</a> method, if the application is <a class="xref" href="rep_mastersync.html#rep_delay_sync" title="Delaying client synchronization">Delaying client synchronization</a>.) </p> <div class="variablelist"> @@ -57,38 +55,41 @@ </dt> <dd> <p> - The <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle is used to configure a site that - belongs to the replication group. You can obtain a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> - handle by calling the <a href="../api_reference/C/repmgr_site.html" class="olink">DB_ENV->repmgr_site()</a> method. When you do this, you - provide the TCP/IP host name and port that the replication - site uses for incoming connections. + The <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle is used to configure a site + that belongs to the replication group. You can + obtain a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle by calling the + <a href="../api_reference/C/repmgr_site.html" class="olink">DB_ENV->repmgr_site()</a> method. When you do this, you + provide the TCP/IP host name and port that the + replication site uses for incoming connections. </p> - <p> + <p> Once you have the <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle, you use the - <a href="../api_reference/C/dbsite_set_config.html" class="olink">DB_SITE->set_config()</a> method to configure the handle. One of - the things you can configure about the handle is whether it - is the local site (using the <code class="literal">DB_LOCAL_SITE</code> - parameter). You must configure one and only one <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle - to be a local site before you start replication. + <a href="../api_reference/C/dbsite_set_config.html" class="olink">DB_SITE->set_config()</a> method to configure the + handle. One of the things you can configure about + the handle is whether it is the local site (using + the <code class="literal">DB_LOCAL_SITE</code> parameter). + You must configure one and only one <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> + handle to be a local site before you start + replication. </p> - <p> - You can also optionally configure <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handles for - remote sites to help Replication Manager startup more - efficiently. Note that it is usually not necessary for - each site in the replication group initially to know about - all other sites in the group. Sites can discover each - other dynamically, as described in - <a class="xref" href="rep_newsite.html" title="Connecting to a new site">Connecting to a new site</a>. + <p> + You can also optionally configure <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> + handles for remote sites to help Replication + Manager start up more efficiently. Note that it is + usually not necessary for each site in the + replication group initially to know about all + other sites in the group. Sites can discover each + other dynamically, as described in <a class="xref" href="rep_newsite.html" title="Connecting to a new site">Connecting to a new site</a>. </p> - <p> - Once you have configured your <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handles, you start - replication using <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a>. + <p> + Once you have configured your <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handles, + you start replication using <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a>. </p> - <p> - When you are shutting down your application, you must - use the <a href="../api_reference/C/dbsite_close.html" class="olink">DB_SITE->close()</a> method to close - all your open <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handles before you close your - environment handles. + <p> + When you are shutting down your application, + you must use the <a href="../api_reference/C/dbsite_close.html" class="olink">DB_SITE->close()</a> method to close + all your open <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handles before you close + your environment handles. </p> </dd> <dt> @@ -97,13 +98,15 @@ </span> </dt> <dd> - <p> - The <a href="../api_reference/C/repmgrset_ack_policy.html" class="olink">DB_ENV->repmgr_set_ack_policy()</a> method configures the acknowledgement - policy to be used in the replication group, in other words, the - behavior of the master with respect to acknowledgements for - "permanent" messages, which implements the application's - requirements for <a class="xref" href="rep_trans.html" title="Transactional guarantees">Transactional guarantees</a>. The current implementation - requires all sites in the replication group to configure the same + <p> + The <a href="../api_reference/C/repmgrset_ack_policy.html" class="olink">DB_ENV->repmgr_set_ack_policy()</a> method configures the + acknowledgement policy to be used in the + replication group, in other words, the behavior of + the master with respect to acknowledgements for + "permanent" messages, which implements the + application's requirements for <a class="xref" href="rep_trans.html" title="Transactional guarantees">Transactional guarantees</a>. + The current implementation requires all sites + in the replication group to configure the same acknowledgement policy. </p> </dd> @@ -113,9 +116,9 @@ </span> </dt> <dd> - <p> - The <a href="../api_reference/C/reppriority.html" class="olink">DB_ENV->rep_set_priority()</a> method configures the local site's priority - for the purpose of elections. + <p> + The <a href="../api_reference/C/reppriority.html" class="olink">DB_ENV->rep_set_priority()</a> method configures the local + site's priority for the purpose of elections. </p> </dd> <dt> @@ -124,17 +127,19 @@ </span> </dt> <dd> - <p> - This method optionally configures various timeout values. - Otherwise default timeout values as specified in <a href="../api_reference/C/repset_timeout.html" class="olink">DB_ENV->rep_set_timeout()</a> - are used. In particular, Replication Manager client sites can - be configured to monitor the health of the TCP/IP connection to - the master site using heartbeat messages. If the client - receives no messages from the master for a certain amount of - time, it considers the connection to be broken, and calls for - an election to choose a new master. Heartbeat messages - also help clients request missing master changes in the - absence of master activity. + <p> + This method optionally configures various + timeout values. Otherwise default timeout values + as specified in <a href="../api_reference/C/repset_timeout.html" class="olink">DB_ENV->rep_set_timeout()</a> are used. In + particular, Replication Manager client sites can + be configured to monitor the health of the TCP/IP + connection to the master site using heartbeat + messages. If the client receives no messages from + the master for a certain amount of time, it + considers the connection to be broken, and calls + for an election to choose a new master. Heartbeat + messages also help clients request missing master + changes in the absence of master activity. </p> </dd> <dt> @@ -144,13 +149,14 @@ </dt> <dd> <p> - Once configured and started, Replication Manager does virtually - all of its work in the background, usually without the need for - any direct communication with the application. However, - occasionally events occur which the application may be - interested in knowing about. The application can request - notification of these events by calling the <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> - method. + Once configured and started, Replication + Manager does virtually all of its work in the + background, usually without the need for any + direct communication with the application. + However, occasionally events occur which the + application may be interested in knowing about. + The application can request notification of these + events by calling the <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> method. </p> </dd> <dt> @@ -159,25 +165,40 @@ </span> </dt> <dd> - <p> - The <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a> method starts the replication system. It - opens the listening TCP/IP socket and creates all the - background processing threads that will be needed. + <p> + The <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a> method starts the replication + system. It opens the listening TCP/IP socket and + creates all the background processing threads that + will be needed. </p> </dd> </dl> </div> <p> - In addition to the methods previously described, Replication Manager - applications may also call the following methods, as needed: - <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a>, <a href="../api_reference/C/repset_limit.html" class="olink">DB_ENV->rep_set_limit()</a>, <a href="../api_reference/C/repset_request.html" class="olink">DB_ENV->rep_set_request()</a>, <a href="../api_reference/C/repsync.html" class="olink">DB_ENV->rep_sync()</a> and <a href="../api_reference/C/repstat.html" class="olink">DB_ENV->rep_stat()</a>. + In addition to the methods previously described, + Replication Manager applications may also call the following + methods, as needed: <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a>, <a href="../api_reference/C/repset_limit.html" class="olink">DB_ENV->rep_set_limit()</a>, <a href="../api_reference/C/repset_request.html" class="olink">DB_ENV->rep_set_request()</a>, + <a href="../api_reference/C/repsync.html" class="olink">DB_ENV->rep_sync()</a> and <a href="../api_reference/C/repstat.html" class="olink">DB_ENV->rep_stat()</a>. + </p> + <p> + If a Replication Manager replication group contains only two + sites and one particular site should be master whenever + possible, the replication group can be configured to run in + preferred master mode. For more information, see + <a class="xref" href="rep_twosite.html#twosite_prefmas" title="Preferred master mode">Preferred master mode</a>. + </p> + <p> + Replication Manager applications may configure one or more + view sites. A view is a full or partial copy of the replicated + data that does not otherwise participate in the replication + group. For more information, see <a class="xref" href="rep_partview.html" title="Replication views">Replication views</a>. </p> <p> - Finally, Replication Manager applications can also make use of the - Replication Manager's message channels. This allows the various - sites in the replication group to pass messages that are tailored - to the application's requirements. For more information, see - <a class="xref" href="repmgr_channels.html" title="Using Replication Manager message channels">Using Replication Manager message channels</a>. + Finally, Replication Manager applications can also make use + of the Replication Manager's message channels. This allows the + various sites in the replication group to pass messages that + are tailored to the application's requirements. For more + information, see <a class="xref" href="repmgr_channels.html" title="Using Replication Manager message channels">Using Replication Manager message channels</a>. </p> </div> <div class="navfooter"> @@ -195,7 +216,7 @@ <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Base API Methods</td> + <td width="40%" align="right" valign="top"> Base API methods</td> </tr> </table> </div> diff --git a/docs/programmer_reference/rep_mgrmulti.html b/docs/programmer_reference/rep_mgrmulti.html index 95ce092a..8adf92d1 100644 --- a/docs/programmer_reference/rep_mgrmulti.html +++ b/docs/programmer_reference/rep_mgrmulti.html @@ -8,23 +8,22 @@ <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> <link rel="up" href="rep.html" title="Chapter 12. Berkeley DB Replication" /> - <link rel="prev" href="rep_filename.html" title="Managing Replication Files" /> - <link rel="next" href="rep_replicate.html" title="Running Replication using the db_replicate Utility" /> + <link rel="prev" href="rep_filename.html" title="Managing replication directories and files" /> + <link rel="next" href="rep_replicate.html" title="Running replication using the db_replicate utility" /> </head> <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Running Replication Manager in multiple processes</th> + <th colspan="3" align="center">Running Replication Manager in + multiple processes</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_filename.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_replicate.html">Next</a></td> </tr> </table> @@ -34,7 +33,8 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="rep_mgrmulti"></a>Running Replication Manager in multiple processes</h2> + <h2 class="title" style="clear: both"><a id="rep_mgrmulti"></a>Running Replication Manager in + multiple processes</h2> </div> </div> </div> @@ -42,164 +42,256 @@ <dl> <dt> <span class="sect2"> - <a href="rep_mgrmulti.html#idp2239216">One replication process and multiple subordinate processes</a> + <a href="rep_mgrmulti.html#idp1913552">One replication process and multiple subordinate + processes</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_mgrmulti.html#idp2202400">Persistence of local site network address configuration</a> + <a href="rep_mgrmulti.html#idp1910720">Persistence of local site network address + configuration</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_mgrmulti.html#idp2221464">Programming considerations</a> + <a href="rep_mgrmulti.html#idp1905672">Programming considerations</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_mgrmulti.html#idp2233184">Handling failure</a> + <a href="rep_mgrmulti.html#idp1902040">Handling failure</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_mgrmulti.html#idp2233360">Other miscellaneous rules</a> + <a href="rep_mgrmulti.html#idp1877896">Other miscellaneous rules</a> </span> </dt> </dl> </div> - <p>Replication Manager supports shared access to a database environment -from multiple processes.</p> + <p> + Replication Manager supports shared access to a database + environment from multiple processes or environment handles. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2239216"></a>One replication process and multiple subordinate processes</h3> + <h3 class="title"><a id="idp1913552"></a>One replication process and multiple subordinate + processes</h3> </div> </div> </div> - <p>Each site in a replication group has just one network address (TCP/IP -host name and port number). This means that only one process can -accept incoming connections. At least one application process must -invoke the <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a> method to initiate communications -and management of the replication state.</p> - <p>If it is convenient, multiple processes may issue calls to the -Replication Manager configuration methods, and multiple processes may -call <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a>. Replication Manager automatically opens -the TCP/IP listening socket in the first process to do so (we'll call -it the "replication process" here), and ignores this step in any -subsequent processes ("subordinate processes").</p> + <p> + Each site in a replication group has just one network + address (TCP/IP host name and port number). This means + that only one process can accept incoming connections. At + least one application process must invoke the + <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a> method to initiate communications and + management of the replication state. This is called the + <span class="emphasis"><em>replication process</em></span>. That is, the + replication process is the Replication Manager process + that is responsible for initiating and processing most + replication messages. + </p> + <p> + If it is convenient, multiple processes may issue calls + to the Replication Manager configuration methods, and + multiple processes may call <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a>. Replication + Manager automatically opens the TCP/IP listening socket in + the first process to do so, and so this becomes the + replication process. Replication Manager ignores this step + in any subsequent processes, called <span class="emphasis"><em>subordinate + processes</em></span>. + </p> + <p> + If the replication process quits and there are one or + more subordinate processes available, one subordinate + process will automatically take over as the replication + process. During an automatic takeover on the master site, + the rest of the replication group briefly delays elections + to prevent an unnecessary change of master. Automatic + takeover does not occur if the replication process fails + unexpectedly. Automatic takeover is most reliable when the + <a href="../api_reference/C/repset_timeout.html#set_timeout_DB_REP_ACK_TIMEOUT" class="olink">DB_REP_ACK_TIMEOUT</a> is the same value on all sites in the + replication group. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2202400"></a>Persistence of local site network address configuration</h3> + <h3 class="title"><a id="idp1910720"></a>Persistence of local site network address + configuration</h3> </div> </div> </div> - <p>The local site network address is stored in shared memory, -and remains intact even when (all) processes close their environment -handles gracefully and terminate. A process which opens an -environment handle without running recovery automatically inherits the -existing local site network address configuration. Such a process may not -change the local site address (although it is allowed to redundantly -specify a local site configuration matching that which is already in -effect).</p> - <p>In order to change the local site network address, the application -must run recovery. The application can then specify a new local site -address before restarting Replication Manager. The application should also -remove the old local site address from the replication group if it is no -longer needed.</p> + <p> + The local site network address is stored in shared + memory, and remains intact even when (all) processes close + their environment handles gracefully and terminate. A + process which opens an environment handle without running + recovery automatically inherits the existing local site + network address configuration. Such a process may not + change the local site address (although it is allowed to + redundantly specify a local site configuration matching + that which is already in effect). + </p> + <p> + In order to change the local site network address, the + application must run recovery. The application can then + specify a new local site address before restarting + Replication Manager. The application should also remove + the old local site address from the replication group if + it is no longer needed. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2221464"></a>Programming considerations</h3> + <h3 class="title"><a id="idp1905672"></a>Programming considerations</h3> </div> </div> </div> - <p>Note that Replication Manager applications must follow all the usual -rules for Berkeley DB multi-threaded and/or multi-process -applications, such as ensuring that the recovery operation occurs -single-threaded, only once, before any other thread or processes -operate in the environment. -Since Replication Manager creates its own background threads which -operate on the environment, -all environment handles must be opened with the <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flag, even -if the application is otherwise single-threaded per process.</p> - <p>At the replication master site, each Replication Manager process opens -outgoing TCP/IP connections to all clients in the replication group. -It uses these direct connections to send to clients any log records -resulting from update transactions that the process executes. But all -other replication activity —message processing, elections, -etc.— takes place only in the "replication process".</p> - <p>Replication Manager notifies the application of certain events, using -the callback function configured with the <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> -method. These notifications occur only in the process where the event -itself occurred. Generally this means that most notifications occur -only in the "replication process". Currently the only replication -notification that can occur in a "subordinate process" is -<a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_PERM_FAILED" class="olink">DB_EVENT_REP_PERM_FAILED</a>.</p> - <p>It is not supported for a process running Replication Manager to -spawn a subprocess.</p> + <p> + Note that Replication Manager applications must follow + all the usual rules for Berkeley DB multi-threaded and/or + multi-process applications, such as ensuring that the + recovery operation occurs single-threaded, only once, + before any other thread or processes operate in the + environment. Since Replication Manager creates its own + background threads which operate on the environment, all + environment handles must be opened with the <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> + flag, even if the application is otherwise single-threaded + per process. + </p> + <p> + At the replication master site, each Replication + Manager process opens outgoing TCP/IP connections to all + clients in the replication group. It uses these direct + connections to send to clients any log records resulting + from update transactions that the process executes. But + all other replication activity —message processing, + elections, etc.— takes place only in the + "replication process". + </p> + <p> + Replication Manager notifies the application of certain + events, using the callback function configured with the + <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> method. These notifications occur only + in the process where the event itself occurred. Generally + this means that most notifications occur only in the + replication process. Currently there are only two + replication notification that can occur in a subordinate + process: + </p> + <div class="orderedlist"> + <ol type="1"> + <li> + <p> + <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_PERM_FAILED" class="olink">DB_EVENT_REP_PERM_FAILED</a>. + </p> + </li> + <li> + <p> + <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_AUTOTAKEOVER_FAILED" class="olink">DB_EVENT_REP_AUTOTAKEOVER_FAILED</a> + </p> + </li> + </ol> + </div> + <p> + It is not supported for a process running Replication + Manager to spawn a subprocess. In an application where + a parent process spawns child subprocesses, Replication + Manager can only run in the spawned child subprocesses. + The following operations do not run Replication Manager + and can safely be performed in the parent process: opening + or closing an environment handle, running <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a>, + running recovery. The following operations can implicitly + run a Replication Manager thread and should only be performed + in a child subprocess: opening or closing a database handle, + transactions, gets, puts, checkpoints. + </p> + <p> + If your application is a group of related processes managed + by a single "watcher" process (as described in + <a class="xref" href="transapp_app.html" title="Architecting Transactional Data Store applications">Architecting Transactional Data + Store applications</a>), + the "watcher" process is the parent process and is subject to + the same restrictions. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2233184"></a>Handling failure</h3> + <h3 class="title"><a id="idp1902040"></a>Handling failure</h3> </div> </div> </div> - <p>Multi-process Replication Manager applications should handle failures -in a manner consistent with the rules described in -<a class="xref" href="transapp_fail.html" title="Handling failure in Transactional Data Store applications">Handling failure in Transactional Data Store applications</a>. -To summarize, there are -two ways to handle failure of a process:</p> - <div class="orderedlist"> - <ol type="1"> + <p> + Multi-process Replication Manager applications should + handle failures in a manner consistent with the rules + described in <a class="xref" href="transapp_fail.html" title="Handling failure in Transactional Data Store applications">Handling failure in Transactional Data Store applications</a>. To summarize, there + are two ways to handle failure of a process: + </p> + <div class="itemizedlist"> + <ul type="disc"> <li> <p> - The simple way is to kill all remaining processes, run - recovery, and then restart all processes from the beginning. - But this can be a bit drastic. - </p> + The simple way is to kill all remaining + processes, run recovery, and then restart all + processes from the beginning. But this can be a + bit drastic. + </p> </li> <li> <p> - Using the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method, it is sometimes possible to - leave surviving processes running, and just restart the failed - process. - </p> - <p> - Multi-process Replication Manager applications using this - technique must start a new process when an old process fails. - It is not possible for a "subordinate process" to take over the - duties of a failed "replication process". If the failed - process happens to be the replication process, then after a - failchk() call the next process to call <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a> will - become the new replication process. - </p> + Using the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method, it is sometimes + possible to leave surviving processes running, and + just restart the failed process. + </p> + <p> + Multi-process Replication Manager applications + using this technique must start a new process when + an old process fails. A subordinate process cannot + automatically take over as the replication process + if the previous replication process failed. If the + failed process happens to be the replication + process, then after a + <code class="methodname">failchk()</code> call, the + next process to call <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a> will become the + new replication process. + </p> </li> - </ol> + </ul> </div> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2233360"></a>Other miscellaneous rules</h3> + <h3 class="title"><a id="idp1877896"></a>Other miscellaneous rules</h3> </div> </div> </div> <div class="orderedlist"> <ol type="1"> - <li>A database environment may not be shared between a Replication -Manager application process and a Base API application process.</li> - <li>It is not possible to run multiple Replication Manager processes -during mixed-version live upgrades from Berkeley DB versions prior -to 4.8.</li> + <li> + <p> + A database environment may not be shared + between a Replication Manager application process + and a Base API application process. + </p> + </li> + <li> + <p> + It is not possible to run multiple Replication + Manager processes during mixed-version live + upgrades from Berkeley DB versions prior to 4.8. + </p> + </li> </ol> </div> </div> @@ -215,11 +307,12 @@ to 4.8.</li> <td width="40%" align="right"> <a accesskey="n" href="rep_replicate.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Managing Replication Files </td> + <td width="40%" align="left" valign="top">Managing replication directories and files </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Running Replication using the db_replicate Utility</td> + <td width="40%" align="right" valign="top"> Running replication using the + db_replicate utility</td> </tr> </table> </div> diff --git a/docs/programmer_reference/rep_newsite.html b/docs/programmer_reference/rep_newsite.html index 072cc24f..d9360287 100644 --- a/docs/programmer_reference/rep_newsite.html +++ b/docs/programmer_reference/rep_newsite.html @@ -9,12 +9,12 @@ <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> <link rel="up" href="rep.html" title="Chapter 12. Berkeley DB Replication" /> <link rel="prev" href="rep_comm.html" title="Building the communications infrastructure" /> - <link rel="next" href="group_membership.html" title="Managing Replication Manager Group Membership" /> + <link rel="next" href="group_membership.html" title="Managing Replication Manager group membership" /> </head> <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_comm.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="group_membership.html">Next</a></td> </tr> </table> @@ -38,36 +36,47 @@ </div> </div> </div> - <p>To add a new site to the replication group all that is needed -is for the client member to join. Berkeley DB will perform an -internal initialization from the master to the client automatically -and will run recovery on the client to bring it up to date -with the master.</p> - <p>For Base API applications, connecting to a -new site in the replication group happens whenever the -<a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method returns <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_NEWSITE" class="olink">DB_REP_NEWSITE</a>. The application -should assign the new site a local environment ID number, and all future -messages from the site passed to <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> should include that -environment ID number. It is possible, of course, for the application -to be aware of a new site before the return of <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> (for -example, applications using connection-oriented protocols are likely to -detect new sites immediately, while applications using broadcast -protocols may not).</p> - <p>Regardless, in applications supporting the dynamic addition of database -environments to replication groups, environments joining an existing -replication group may need to provide contact information. (For -example, in an application using TCP/IP sockets, a DNS name or IP -address might be a reasonable value to provide.) This can be done using -the <span class="bold"><strong>cdata</strong></span> parameter to the <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> method. The information -referenced by <span class="bold"><strong>cdata</strong></span> is wrapped in the initial contact message -sent by the new environment, and is provided to the existing members of -the group using the <span class="bold"><strong>rec</strong></span> parameter returned by <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a>. -If no additional information was provided for Berkeley DB to forward to the -existing members of the group, the <span class="bold"><strong>data</strong></span> field of the <span class="bold"><strong>rec</strong></span> -parameter passed to the <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method will be NULL after -<a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> returns <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_NEWSITE" class="olink">DB_REP_NEWSITE</a>.</p> - <p>Replication Manager automatically distributes contact information -using the mechanisms previously described.</p> + <p> + To add a new site to the replication group all that is + needed is for the client member to join. Berkeley DB will + perform an internal initialization from the master to the + client automatically and will run recovery on the client to + bring it up to date with the master. + </p> + <p> + For Base API applications, connecting to a new site in the + replication group happens whenever the <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method + returns <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_NEWSITE" class="olink">DB_REP_NEWSITE</a>. The application should assign the + new site a local environment ID number, and all future + messages from the site passed to <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> should include + that environment ID number. It is possible, of course, for the + application to be aware of a new site before the return of + <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> (for example, applications using + connection-oriented protocols are likely to detect new sites + immediately, while applications using broadcast protocols may + not). + </p> + <p> + Regardless, in applications supporting the dynamic addition + of database environments to replication groups, environments + joining an existing replication group may need to provide + contact information. (For example, in an application using + TCP/IP sockets, a DNS name or IP address might be a reasonable + value to provide.) This can be done using the <span class="bold"><strong>cdata</strong></span> parameter to the <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a> + method. The information referenced by <span class="bold"><strong>cdata</strong></span> + is wrapped in the initial contact + message sent by the new environment, and is provided to the + existing members of the group using the <span class="bold"><strong>rec</strong></span> + parameter returned by <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a>. If no + additional information was provided for Berkeley DB to forward + to the existing members of the group, the <span class="bold"><strong>data</strong></span> field of the <span class="bold"><strong>rec</strong></span> parameter passed to the + <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method will be NULL after <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> returns + <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_NEWSITE" class="olink">DB_REP_NEWSITE</a>. + </p> + <p> + Replication Manager automatically distributes contact + information using the mechanisms previously described. + </p> </div> <div class="navfooter"> <hr /> @@ -84,7 +93,8 @@ using the mechanisms previously described.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Managing Replication Manager Group Membership</td> + <td width="40%" align="right" valign="top"> Managing Replication Manager + group membership</td> </tr> </table> </div> diff --git a/docs/programmer_reference/rep_partition.html b/docs/programmer_reference/rep_partition.html index e5740736..9b662f4b 100644 --- a/docs/programmer_reference/rep_partition.html +++ b/docs/programmer_reference/rep_partition.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_twosite.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_faq.html">Next</a></td> </tr> </table> @@ -38,93 +36,133 @@ </div> </div> </div> - <p>The Berkeley DB replication implementation can be affected by network -partitioning problems.</p> - <p>For example, consider a replication group with N members. The network -partitions with the master on one side and more than N/2 of the sites -on the other side. The sites on the side with the master will continue -forward, and the master will continue to accept write queries for the -databases. Unfortunately, the sites on the other side of the partition, -realizing they no longer have a master, will hold an election. The -election will succeed as there are more than N/2 of the total sites -participating, and there will then be two masters for the replication -group. Since both masters are potentially accepting write queries, the -databases could diverge in incompatible ways.</p> - <p>If multiple masters are ever found to exist in a replication group, a -master detecting the problem will return <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_DUPMASTER" class="olink">DB_REP_DUPMASTER</a>. If -the application sees this return, it should reconfigure itself as a -client (by calling <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a>), and then call for an election -(by calling <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a>). The site that wins the election may be -one of the two previous masters, or it may be another site entirely. -Regardless, the winning system will bring all of the other systems into -conformance.</p> - <p>As another example, consider a replication group with a master -environment and two clients A and B, where client A may upgrade to -master status and client B cannot. Then, assume client A is partitioned -from the other two database environments, and it becomes out-of-date -with respect to the master. Then, assume the master crashes and does -not come back on-line. Subsequently, the network partition is restored, -and clients A and B hold an election. As client B cannot win the -election, client A will win by default, and in order to get back into -sync with client B, possibly committed transactions on client B will be -unrolled until the two sites can once again move forward together.</p> - <p>In both of these examples, there is a phase where a newly elected master -brings the members of a replication group into conformance with itself -so that it can start sending new information to them. This can result -in the loss of information as previously committed transactions are -unrolled.</p> - <p>In architectures where network partitions are an issue, applications -may want to implement a heart-beat protocol to minimize the consequences -of a bad network partition. As long as a master is able to contact at -least half of the sites in the replication group, it is impossible for -there to be two masters. If the master can no longer contact a -sufficient number of systems, it should reconfigure itself as a client, -and hold an election. Replication Manager does not currently -implement such a feature, so this technique is only available to Base API -applications.</p> - <p>There is another tool applications can use to minimize the damage in -the case of a network partition. By specifying an <span class="bold"><strong>nsites</strong></span> -argument to <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> that is larger than the actual number of -database environments in the replication group, Base API applications can keep -systems from declaring themselves the master unless they can talk to -a large percentage of the sites in the system. For example, if there -are 20 database environments in the replication group, and an argument -of 30 is specified to the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method, then a system will have -to be able to talk to at least 16 of the sites to declare itself the -master.</p> - <p>Replication Manager uses the value of <span class="bold"><strong>nsites</strong></span> (configured by -the <a href="../api_reference/C/repnsites.html" class="olink">DB_ENV->rep_set_nsites()</a> method) for elections as well as in calculating how -many acknowledgements to wait for when sending a -<a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a> message. So this technique may be useful here -as well, unless the application uses the <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ALL" class="olink">DB_REPMGR_ACKS_ALL</a> or -<a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_ALL_PEERS" class="olink">DB_REPMGR_ACKS_ALL_PEERS</a> acknowledgement policies.</p> - <p>Specifying a <span class="bold"><strong>nsites</strong></span> argument to <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> that is -smaller than the actual number of database environments in the -replication group has its uses as well. For example, consider a -replication group with 2 environments. If they are partitioned from -each other, neither of the sites could ever get enough votes to become -the master. A reasonable alternative would be to specify a -<span class="bold"><strong>nsites</strong></span> argument of 2 to one of the systems -and a <span class="bold"><strong>nsites</strong></span> -argument of 1 to the other. That way, one of the systems could win -elections even when partitioned, while the other one could not. This -would allow one of the systems to continue accepting write -queries after the partition.</p> - <p>In a 2-site group, Replication Manager by default reacts to the loss of -communication with the master by observing a strict majority rule that -prevents the survivor from taking over. Thus it avoids multiple masters and -the need to unroll some transactions if both sites are running but cannot -communicate. But it does leave the group in a read-only state until both -sites are available. If application availability while one site is down is a -priority and it is acceptable to risk unrolling some transactions, there -is a configuration option to turn off the strict majority rule and allow -the surviving client to declare itself to be master. See the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> -method <a href="../api_reference/C/repconfig.html#config_DB_REPMGR_CONF_2SITE_STRICT" class="olink">DB_REPMGR_CONF_2SITE_STRICT</a> flag for more information.</p> - <p>These scenarios stress the importance of good network infrastructure in -Berkeley DB replicated environments. When replicating database environments -over sufficiently lossy networking, the best solution may well be to -pick a single master, and only hold elections when human intervention -has determined the selected master is unable to recover at all.</p> + <p> + The Berkeley DB replication implementation can be affected + by network partitioning problems. + </p> + <p> + For example, consider a replication group with N members. + The network partitions with the master on one side and more + than N/2 of the sites on the other side. The sites on the side + with the master will continue forward, and the master will + continue to accept write queries for the databases. + Unfortunately, the sites on the other side of the partition, + realizing they no longer have a master, will hold an election. + The election will succeed as there are more than N/2 of the + total sites participating, and there will then be two masters + for the replication group. Since both masters are potentially + accepting write queries, the databases could diverge in + incompatible ways. + </p> + <p> + If multiple masters are ever found to exist in a replication + group, a master detecting the problem will return + <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_DUPMASTER" class="olink">DB_REP_DUPMASTER</a>. Replication Manager applications + automatically handle duplicate master situations. If a Base + API application sees this return, it should reconfigure itself + as a client (by calling <a href="../api_reference/C/repstart.html" class="olink">DB_ENV->rep_start()</a>), and then call for an + election (by calling <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a>). The site that wins the + election may be one of the two previous masters, or it may be + another site entirely. Regardless, the winning system will + bring all of the other systems into conformance. + </p> + <p> + As another example, consider a replication group with a + master environment and two clients A and B, where client A may + upgrade to master status and client B cannot. Then, assume + client A is partitioned from the other two database + environments, and it becomes out-of-date with respect to the + master. Then, assume the master crashes and does not come back + on-line. Subsequently, the network partition is restored, and + clients A and B hold an election. As client B cannot win the + election, client A will win by default, and in order to get + back into sync with client B, possibly committed transactions + on client B will be unrolled until the two sites can once + again move forward together. + </p> + <p> + In both of these examples, there is a phase where a newly + elected master brings the members of a replication group into + conformance with itself so that it can start sending new + information to them. This can result in the loss of + information as previously committed transactions are + unrolled. + </p> + <p> + In architectures where network partitions are an issue, + applications may want to implement a heartbeat protocol to + minimize the consequences of a bad network partition. As long + as a master is able to contact at least half of the sites in + the replication group, it is impossible for there to be two + masters. If the master can no longer contact a sufficient + number of systems, it should reconfigure itself as a client, + and hold an election. Replication Manager does not currently + implement such a feature, so this technique is only available + to Base API applications. + </p> + <p> + There is another tool applications can use to minimize the + damage in the case of a network partition. By specifying an + <span class="bold"><strong>nsites</strong></span> argument to + <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> that is larger than the actual number of database + environments in the replication group, Base API applications + can keep systems from declaring themselves the master unless + they can talk to a large percentage of the sites in the + system. For example, if there are 20 database environments in + the replication group, and an argument of 30 is specified to + the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method, then a system will have to be able to + talk to at least 16 of the sites to declare itself the master. + Replication Manager automatically maintains the number of + sites in the replication group, so this technique is only + available to Base API applications. + </p> + <p> + Specifying a <span class="bold"><strong>nsites</strong></span> + argument to <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> that is smaller than the actual number + of database environments in the replication group has its uses + as well. For example, consider a replication group with 2 + environments. If they are partitioned from each other, neither + of the sites could ever get enough votes to become the master. + A reasonable alternative would be to specify a <span class="bold"><strong>nsites</strong></span> argument of 2 to one of the + systems and a <span class="bold"><strong>nsites</strong></span> argument + of 1 to the other. That way, one of the systems could win + elections even when partitioned, while the other one could + not. This would allow one of the systems to continue accepting + write queries after the partition. + </p> + <p> + In a two-site group, Replication Manager by default reacts to + the loss of communication with the master by observing a + strict majority rule that prevents the survivor from taking + over. Thus it avoids multiple masters and the need to unroll + some transactions if both sites are running but cannot + communicate. But it does leave the group in a read-only state + until both sites are available. If application availability + while one site is down is a priority and it is acceptable to + risk unrolling some transactions, there is a configuration + option to turn off the strict majority rule and allow the + surviving client to declare itself to be master. See the + <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method <a href="../api_reference/C/repconfig.html#config_DB_REPMGR_CONF_2SITE_STRICT" class="olink">DB_REPMGR_CONF_2SITE_STRICT</a> flag for more + information. + </p> + <p> + Preferred master mode is another alternative for two-site + Replication Manager replication groups. It allows the survivor + to take over after the loss of communication with the master. + When communications are restored, it always preserves the + transactions from the preferred master site. See + <a class="xref" href="rep_twosite.html#twosite_prefmas" title="Preferred master mode">Preferred master mode</a> + for more information. + </p> + <p> + These scenarios stress the importance of good network + infrastructure in Berkeley DB replicated environments. When + replicating database environments over sufficiently lossy + networking, the best solution may well be to pick a single + master, and only hold elections when human intervention has + determined the selected master is unable to recover at + all. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/rep_partview.html b/docs/programmer_reference/rep_partview.html new file mode 100644 index 00000000..8da359a7 --- /dev/null +++ b/docs/programmer_reference/rep_partview.html @@ -0,0 +1,169 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> +<html xmlns="http://www.w3.org/1999/xhtml"> + <head> + <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> + <title>Replication views</title> + <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> + <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> + <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> + <link rel="up" href="rep.html" title="Chapter 12. Berkeley DB Replication" /> + <link rel="prev" href="group_membership.html" title="Managing Replication Manager group membership" /> + <link rel="next" href="rep_filename.html" title="Managing replication directories and files" /> + </head> + <body> + <div xmlns="" class="navheader"> + <div class="libver"> + <p>Library Version 12.1.6.1</p> + </div> + <table width="100%" summary="Navigation header"> + <tr> + <th colspan="3" align="center">Replication views</th> + </tr> + <tr> + <td width="20%" align="left"><a accesskey="p" href="group_membership.html">Prev</a> </td> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> + <td width="20%" align="right"> <a accesskey="n" href="rep_filename.html">Next</a></td> + </tr> + </table> + <hr /> + </div> + <div class="sect1" lang="en" xml:lang="en"> + <div class="titlepage"> + <div> + <div> + <h2 class="title" style="clear: both"><a id="rep_partview"></a>Replication views</h2> + </div> + </div> + </div> + <p> + A <span class="bold"><strong>view</strong></span> is a replication + site that maintains a copy of the replication group's data + without incurring other overhead from participating in the + replication group. Views are useful for applications that + require read scalability, are geographically distributed, have + slow network connections to some sites, or require only a + subset of replicated databases at some sites. + </p> + <p> + A client is a <span class="bold"><strong>participant</strong></span> + in the replication group because it helps to determine the + master and contributes to transaction durability. A view + receives its copy of the replicated data without participating + in the replication group in these other ways. Specifically, a + view cannot ever become master, a view does not vote in + elections, and a view does not contribute to transaction + durability. A view is similar to an unelectable client because + neither can become master. They differ because an unelectable + client participates in the replication group by voting and + contributing to transaction durability. + </p> + <p> + A <span class="bold"><strong>full view</strong></span> contains a + copy of all replicated databases. You define a full view by + calling the <a href="../api_reference/C/repset_view.html" class="olink">DB_ENV->rep_set_view()</a> method before opening the + environment and supplying a NULL value for the <span class="bold"><strong>partial_func</strong></span> parameter. You can + then complete any other environment configuration, open the + environment, and eventually start replication using the + <a href="../api_reference/C/repmgrstart.html#repmgrstart_DB_REP_CLIENT" class="olink">DB_REP_CLIENT</a> flags value. + </p> + <p> + A <span class="bold"><strong>partial view</strong></span> contains a + subset of the replicated databases. You define a partial view + using the same steps used to define a full view, except that + you supply a <span class="bold"><strong>partial_func</strong></span> + callback to the <a href="../api_reference/C/repset_view.html" class="olink">DB_ENV->rep_set_view()</a> method. The partial function + uses application-specific logic to determine the names of the + database files to replicate. The partial function should set + its <span class="bold"><strong>result</strong></span> output parameter + to 0 to reject a database file or to a non-zero value to + replicate it. Note that named in-memory databases are always + replicated to partial views regardless of partial function + logic. + </p> + <p> + The decision about whether to replicate a particular + database file to a partial view is made at the time the + database file is first created. It is possible to change the + partial function to replicate a different set of database + files. However, any database file that was already being + replicated will continue to be replicated regardless of the + new partial function logic because it already exists at that + site. + </p> + <p> + Recovery on a partial view should always be performed using + the <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> flag to the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method after calling + the <a href="../api_reference/C/repset_view.html" class="olink">DB_ENV->rep_set_view()</a> method to supply the partial function. You + should not use the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility to recover a partial view + because it cannot use a partial function. + </p> + <p> + Defining a site as a view is a permanent decision; once a + site is defined as a view it can never be transformed into a + participant. This means that on a view site, the <a href="../api_reference/C/repset_view.html" class="olink">DB_ENV->rep_set_view()</a> + method must always be called before opening the environment. + </p> + <p> + An existing participant can be permanently demoted to a + view at any time by calling the <a href="../api_reference/C/repset_view.html" class="olink">DB_ENV->rep_set_view()</a> method before + opening its environment and starting replication. Demoting a + participant to a view can fail when starting replication and + return <code class="literal">DB_REP_UNAVAIL</code> if the master is + unavailable or there are insufficient participants to + acknowledge this operation. You can pause and retry starting + replication until it succeeds. + </p> + <p> + A view cannot become master and does not vote in elections, + but elections among the participant sites need to know the + number of participants in the replication group. Replication + Manager automatically manages elections and tracks the number + of participants. In Base API applications, the number of sites + in the replication group set by the <a href="../api_reference/C/repnsites.html" class="olink">DB_ENV->rep_set_nsites()</a> method should + only count participant sites. Similarly, when a participant + site calls the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method, the <span class="bold"><strong>nsites</strong></span> + value should only count participant + sites. A view should never call the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method. + </p> + <p> + A transaction's durability within a replication group is + based on its replication to other sites that can potentially + take over as master if the current master fails. A view cannot + become master, so it cannot contribute to transaction + durabilty. As a result, Replication Manager views do not send + acknowledgements. Base API applications should not count + messages returning <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_ISPERM" class="olink">DB_REP_ISPERM</a> from a view as contributing + to transaction durability. + </p> + <p> + Replication Manager views participate in replication + group-aware log archiving. A view can also be configured as a + client-to-client peer of another site in the replication + group. If a partial view does not contain the information + requested, Replication Manager will redirect the request to + the master. + </p> + </div> + <div class="navfooter"> + <hr /> + <table width="100%" summary="Navigation footer"> + <tr> + <td width="40%" align="left"><a accesskey="p" href="group_membership.html">Prev</a> </td> + <td width="20%" align="center"> + <a accesskey="u" href="rep.html">Up</a> + </td> + <td width="40%" align="right"> <a accesskey="n" href="rep_filename.html">Next</a></td> + </tr> + <tr> + <td width="40%" align="left" valign="top">Managing Replication Manager + group membership </td> + <td width="20%" align="center"> + <a accesskey="h" href="index.html">Home</a> + </td> + <td width="40%" align="right" valign="top"> Managing replication directories and files</td> + </tr> + </table> + </div> + </body> +</html> diff --git a/docs/programmer_reference/rep_pri.html b/docs/programmer_reference/rep_pri.html index dc58d591..049c6265 100644 --- a/docs/programmer_reference/rep_pri.html +++ b/docs/programmer_reference/rep_pri.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_id.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_app.html">Next</a></td> </tr> </table> @@ -38,25 +36,32 @@ </div> </div> </div> - <p>Each database environment included in a replication group must have -a priority, which specifies a relative ordering among the different -environments in a replication group. This ordering is a factor in -determining which environment will be selected as a new master in case -the existing master fails. Both Replication Manager applications and -Base API applications should specify -environment priorities.</p> - <p>Priorities are an unsigned integer, but do not need to be unique -throughout the replication group. A priority of 0 means the system can -never become a master. Otherwise, larger valued priorities -indicate a more desirable master. For example, if a replication group -consists of three database environments, two of which are connected by -an OC3 and the third of which is connected by a T1, the third database -environment should be assigned a priority value which is lower than -either of the other two.</p> - <p>Desirability of the master is first determined by the client having -the most recent log records. Ties in log records are broken with -the client priority. If both sites have the same log -and the same priority, one is selected at random.</p> + <p> + Each database environment included in a replication group + must have a priority, which specifies a relative ordering + among the different environments in a replication group. This + ordering is a factor in determining which environment will be + selected as a new master in case the existing master fails. + Both Replication Manager applications and Base API + applications should specify environment priorities. + </p> + <p> + Priorities are an unsigned integer, but do not need to be + unique throughout the replication group. A priority of 0 means + the system can never become a master. Otherwise, larger valued + priorities indicate a more desirable master. For example, if a + replication group consists of three database environments, two + of which are connected by an OC3 and the third of which is + connected by a T1, the third database environment should be + assigned a priority value which is lower than either of the + other two. + </p> + <p> + Desirability of the master is first determined by the client + having the most recent log records. Ties in log records are + broken with the client priority. If both sites have the same + log and the same priority, one is selected at random. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/rep_replicate.html b/docs/programmer_reference/rep_replicate.html index 4068e278..dd848b85 100644 --- a/docs/programmer_reference/rep_replicate.html +++ b/docs/programmer_reference/rep_replicate.html @@ -3,28 +3,27 @@ <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> - <title>Running Replication using the db_replicate Utility</title> + <title>Running replication using the db_replicate utility</title> <link rel="stylesheet" href="gettingStarted.css" type="text/css" /> <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> <link rel="up" href="rep.html" title="Chapter 12. Berkeley DB Replication" /> <link rel="prev" href="rep_mgrmulti.html" title="Running Replication Manager in multiple processes" /> - <link rel="next" href="rep_mgr_ack.html" title="Choosing a Replication Manager Ack Policy" /> + <link rel="next" href="rep_mgr_ack.html" title="Choosing a Replication Manager acknowledgement policy" /> </head> <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Running Replication using the db_replicate Utility</th> + <th colspan="3" align="center">Running replication using the + db_replicate utility</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_mgrmulti.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_mgr_ack.html">Next</a></td> </tr> </table> @@ -34,7 +33,8 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="rep_replicate"></a>Running Replication using the db_replicate Utility</h2> + <h2 class="title" style="clear: both"><a id="rep_replicate"></a>Running replication using the + db_replicate utility</h2> </div> </div> </div> @@ -42,125 +42,159 @@ <dl> <dt> <span class="sect2"> - <a href="rep_replicate.html#idp2251336">One Replication Process and Multiple Subordinate Processes</a> + <a href="rep_replicate.html#idp1926936">One replication process and multiple subordinate processes</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_replicate.html#idp2268704">Common Use Case</a> + <a href="rep_replicate.html#idp1906152">Common use case</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_replicate.html#idp2278848">Avoiding Rollback</a> + <a href="rep_replicate.html#idp1939624">Avoiding rollback</a> </span> </dt> <dt> <span class="sect2"> - <a href="rep_replicate.html#idp2283896">When to Consider an Integrated HA Application</a> + <a href="rep_replicate.html#idp1937856">When to consider an integrated replication application</a> </span> </dt> </dl> </div> - <p>Replication Manager supports shared access to a database -environment from multiple processes. Berkeley DB provides a -replication-aware utility, db_replicate, that enables you to upgrade an existing Transactional Data Store -application, as discussed in the <a class="xref" href="transapp.html#transapp_intro" title="Transactional Data Store introduction">Transactional Data Store introduction</a> section, -to an HA application with -minor modifications. While the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility simplifies the use -of replication with a TDS application, you -must still understand replication and its impact on the application. -</p> + <p> + Replication Manager supports shared access to a database + environment from multiple processes. Berkeley DB provides a + replication-aware utility, db_replicate, that enables you to + upgrade an existing Transactional Data Store application, as + discussed in the <a class="xref" href="transapp.html#transapp_intro" title="Transactional Data Store introduction">Transactional Data Store introduction</a> section, to a replication + application with minor modifications. While the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility + simplifies the use of replication with a Transactional Data + Store application, you must still understand replication and + its impact on the application. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2251336"></a>One Replication Process and Multiple Subordinate Processes</h3> + <h3 class="title"><a id="idp1926936"></a>One replication process and multiple subordinate processes</h3> </div> </div> </div> - <p>Based on the terminology introduced in the <a class="xref" href="rep_mgrmulti.html" title="Running Replication Manager in multiple processes">Running Replication Manager in multiple processes</a> section, -application processes are "subordinate processes" and the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility -is the "primary replication process".</p> <p> -You must consider the following items when -planning to use the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility in combination with -a TDS application. -</p> + Based on the terminology introduced in the <a class="xref" href="rep_mgrmulti.html" title="Running Replication Manager in multiple processes">Running Replication Manager in + multiple processes</a> + section, application processes are subordinate processes + and the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility is the replication process. + </p> + <p> + You must consider the following items when planning to + use the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility in combination with a Transactional + Data Store application. + </p> <div class="itemizedlist"> <ul type="disc"> - <li>Memory regions - <p> - The <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility requires shared memory access among - separate processes, and therefore cannot be used with <a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a>. - </p></li> - <li>Multi-process implications - <p> - You must understand and accept all of the TDS implications - of multi-process use as specified in <a class="xref" href="transapp_app.html" title="Architecting Transactional Data Store applications">Architecting Transactional Data Store applications</a>. Special attention should be - paid to the coordination needed for unrelated processes to - start up correctly. - </p></li> - <li>Replication configuration - <p> - Several configuration settings are required for replication. - You must set the <a href="../api_reference/C/envopen.html#envopen_DB_INIT_REP" class="olink">DB_INIT_REP</a> - and <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flags for the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method. - Another required configuration item is the local address. You identify - this by creating a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handle and then setting the - <code class="literal">DB_LOCAL_SITE</code> parameter using the - <a href="../api_reference/C/dbsite_set_config.html" class="olink">DB_SITE->set_config()</a> method. You also tell sites how to contact other - sites by creating <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handles for those sites. - Most replication configuration options start with reasonable - defaults, but applications have to customize - at least some of them. You can set all replication related configuration options - either programmatically or in the <a href="../api_reference/C/configuration_reference.html" class="olink">DB_CONFIG</a> file. - </p></li> - <li> Starting the application and replication - <p> - The <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility assumes that an - environment exists and that the application has run recovery, - if necessary, and created and configured the environment. - The startup flow of a typical TDS application may not be the - best flow for a replication application and you - must understand the issues involved. - For instance, if an application starts, runs recovery, - and performs update operations before starting the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility, - then if that site becomes a client when replication starts, - those update operations will be rolled back. - </p></li> - <li> Handling events - <p> - Almost all of the replication-specific events are handled - by the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility process, and therefore the application - process does not see them. If the application needs to know - the information from those replication-specific events, such as - role changes, the application must call the <a href="../api_reference/C/repstat.html" class="olink">rep_stat()</a> method method. - The one replication-specific event the application can now - receive is the <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_PERM_FAILED" class="olink">DB_EVENT_REP_PERM_FAILED</a> event. - See <a class="xref" href="rep_mgr_ack.html" title="Choosing a Replication Manager Ack Policy">Choosing a Replication Manager Ack Policy</a> - for additional information about this event. - </p></li> - <li> Handling errors - <p> - There are some error return values that relate only to replication. - Specifically, the <code class="literal">DB_REP_HANDLE_DEAD</code> error should now be handled - by the application. Also, if master leases are in use, then the - application also needs to consider the <code class="literal">DB_REP_LEASE_EXPIRED</code> error. - </p></li> - <li> Flexibility tradeoff - <p> - You are giving up flexibility - for the ease of use of the utility. - Application complexity or requirements may eventually - dictate integrating HA calls into the application - over using the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility. </p></li> - <li> Read-only client application - <p> - The application requires additional changes to manage - the read-only status when the application takes on the role of - a client. - </p></li> + <li> + Memory regions + <p> + The <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility + requires shared memory access among separate + processes, and therefore cannot be used with + <a href="../api_reference/C/envopen.html#envopen_DB_PRIVATE" class="olink">DB_PRIVATE</a>. + </p></li> + <li> + Multi-process implications + <p> + You must + understand and accept all of the Transactional + Data Store implications of multi-process use as + specified in <a class="xref" href="transapp_app.html" title="Architecting Transactional Data Store applications">Architecting Transactional Data + Store applications</a>. Special + attention should be paid to the coordination + needed for unrelated processes to start up + correctly. + </p></li> + <li> + Replication configuration + <p> + Several + configuration settings are required for + replication. You must set the <a href="../api_reference/C/envopen.html#envopen_DB_INIT_REP" class="olink">DB_INIT_REP</a> and + <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flags for the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method. + Another required configuration item is the local + address. You identify this by creating a <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> + handle and then setting the + <code class="literal">DB_LOCAL_SITE</code> parameter + using the <a href="../api_reference/C/dbsite_set_config.html" class="olink">DB_SITE->set_config()</a> method. You also + tell sites how to contact other sites by creating + <a href="../api_reference/C/db_site.html" class="olink">DB_SITE</a> handles for those sites. Most replication + configuration options start with reasonable + defaults, but applications have to customize at + least some of them. You can set all replication + related configuration options either + programmatically or in the <a href="../api_reference/C/configuration_reference.html" class="olink">DB_CONFIG</a> file. + </p></li> + <li> + Starting the application and replication + <p> + The <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility assumes that an environment + exists and that the application has run recovery, + if necessary, and created and configured the + environment. The startup flow of a typical + Transactional Data Store application may not be + the best flow for a replication application and + you must understand the issues involved. For + instance, if an application starts, runs recovery, + and performs update operations before starting the + <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility, then if that site becomes a client + when replication starts, those update operations + will be rolled back. + </p></li> + <li> + Handling events + <p> + Almost all of the + replication-specific events are handled by the + <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility process, and therefore the + application process does not see them. If the + application needs to know the information from + those replication-specific events, such as role + changes, the application must call the <a href="../api_reference/C/repstat.html" class="olink">rep_stat()</a> method + method. The one replication-specific event the + application can now receive is the + <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REP_PERM_FAILED" class="olink">DB_EVENT_REP_PERM_FAILED</a> event. See <a class="xref" href="rep_mgr_ack.html" title="Choosing a Replication Manager acknowledgement policy">Choosing a Replication Manager acknowledgement policy</a> for additional + information about this event. + </p></li> + <li> + Handling errors + <p> + There are some error + return values that relate only to replication. + Specifically, the + <code class="literal">DB_REP_HANDLE_DEAD</code> error + should now be handled by the application. Also, if + master leases are in use, then the application + also needs to consider the + <code class="literal">DB_REP_LEASE_EXPIRED</code> error. + </p></li> + <li> + Flexibility tradeoff + <p> + You are giving up + flexibility for the ease of use of the utility. + Application complexity or requirements may + eventually dictate integrating replication calls + into the application instead of using the + <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility. + </p></li> + <li> + Read-only client application + <p> + The + application requires additional changes to manage + the read-only status when the application takes on + the role of a client. + </p></li> </ul> </div> </div> @@ -168,197 +202,218 @@ a TDS application. <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2268704"></a>Common Use Case</h3> + <h3 class="title"><a id="idp1906152"></a>Common use case</h3> </div> </div> </div> <p> -This section lists the steps needed to get replication running for -a common use case of the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility. -The use case presented is an existing TDS application that -already has its environment and databases created and is up and running. -At some point, HA is considered because failover protection -or balancing the read load may now be desired. -</p> + This section lists the steps needed to get replication + running for a common use case of the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility. The + use case presented is an existing Transactional Data Store + application that already has its environment and databases + created and is up and running. At some point, replication + is considered because failover protection or balancing the + read load may now be desired. + </p> <div class="orderedlist"> <ol type="1"> <li> - <p> - To understand the issues involved in a replication/HA application, see - the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility section in the <span class="emphasis"><em>API Reference Guide</em></span>, the <a href="../api_reference/C/rep.html" class="olink">Replication Chapter</a> in the <span class="emphasis"><em> - Programmer's Reference Guide</em></span>, and the source code of the ex_rep_mgr - example program. - </p> + <p> + To understand the issues involved in a + replication application, see the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility + section in the <span class="emphasis"><em>API Reference + Guide</em></span>, the <a href="../api_reference/C/rep.html" class="olink">Replication Chapter</a> in the + <span class="emphasis"><em> Programmer's Reference + Guide</em></span>, and the source code of the + ex_rep_mgr example program. + </p> </li> <li> - <p> - Make a local hot backup of the current - application environment to a new location to use as a testing area. - </p> + <p> + Make a local hot backup of the current + application environment to a new location to use + as a testing area. + </p> </li> <li> - <p> - Add the <a href="../api_reference/C/envopen.html#envopen_DB_INIT_REP" class="olink">DB_INIT_REP</a> and <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flags (if not already - being used) to the application or the <a href="../api_reference/C/configuration_reference.html" class="olink">DB_CONFIG</a> file. - </p> + <p> + Add the <a href="../api_reference/C/envopen.html#envopen_DB_INIT_REP" class="olink">DB_INIT_REP</a> and <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flags (if + not already being used) to the application or the + <a href="../api_reference/C/configuration_reference.html" class="olink">DB_CONFIG</a> file. + </p> </li> <li> <p> - Modify the <a href="../api_reference/C/configuration_reference.html" class="olink">DB_CONFIG</a> file to add the necessary replication - configuration values. At a minimum, the local - host and port information must be added using - the <a href="../api_reference/C/repmgr_site_parameter.html" class="olink">repmgr_site</a> method parameter. - As more sites are added to the group, remote host and port - information can optionally also be added by adding - more <a href="../api_reference/C/repmgr_site_parameter.html" class="olink">repmgr_site</a> method parameters to the <a href="../api_reference/C/configuration_reference.html" class="olink">DB_CONFIG</a> file file. - </p> + Modify the <a href="../api_reference/C/configuration_reference.html" class="olink">DB_CONFIG</a> file to add the necessary + replication configuration values. At a minimum, + the local host and port information must be added + using the <a href="../api_reference/C/repmgr_site_parameter.html" class="olink">repmgr_site</a> method parameter. As + more sites are added to the group, remote host and + port information can optionally also be added by + adding more <a href="../api_reference/C/repmgr_site_parameter.html" class="olink">repmgr_site</a> method parameters to + the <a href="../api_reference/C/configuration_reference.html" class="olink">DB_CONFIG</a> file. + </p> </li> <li> - <p> - Rebuild the application and restart - it in the current testing directory. - </p> + <p> + Rebuild the application and restart it in the + current testing directory. + </p> </li> <li> <p> - Start the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility on the master site with the -M option and - any other options needed such as -h for the home directory. - At this point you have a lone master site running in an - environment with no other replicated sites in the group. - </p> + Start the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility on the master site + with the -M option and any other options needed + such as -h for the home directory. At this point + you have a lone master site running in an + environment with no other replicated sites in the + group. + </p> </li> <li> - <p> - Optionally, prepare to start a client site by performing a - manual hot backup of the running master environment to - initialize a client target directory. While replication - can make its own copy, the hot backup will expedite the - synchronization process. Also, if the application assumes - the existence of a database and the client site is started - without data, the application may have errors or incorrectly - attempt to create the database. - </p> + <p> + Optionally, prepare to start a client site by + performing a manual hot backup of the running + master environment to initialize a client target + directory. While replication can make its own + copy, the hot backup will expedite the + synchronization process. Also, if the application + assumes the existence of a database and the client + site is started without data, the application may + have errors or incorrectly attempt to create the + database. + </p> </li> <li> <p> - Copy the application to the client target. - </p> + Copy the application to the client target. + </p> </li> <li> - <p> - Modify the client environment's <a href="../api_reference/C/configuration_reference.html" class="olink">DB_CONFIG</a> file to set the client's local host and port values and to add - remote site information for the master site and any - other replication configuration choices necessary. - </p> + <p> + Modify the client environment's <a href="../api_reference/C/configuration_reference.html" class="olink">DB_CONFIG</a> file to + set the client's local host and port values and to + add remote site information for the master site + and any other replication configuration choices + necessary. + </p> </li> <li> - <p> - Start the application on the client. The client application - should not update data at this point, as explained previously. - </p> + <p> + Start the application on the client. The client + application should not update data at this point, + as explained previously. + </p> </li> <li> - <p> - Start the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility specifying the - client environment's home directory using the -h option. Omit the -M - option in this case, because the utility defaults to - starting in the client role. - </p> + <p> + Start the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility specifying the client + environment's home directory using the -h option. + Omit the -M option in this case, because the + utility defaults to starting in the client role. + </p> </li> </ol> </div> - <p> -Once the initial replication group is established, do not use the -M option with -the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility. -After the initial start, <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility assumes the use of -elections. If a site crashes, it should rejoin the group as -a client so that it can synchronize with the rest of the group. -</p> + <p> + Once the initial replication group is established, do + not use the -M option with the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility. After the + initial start, the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility assumes the use of + elections. If a site crashes, it should rejoin the group + as a client so that it can synchronize with the rest of + the group. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2278848"></a>Avoiding Rollback</h3> + <h3 class="title"><a id="idp1939624"></a>Avoiding rollback</h3> </div> </div> </div> + <p> + Depending on how an application is structured, + transactional rollback can occur. If this is possible, + then you must make application changes or be prepared for + successful transactions to disappear. Consider a common + program flow where the application first creates and opens + the environment with recovery. Then, immediately after + that, the application opens up the databases it expects to + use. Often an application will use the <a href="../api_reference/C/dbopen.html#open_DB_CREATE" class="olink">DB_CREATE</a> flag so + that if the database does not exist it is created, + otherwise the existing one is used automatically. Then the + application begins servicing transactions to write and + read data. + </p> <p> -Depending on how an application is structured, transactional rollback -can occur. If this is possible, then you must make application -changes or be prepared for successful transactions to disappear. -Consider a common program flow where the application first creates -and opens the environment with recovery. Then, immediately after -that, the application opens up the databases it expects to use. -Often an application will use the <a href="../api_reference/C/dbopen.html#open_DB_CREATE" class="olink">DB_CREATE</a> flag so that if the -database does not exist it is created, otherwise the existing one is -used automatically. Then the application begins servicing transactions -to write and read data. -</p> + When replication is introduced, particularly via the + <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility, the possibility of rollback exists unless + the application takes steps to prevent it. In the + situation described above, if all of the above steps occur + before the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility process starts, and the site is + started as a client, then all the operations will be + rolled back when the site finds the master. The client + site will synchronize with the log and operations on the + master site, so any operations that occurred in the client + application before it knew it was a client will be + discarded. + </p> <p> -When replication is introduced, particularly via the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility, the -possibility of rollback exists unless the application takes steps -to prevent it. In the situation described above, if all of the -above steps occur before the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility process starts, and -the site is started as a client, then all the operations will be -rolled back when the site finds the master. The client site will -synchronize with the log and operations on the master site, so -any operations that occurred in the client application before it -knew it was a client will be discarded. -</p> - <p> -One way to reduce the possibility of rollback is to modify the -application so that it only performs update operations (including -creation of a database) if it is the master site. If the application -refrains from updating until it is the master, then it will not perform -operations when it is in the undefined state before replication -has been started. -The event indicating a site is master will be delivered to the -<a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility process, so the application process must look -for that information via the <a href="../api_reference/C/repstat.html" class="olink">rep_stat()</a> method. -A site that is expecting to perform updates may need to poll -via the <a href="../api_reference/C/repstat.html" class="olink">rep_stat()</a> method to see the state change from an undefined -role to either the master or client role. Similarly, since a -client site cannot create a database, it may need to poll for -the database's existence while the client synchronizes with the master -until the database is created at the client site. -</p> + One way to reduce the possibility of rollback is to + modify the application so that it only performs update + operations (including creation of a database) if it is the + master site. If the application refrains from updating + until it is the master, then it will not perform + operations when it is in the undefined state before + replication has been started. The event indicating a site + is master will be delivered to the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility process, + so the application process must look for that information + via the <a href="../api_reference/C/repstat.html" class="olink">rep_stat()</a> method. A site that is expecting to perform + updates may need to poll via the <a href="../api_reference/C/repstat.html" class="olink">rep_stat()</a> method to see the + state change from an undefined role to either the master + or client role. Similarly, since a client site cannot + create a database, it may need to poll for the database's + existence while the client synchronizes with the master + until the database is created at the client site. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2283896"></a>When to Consider an Integrated HA Application</h3> + <h3 class="title"><a id="idp1937856"></a>When to consider an integrated replication application</h3> </div> </div> </div> + <p> + The <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility provides the means to achieve a + replicated application quickly. However, the trade-off for + this rapid implementation is that the full flexibility of + replication is not available. Some applications may + eventually need to consider integrating directly with + replication rather than using the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility if + greater flexibility is desired. + </p> <p> -The <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility provides the means to achieve -a replicated application quickly. However, -the trade-off for this rapid implementation is that the full -flexibility of replication is not available. Some applications -may eventually need to consider integrating directly with -replication rather than using the <a href="../api_reference/C/db_replicate.html" class="olink">db_replicate</a> utility if greater flexibility is desired. -</p> - <p> -One likely reason for considering integration would be the -convenience of receiving all replication-related events in -the application process and gaining direct knowledge of such -things as role changes. -Using the event callback is cleaner and easier than -polling for state changes via the <a href="../api_reference/C/repstat.html" class="olink">rep_stat()</a> method. -</p> + One likely reason for considering integration would be + the convenience of receiving all replication-related + events in the application process and gaining direct + knowledge of such things as role changes. Using the event + callback is cleaner and easier than polling for state + changes via the <a href="../api_reference/C/repstat.html" class="olink">rep_stat()</a> method. + </p> <p> -A second likely reason for integrating replication directly -into the application is the multi-process aspect of the -utility program. The developer may find it easier to insert -the start of replication directly into the code once the -environment is created, recovered, or opened, and avoid -the scenario where the application is running in the -undefined state. Also it may simply be easier to start -the application once than to coordinate different processes -and their startup order in the system. -</p> + A second likely reason for integrating replication + directly into the application is the multi-process aspect + of the utility program. The developer may find it easier + to insert the start of replication directly into the code + once the environment is created, recovered, or opened, and + avoid the scenario where the application is running in the + undefined state. Also it may simply be easier to start the + application once than to coordinate different processes + and their startup order in the system. + </p> </div> </div> <div class="navfooter"> @@ -372,11 +427,12 @@ and their startup order in the system. <td width="40%" align="right"> <a accesskey="n" href="rep_mgr_ack.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Running Replication Manager in multiple processes </td> + <td width="40%" align="left" valign="top">Running Replication Manager in + multiple processes </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Choosing a Replication Manager Ack Policy</td> + <td width="40%" align="right" valign="top"> Choosing a Replication Manager acknowledgement policy</td> </tr> </table> </div> diff --git a/docs/programmer_reference/rep_ryw.html b/docs/programmer_reference/rep_ryw.html index 938738ab..93cba07b 100644 --- a/docs/programmer_reference/rep_ryw.html +++ b/docs/programmer_reference/rep_ryw.html @@ -8,13 +8,13 @@ <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> <link rel="up" href="rep.html" title="Chapter 12. Berkeley DB Replication" /> - <link rel="prev" href="rep_lease.html" title="Master Leases" /> - <link rel="next" href="rep_clock_skew.html" title="Clock Skew" /> + <link rel="prev" href="rep_lease.html" title="Master leases" /> + <link rel="next" href="rep_clock_skew.html" title="Clock skew" /> </head> <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_lease.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_clock_skew.html">Next</a></td> </tr> </table> @@ -57,44 +55,46 @@ </dt> </dl> </div> - <p> - Some applications require the ability to read replicated data at a - client site, and determine whether it is consistent with data that - has been written previously at the master site. + <p> + Some applications require the ability to read replicated + data at a client site, and determine whether it is consistent + with data that has been written previously at the master site. </p> - <p> - For example, a web application may be backed by multiple database - environments, linked to form a replication group, in order to share - the workload. Web requests that update data must be served by the - replication master, but any site in the group may serve a read-only - request. Consider a work flow of a series of web requests from one - specific user at a web browser: the first request generates a - database update, but the second request merely reads data. If the - read-only request is served by a replication client database - environment, it may be important to make sure that the updated data - has been replicated to the client before performing the read (or to wait - until it has been replicated) in order to show this user a - consistent view of the data. + <p> + For example, a web application may be backed by multiple + database environments, linked to form a replication group, in + order to share the workload. Web requests that update data + must be served by the replication master, but any site in the + group may serve a read-only request. Consider a work flow of a + series of web requests from one specific user at a web + browser: the first request generates a database update, but + the second request merely reads data. If the read-only request + is served by a replication client database environment, it may + be important to make sure that the updated data has been + replicated to the client before performing the read (or to + wait until it has been replicated) in order to show this user + a consistent view of the data. </p> - <p> - Berkeley DB supports this requirement through the use of transaction - "tokens". A token is a form of identification for a transaction - within the scope of the replication group. The application may - request a copy of the transaction's token at the master site during - the execution of the transaction. Later, the application running - on a client site can use a copy of the token to determine whether - the transaction has been applied at that site. + <p> + Berkeley DB supports this requirement through the use of + transaction "tokens". A token is a form of identification for + a transaction within the scope of the replication group. The + application may request a copy of the transaction's token at + the master site during the execution of the transaction. + Later, the application running on a client site can use a copy + of the token to determine whether the transaction has been + applied at that site. </p> - <p> - It is the application's responsibility to keep track of the token - during the interim. In the web example, the token might be sent to - the browser as a "cookie", or stored on the application server in - the user's session context. + <p> + It is the application's responsibility to keep track of the + token during the interim. In the web example, the token might + be sent to the browser as a "cookie", or stored on the + application server in the user's session context. </p> <p> - The operations described here are supported both for Replication - Manager applications and for applications that use the replication - Base API. + The operations described here are supported both for + Replication Manager applications and for applications that use + the replication Base API. </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> @@ -105,20 +105,20 @@ </div> </div> <p> - In order to get a token, the application must supply a small - memory buffer, using the <a href="../api_reference/C/txnset_commit_token.html" class="olink">DB_TXN->set_commit_token()</a> method. - </p> + In order to get a token, the application must supply a + small memory buffer, using the <a href="../api_reference/C/txnset_commit_token.html" class="olink">DB_TXN->set_commit_token()</a> method. </p> <p> - Note that a token is generated only upon a successful commit - operation, and therefore the token buffer content is valid - only after a successful commit. Also, if a transaction does - not perform any update operations it does not generate a useful - token. + Note that a token is generated only upon a successful + commit operation, and therefore the token buffer content + is valid only after a successful commit. Also, if a + transaction does not perform any update operations it does + not generate a useful token. </p> - <p> - In the Berkeley DB Java and C# API, getting a token is simpler. - The application need only invoke the <a class="ulink" href="../java/com/sleepycat/db/Transaction.html#getCommitToken()" target="_top">Transaction.getCommitToken()</a> method, - after the transaction has committed. + <p> + In the Berkeley DB Java and C# API, getting a token is + simpler. The application need only invoke the + <a class="ulink" href="../java/com/sleepycat/db/Transaction.html#getCommitToken()" target="_top">Transaction.getCommitToken()</a> method, after the transaction has + committed. </p> </div> <div class="sect2" lang="en" xml:lang="en"> @@ -129,18 +129,18 @@ </div> </div> </div> - <p> - The application should not try to interpret the content of the - token buffer, but may store and/or transmit it freely between - systems. However, since the buffer contains binary data it - may be necessary to apply some encoding for transmission (e.g., - base 64). + <p> + The application should not try to interpret the content + of the token buffer, but may store and/or transmit it + freely between systems. However, since the buffer contains + binary data it may be necessary to apply some encoding for + transmission (e.g., base 64). </p> - <p> - The data is resilient to differences in byte order between - different systems. It does not expire: it may be retained - indefinitely for later use, even across Berkeley DB version - upgrades. + <p> + The data is resilient to differences in byte order + between different systems. It does not expire: it may be + retained indefinitely for later use, even across Berkeley + DB version upgrades. </p> </div> <div class="sect2" lang="en" xml:lang="en"> @@ -151,43 +151,46 @@ </div> </div> </div> - <p> + <p> The <a href="../api_reference/C/envtxn_applied.html" class="olink">DB_ENV->txn_applied()</a> method takes a copy of a token, and - determines whether the corresponding transaction is currently - applied at the local site. The timeout argument allows the - application to block for a bounded amount of time for cases - where the transaction has not yet been applied. + determines whether the corresponding transaction is + currently applied at the local site. The timeout argument + allows the application to block for a bounded amount of + time for cases where the transaction has not yet been + applied. </p> <p> - Depending on the transaction durability levels implemented or - configured by the application, it is sometimes possible for a - transaction to disappear from a replication group if an - original master site fails and a different site becomes the new - master without having received the transaction. When the - <a href="../api_reference/C/envtxn_applied.html" class="olink">DB_ENV->txn_applied()</a> method discovers this, it produces the + Depending on the transaction durability levels + implemented or configured by the application, it is + sometimes possible for a transaction to disappear from a + replication group if an original master site fails and a + different site becomes the new master without having + received the transaction. When the <a href="../api_reference/C/envtxn_applied.html" class="olink">DB_ENV->txn_applied()</a> method + discovers this, it produces the <code class="literal">DB_NOTFOUND</code> return code. </p> <p> - This means that the results of <a href="../api_reference/C/envtxn_applied.html" class="olink">DB_ENV->txn_applied()</a> are not guaranteed - forever. Even after a successful call to <a href="../api_reference/C/envtxn_applied.html" class="olink">DB_ENV->txn_applied()</a>, it is - possible that by the time the application tries to read the - data, the transaction and its data could have disappeared. + This means that the results of <a href="../api_reference/C/envtxn_applied.html" class="olink">DB_ENV->txn_applied()</a> are not + guaranteed forever. Even after a successful call to + <a href="../api_reference/C/envtxn_applied.html" class="olink">DB_ENV->txn_applied()</a>, it is possible that by the time the + application tries to read the data, the transaction and + its data could have disappeared. </p> <p> - To avoid this problem the application should do the read - operations in the context of a transaction, and hold the - transaction handle open during the <a href="../api_reference/C/envtxn_applied.html" class="olink">DB_ENV->txn_applied()</a> call. The - <a href="../api_reference/C/envtxn_applied.html" class="olink">DB_ENV->txn_applied()</a> method itself does not actually execute in the - context of the transaction; but no rollbacks due to new master - synchronization ever occur while a transaction is active, even - a read-only transaction at a client site. + To avoid this problem the application should do the + read operations in the context of a transaction, and hold + the transaction handle open during the <a href="../api_reference/C/envtxn_applied.html" class="olink">DB_ENV->txn_applied()</a> call. + The <a href="../api_reference/C/envtxn_applied.html" class="olink">DB_ENV->txn_applied()</a> method itself does not actually execute + in the context of the transaction; but no rollbacks due to + new master synchronization ever occur while a transaction + is active, even a read-only transaction at a client site. </p> <p> Note that the <a href="../api_reference/C/envtxn_applied.html" class="olink">DB_ENV->txn_applied()</a> method can return - <code class="literal">DB_LOCK_DEADLOCK</code>. The application should - respond to this situation just as it does for any other normal - operation: abort any existing transaction, and then pause - briefly before retrying. + <code class="literal">DB_LOCK_DEADLOCK</code>. The application + should respond to this situation just as it does for any + other normal operation: abort any existing transaction, + and then pause briefly before retrying. </p> </div> </div> @@ -202,11 +205,11 @@ <td width="40%" align="right"> <a accesskey="n" href="rep_clock_skew.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Master Leases </td> + <td width="40%" align="left" valign="top">Master leases </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Clock Skew</td> + <td width="40%" align="right" valign="top"> Clock skew</td> </tr> </table> </div> diff --git a/docs/programmer_reference/rep_trans.html b/docs/programmer_reference/rep_trans.html index 059ad396..91c2a827 100644 --- a/docs/programmer_reference/rep_trans.html +++ b/docs/programmer_reference/rep_trans.html @@ -9,12 +9,12 @@ <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> <link rel="up" href="rep.html" title="Chapter 12. Berkeley DB Replication" /> <link rel="prev" href="rep_bulk.html" title="Bulk transfer" /> - <link rel="next" href="rep_lease.html" title="Master Leases" /> + <link rel="next" href="rep_lease.html" title="Master leases" /> </head> <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_bulk.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_lease.html">Next</a></td> </tr> </table> @@ -38,269 +36,350 @@ </div> </div> </div> - <p>It is important to consider replication in the context of the overall -database environment's transactional guarantees. To briefly review, -transactional guarantees in a non-replicated application are based on -the writing of log file records to "stable storage", usually a disk -drive. If the application or system then fails, the Berkeley DB logging -information is reviewed during recovery, and the databases are updated -so that all changes made as part of committed transactions appear, and -all changes made as part of uncommitted transactions do not appear. In -this case, no information will have been lost.</p> - <p>If a database environment does not require the log be flushed to -stable storage on transaction commit (using the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> -flag to increase performance at the cost of sacrificing transactional -durability), Berkeley DB recovery will only be able to restore the system to -the state of the last commit found on stable storage. In this case, -information may have been lost (for example, the changes made by some -committed transactions may not appear in the databases after recovery).</p> - <p>Further, if there is database or log file loss or corruption (for -example, if a disk drive fails), then catastrophic recovery is -necessary, and Berkeley DB recovery will only be able to restore the system -to the state of the last archived log file. In this case, information -may also have been lost.</p> - <p>Replicating the database environment extends this model, by adding a -new component to "stable storage": the client's replicated information. -If a database environment is replicated, there is no lost information -in the case of database or log file loss, because the replicated system -can be configured to contain a complete set of databases and log records -up to the point of failure. A database environment that loses a disk -drive can have the drive replaced, and it can then rejoin the -replication group.</p> - <p>Because of this new component of stable storage, specifying -<a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> in a replicated environment no longer sacrifices -durability, as long as one or more clients have acknowledged receipt of -the messages sent by the master. Since network connections are often -faster than local synchronous disk writes, replication becomes a way -for applications to significantly improve their performance as well as -their reliability.</p> - <p>The return status from the application's <span class="bold"><strong>send</strong></span> function must be -set by the application to ensure the transactional guarantees the -application wants to provide. Whenever the <span class="bold"><strong>send</strong></span> function -returns failure, the local database environment's log is flushed as -necessary to ensure that any information critical to database integrity -is not lost. Because this flush is an expensive operation in terms of -database performance, applications should avoid returning an error from -the <span class="bold"><strong>send</strong></span> function, if at all possible.</p> - <p>The only interesting message type for replication transactional -guarantees is when the application's <span class="bold"><strong>send</strong></span> function was called -with the <a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a> flag specified. There is no reason -for the <span class="bold"><strong>send</strong></span> function to ever return failure unless the -<a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a> flag was specified -- messages without the -<a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a> flag do not make visible changes to databases, -and the <span class="bold"><strong>send</strong></span> function can return success to Berkeley DB as soon as -the message has been sent to the client(s) or even just copied to local -application memory in preparation for being sent.</p> - <p>When a client receives a <a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a> message, the client -will flush its log to stable storage before returning (unless the client -environment has been configured with the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> option). -If the client is unable to flush a complete transactional record to disk -for any reason (for example, there is a missing log record before the -flagged message), the call to the <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method on the client -will return <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_NOTPERM" class="olink">DB_REP_NOTPERM</a> and return the LSN of this record -to the application in the <span class="bold"><strong>ret_lsnp</strong></span> parameter. -The application's client or master -message handling loops should take proper action to ensure the correct -transactional guarantees in this case. When missing records arrive -and allow subsequent processing of previously stored permanent -records, the call to the <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method on the client will -return <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_ISPERM" class="olink">DB_REP_ISPERM</a> and return the largest LSN of the -permanent records that were flushed to disk. Client applications -can use these LSNs to know definitively if any particular LSN is -permanently stored or not.</p> - <p>An application relying on a client's ability to become a master and -guarantee that no data has been lost will need to write the <span class="bold"><strong>send</strong></span> -function to return an error whenever it cannot guarantee the site that -will win the next election has the record. Applications not requiring -this level of transactional guarantees need not have the <span class="bold"><strong>send</strong></span> -function return failure (unless the master's database environment has -been configured with <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a>), as any information critical -to database integrity has already been flushed to the local log before -<span class="bold"><strong>send</strong></span> was called.</p> - <p>To sum up, the only reason for the <span class="bold"><strong>send</strong></span> function to return -failure is when the master database environment has been configured to -not synchronously flush the log on transaction commit (that is, -<a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> was configured on the master), the -<a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a> flag is specified for the message, and the -<span class="bold"><strong>send</strong></span> function was unable to determine that some number of -clients have received the current message (and all messages preceding -the current message). How many clients need to receive the message -before the <span class="bold"><strong>send</strong></span> function can return success is an application -choice (and may not depend as much on a specific number of clients -reporting success as one or more geographically distributed clients).</p> - <p>If, however, the application does require on-disk durability on the master, -the master should be configured to synchronously flush the log on commit. -If clients are not configured to synchronously flush the log, -that is, if a client is running with <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> configured, -then it is up to the application to reconfigure that client -appropriately when it becomes a master. That is, the -application must explicitly call <a href="../api_reference/C/envset_flags.html" class="olink">DB_ENV->set_flags()</a> to -disable asynchronous log flushing as part of re-configuring -the client as the new master.</p> - <p>Of course, it is important to ensure that the replicated master and -client environments are truly independent of each other. For example, -it does not help matters that a client has acknowledged receipt of a -message if both master and clients are on the same power supply, as the -failure of the power supply will still potentially lose information.</p> - <p>Configuring your replication-based application to achieve the proper -mix of performance and transactional guarantees can be complex. In -brief, there are a few controls an application can set to configure the -guarantees it makes: specification of <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> for the -master environment, specification of <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> for the -client environment, the priorities of different sites participating in -an election, and the behavior of the application's <span class="bold"><strong>send</strong></span> -function.</p> - <p>Applications using Replication Manager are free to use -<a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> at the master and/or clients as they see fit. The -behavior of the <span class="bold"><strong>send</strong></span> function that Replication Manager provides -on the application's behalf is determined by an "acknowledgement -policy", which is configured by the <a href="../api_reference/C/repmgrset_ack_policy.html" class="olink">DB_ENV->repmgr_set_ack_policy()</a> method. -Clients always send acknowledgements for <a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a> -messages (unless the acknowledgement policy in effect indicates that the -master doesn't care about them). For a <a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a> -message, the master blocks the sending thread until either it receives -the proper number of acknowledgements, or the <a href="../api_reference/C/repset_timeout.html#set_timeout_DB_REP_ACK_TIMEOUT" class="olink">DB_REP_ACK_TIMEOUT</a> -expires. In the case of timeout, Replication Manager returns an error -code from the <span class="bold"><strong>send</strong></span> function, causing Berkeley DB to flush the -transaction log before returning to the application, as previously -described. The default acknowledgement policy is -<a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_QUORUM" class="olink">DB_REPMGR_ACKS_QUORUM</a>, which ensures that the effect of a -permanent record remains durable following an election.</p> - <p>First, it is rarely useful to write and synchronously flush the log when -a transaction commits on a replication client. It may be useful where -systems share resources and multiple systems commonly fail at the same -time. By default, all Berkeley DB database environments, whether master or -client, synchronously flush the log on transaction commit or prepare. -Generally, replication masters and clients turn log flush off for -transaction commit using the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> flag.</p> - <p>Consider two systems connected by a network interface. One acts as the -master, the other as a read-only client. The client takes over as -master if the master crashes and the master rejoins the replication -group after such a failure. Both master and client are configured to -not synchronously flush the log on transaction commit (that is, -<a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> was configured on both systems). The -application's <span class="bold"><strong>send</strong></span> function never returns failure to the Berkeley DB -library, simply forwarding messages to the client (perhaps over a -broadcast mechanism), and always returning success. On the client, any -<a href="../api_reference/C/repmessage.html#repmsg_DB_REP_NOTPERM" class="olink">DB_REP_NOTPERM</a> returns from the client's <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method -are ignored, as well. This system configuration has excellent -performance, but may lose data in some failure modes.</p> - <p>If both the master and the client crash at once, it is possible to lose -committed transactions, that is, transactional durability is not being -maintained. Reliability can be increased by providing separate power -supplies for the systems and placing them in separate physical locations.</p> - <p>If the connection between the two machines fails (or just some number -of messages are lost), and subsequently the master crashes, it is -possible to lose committed transactions. Again, transactional -durability is not being maintained. Reliability can be improved in a -couple of ways:</p> + <p> + It is important to consider replication in the context of + the overall database environment's transactional guarantees. + To briefly review, transactional guarantees in a + non-replicated application are based on the writing of log + file records to "stable storage", usually a disk drive. If the + application or system then fails, the Berkeley DB logging + information is reviewed during recovery, and the databases are + updated so that all changes made as part of committed + transactions appear, and all changes made as part of + uncommitted transactions do not appear. In this case, no + information will have been lost. + </p> + <p> + If a database environment does not require the log be + flushed to stable storage on transaction commit (using the + <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> flag to increase performance at the cost of + sacrificing transactional durability), Berkeley DB recovery + will only be able to restore the system to the state of the + last commit found on stable storage. In this case, information + may have been lost (for example, the changes made by some + committed transactions may not appear in the databases after + recovery). + </p> + <p> + Further, if there is database or log file loss or corruption + (for example, if a disk drive fails), then catastrophic + recovery is necessary, and Berkeley DB recovery will only be + able to restore the system to the state of the last archived + log file. In this case, information may also have been + lost. + </p> + <p> + Replicating the database environment extends this model, by + adding a new component to "stable storage": the client's + replicated information. If a database environment is + replicated, there is no lost information in the case of + database or log file loss, because the replicated system can + be configured to contain a complete set of databases and log + records up to the point of failure. A database environment + that loses a disk drive can have the drive replaced, and it + can then rejoin the replication group. + </p> + <p> + Because of this new component of stable storage, specifying + <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> in a replicated environment no longer + sacrifices durability, as long as one or more clients have + acknowledged receipt of the messages sent by the master. Since + network connections are often faster than local synchronous + disk writes, replication becomes a way for applications to + significantly improve their performance as well as their + reliability. + </p> + <p> + Applications using Replication Manager are free to use + <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> at the master and/or clients as they see fit. + The behavior of the <span class="bold"><strong>send</strong></span> + function that Replication Manager provides on the + application's behalf is determined by an "acknowledgement + policy", which is configured by the <a href="../api_reference/C/repmgrset_ack_policy.html" class="olink">DB_ENV->repmgr_set_ack_policy()</a> method. + Clients always send acknowledgements for <a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a> + messages (unless the acknowledgement policy in effect + indicates that the master doesn't care about them). For a + <a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a> message, the master blocks the sending + thread until either it receives the proper number of + acknowledgements, or the <a href="../api_reference/C/repset_timeout.html#set_timeout_DB_REP_ACK_TIMEOUT" class="olink">DB_REP_ACK_TIMEOUT</a> expires. In the + case of timeout, Replication Manager returns an error code + from the <span class="bold"><strong>send</strong></span> function, + causing Berkeley DB to flush the transaction log before + returning to the application, as previously described. The + default acknowledgement policy is <a href="../api_reference/C/repmgrset_ack_policy.html#ackspolicy_DB_REPMGR_ACKS_QUORUM" class="olink">DB_REPMGR_ACKS_QUORUM</a>, + which ensures that the effect of a permanent record remains + durable following an election. + </p> + <p> + The rest of this section discusses transactional guarantee + considerations in Base API applications. + </p> + <p> + The return status from the application's <span class="bold"><strong>send</strong></span> function must be set by the + application to ensure the transactional guarantees the + application wants to provide. Whenever the <span class="bold"><strong>send</strong></span> function returns failure, the + local database environment's log is flushed as necessary to + ensure that any information critical to database integrity is + not lost. Because this flush is an expensive operation in + terms of database performance, applications should avoid + returning an error from the <span class="bold"><strong>send</strong></span> function, + if at all possible. + </p> + <p> + The only interesting message type for replication + transactional guarantees is when the application's <span class="bold"><strong>send</strong></span> function was called with the + <a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a> flag specified. There is no reason for the + <span class="bold"><strong>send</strong></span> function to ever + return failure unless the <a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a> flag was + specified -- messages without the <a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a> flag do + not make visible changes to databases, and the <span class="bold"><strong>send</strong></span> function can return success to + Berkeley DB as soon as the message has been sent to the + client(s) or even just copied to local application memory in + preparation for being sent. + </p> + <p> + When a client receives a <a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a> message, the + client will flush its log to stable storage before returning + (unless the client environment has been configured with the + <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> option). If the client is unable to flush a + complete transactional record to disk for any reason (for + example, there is a missing log record before the flagged + message), the call to the <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method on the client + will return <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_NOTPERM" class="olink">DB_REP_NOTPERM</a> and return the LSN of this record + to the application in the <span class="bold"><strong>ret_lsnp</strong></span> parameter. + The application's client or master message handling loops should take proper + action to ensure the correct transactional guarantees in this case. When + missing records arrive and allow subsequent processing of + previously stored permanent records, the call to the + <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method on the client will return <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_ISPERM" class="olink">DB_REP_ISPERM</a> + and return the largest LSN of the permanent records that were + flushed to disk. Client applications can use these LSNs to + know definitively if any particular LSN is permanently stored + or not. + </p> + <p> + An application relying on a client's ability to become a + master and guarantee that no data has been lost will need to + write the <span class="bold"><strong>send</strong></span> function to + return an error whenever it cannot guarantee the site that + will win the next election has the record. Applications not + requiring this level of transactional guarantees need not have + the <span class="bold"><strong>send</strong></span> function return + failure (unless the master's database environment has been + configured with <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a>), as any information critical + to database integrity has already been flushed to the local + log before <span class="bold"><strong>send</strong></span> was + called. + </p> + <p> + To sum up, the only reason for the <span class="bold"><strong>send</strong></span> function + to return failure is when the + master database environment has been configured to not + synchronously flush the log on transaction commit (that is, + <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> was configured on the master), the + <a href="../api_reference/C/reptransport.html#transport_DB_REP_PERMANENT" class="olink">DB_REP_PERMANENT</a> flag is specified for the message, and the + <span class="bold"><strong>send</strong></span> function was unable + to determine that some number of clients have received the + current message (and all messages preceding the current + message). How many clients need to receive the message before + the <span class="bold"><strong>send</strong></span> function can return + success is an application choice (and may not depend as much + on a specific number of clients reporting success as one or + more geographically distributed clients). + </p> + <p> + If, however, the application does require on-disk durability + on the master, the master should be configured to + synchronously flush the log on commit. If clients are not + configured to synchronously flush the log, that is, if a + client is running with <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> configured, then it is + up to the application to reconfigure that client appropriately + when it becomes a master. That is, the application must + explicitly call <a href="../api_reference/C/envset_flags.html" class="olink">DB_ENV->set_flags()</a> to disable asynchronous log + flushing as part of re-configuring the client as the new + master. + </p> + <p> + Of course, it is important to ensure that the replicated + master and client environments are truly independent of each + other. For example, it does not help matters that a client has + acknowledged receipt of a message if both master and clients + are on the same power supply, as the failure of the power + supply will still potentially lose information. + </p> + <p> + Configuring a Base API application to achieve the proper mix + of performance and transactional guarantees can be complex. In + brief, there are a few controls an application can set to + configure the guarantees it makes: specification of + <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> for the master environment, specification of + <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> for the client environment, the priorities of + different sites participating in an election, and the behavior + of the application's <span class="bold"><strong>send</strong></span> + function. + </p> + <p> + First, it is rarely useful to write and synchronously flush + the log when a transaction commits on a replication client. It + may be useful where systems share resources and multiple + systems commonly fail at the same time. By default, all + Berkeley DB database environments, whether master or client, + synchronously flush the log on transaction commit or prepare. + Generally, replication masters and clients turn log flush off + for transaction commit using the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> flag. + </p> + <p> + Consider two systems connected by a network interface. One + acts as the master, the other as a read-only client. The + client takes over as master if the master crashes and the + master rejoins the replication group after such a failure. + Both master and client are configured to not synchronously + flush the log on transaction commit (that is, <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> + was configured on both systems). The application's <span class="bold"><strong>send</strong></span> function never returns failure + to the Berkeley DB library, simply forwarding messages to the + client (perhaps over a broadcast mechanism), and always + returning success. On the client, any <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_NOTPERM" class="olink">DB_REP_NOTPERM</a> returns + from the client's <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method are ignored, as well. + This system configuration has excellent performance, but may + lose data in some failure modes. + </p> + <p> + If both the master and the client crash at once, it is + possible to lose committed transactions, that is, + transactional durability is not being maintained. Reliability + can be increased by providing separate power supplies for the + systems and placing them in separate physical + locations. + </p> + <p> + If the connection between the two machines fails (or just + some number of messages are lost), and subsequently the master + crashes, it is possible to lose committed transactions. Again, + transactional durability is not being maintained. Reliability + can be improved in a couple of ways: + </p> <div class="orderedlist"> <ol type="1"> <li> <p> - Use a reliable network protocol (for example, TCP/IP instead of UDP). - </p> + Use a reliable network protocol (for example, + TCP/IP instead of UDP). + </p> </li> <li> <p> - Increase the number of clients and network paths to make it - less likely that a message will be lost. In this case, it is - important to also make sure a client that did receive the - message wins any subsequent election. If a client that did not - receive the message wins a subsequent election, data can still - be lost. - </p> + Increase the number of clients and network paths to + make it less likely that a message will be lost. In + this case, it is important to also make sure a client + that did receive the message wins any subsequent + election. If a client that did not receive the message + wins a subsequent election, data can still be lost. + </p> </li> </ol> </div> - <p>Further, systems may want to guarantee message delivery to the client(s) -(for example, to prevent a network connection from simply discarding -messages). Some systems may want to ensure clients never return -out-of-date information, that is, once a transaction commit returns -success on the master, no client will return old information to a -read-only query. Some of the following changes to a Base API application -may be used to address these issues:</p> + <p> + Further, systems may want to guarantee message delivery to + the client(s) (for example, to prevent a network connection + from simply discarding messages). Some systems may want to + ensure clients never return out-of-date information, that is, + once a transaction commit returns success on the master, no + client will return old information to a read-only query. Some + of the following changes to a Base API application may be used + to address these issues: + </p> <div class="orderedlist"> <ol type="1"> <li> - <p> - Write the application's <span class="bold"><strong>send</strong></span> - function to not return to Berkeley DB until one or more clients - have acknowledged receipt of the message. The number of - clients chosen will be dependent on the application: you will - want to consider likely network partitions (ensure that a - client at each physical site receives the message) and - geographical diversity (ensure that a client on each coast - receives the message). - </p> + <p> + Write the application's <span class="bold"><strong>send</strong></span> function + to not return to Berkeley DB until one or more clients have + acknowledged receipt of the message. The number of + clients chosen will be dependent on the application: + you will want to consider likely network partitions + (ensure that a client at each physical site receives + the message) and geographical diversity (ensure that a + client on each coast receives the message). + </p> </li> <li> <p> - Write the client's message processing loop to not acknowledge - receipt of the message until a call to the <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method - has returned success. Messages resulting in a return of - <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_NOTPERM" class="olink">DB_REP_NOTPERM</a> from the <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method mean the message - could not be flushed to the client's disk. If the client does - not acknowledge receipt of such messages to the master until a - subsequent call to the <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method returns - <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_ISPERM" class="olink">DB_REP_ISPERM</a> and the LSN returned is at least as large as - this message's LSN, then the master's <span class="bold"><strong>send</strong></span> function will not return - success to the Berkeley DB library. This means the thread - committing the transaction on the master will not be allowed to - proceed based on the transaction having committed until the - selected set of clients have received the message and consider - it complete. - </p> + Write the client's message processing loop to not + acknowledge receipt of the message until a call to the + <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method has returned success. Messages + resulting in a return of <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_NOTPERM" class="olink">DB_REP_NOTPERM</a> from the + <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method mean the message could not be + flushed to the client's disk. If the client does not + acknowledge receipt of such messages to the master + until a subsequent call to the <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method + returns <a href="../api_reference/C/repmessage.html#repmsg_DB_REP_ISPERM" class="olink">DB_REP_ISPERM</a> and the LSN returned is at + least as large as this message's LSN, then the + master's <span class="bold"><strong>send</strong></span> + function will not return success to the Berkeley DB + library. This means the thread committing the + transaction on the master will not be allowed to + proceed based on the transaction having committed + until the selected set of clients have received the + message and consider it complete. + </p> <p> - Alternatively, the client's message processing loop could - acknowledge the message to the master, but with an error code - indicating that the application's <span class="bold"><strong>send</strong></span> function should not return to - the Berkeley DB library until a subsequent acknowledgement from - the same client indicates success. - </p> + Alternatively, the client's message processing loop + could acknowledge the message to the master, but with + an error code indicating that the application's + <span class="bold"><strong>send</strong></span> function + should not return to the Berkeley DB library until a + subsequent acknowledgement from the same client + indicates success. + </p> <p> - The application send callback function invoked by Berkeley DB - contains an LSN of the record being sent (if appropriate for - that record). When <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> method returns indicators that - a permanent record has been written then it also returns the - maximum LSN of the permanent record written. - </p> + The application send callback function invoked by + Berkeley DB contains an LSN of the record being sent + (if appropriate for that record). When <a href="../api_reference/C/repmessage.html" class="olink">DB_ENV->rep_process_message()</a> + method returns indicators that a permanent record has + been written then it also returns the maximum LSN of + the permanent record written. + </p> </li> </ol> </div> - <p>There is one final pair of failure scenarios to consider. First, it is -not possible to abort transactions after the application's <span class="bold"><strong>send</strong></span> -function has been called, as the master may have already written the -commit log records to disk, and so abort is no longer an option. -Second, a related problem is that even though the master will attempt -to flush the local log if the <span class="bold"><strong>send</strong></span> function returns failure, -that flush may fail (for example, when the local disk is full). Again, -the transaction cannot be aborted as one or more clients may have -committed the transaction even if <span class="bold"><strong>send</strong></span> returns failure. Rare -applications may not be able to tolerate these unlikely failure modes. -In that case the application may want to:</p> + <p> + There is one final pair of failure scenarios to consider. + First, it is not possible to abort transactions after the + application's <span class="bold"><strong>send</strong></span> function + has been called, as the master may have already written the + commit log records to disk, and so abort is no longer an + option. Second, a related problem is that even though the + master will attempt to flush the local log if the <span class="bold"><strong>send</strong></span> function returns failure, that + flush may fail (for example, when the local disk is full). + Again, the transaction cannot be aborted as one or more + clients may have committed the transaction even if <span class="bold"><strong>send</strong></span> returns failure. Rare + applications may not be able to tolerate these unlikely + failure modes. In that case the application may want + to: + </p> <div class="orderedlist"> <ol type="1"> <li> - <p> - Configure the master to do always local synchronous commits - (turning off the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> configuration). This will - decrease performance significantly, of course (one of the - reasons to use replication is to avoid local disk writes.) In - this configuration, failure to write the local log will cause - the transaction to abort in all cases. - </p> + <p> + Configure the master to do always local synchronous + commits (turning off the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> + configuration). This will decrease performance + significantly, of course (one of the reasons to use + replication is to avoid local disk writes.) In this + configuration, failure to write the local log will + cause the transaction to abort in all cases. + </p> </li> <li> - <p> - Do not return from the application's <span class="bold"><strong>send</strong></span> function under any conditions, - until the selected set of clients has acknowledged the message. - Until the <span class="bold"><strong>send</strong></span> function - returns to the Berkeley DB library, the thread committing the - transaction on the master will wait, and so no application will - be able to act on the knowledge that the transaction has - committed. - </p> + <p> + Do not return from the application's <span class="bold"><strong>send</strong></span> function under any + conditions, until the selected set of clients has + acknowledged the message. Until the <span class="bold"><strong>send</strong></span> function returns to + the Berkeley DB library, the thread committing the + transaction on the master will wait, and so no + application will be able to act on the knowledge that + the transaction has committed. + </p> </li> </ol> </div> @@ -320,7 +399,7 @@ In that case the application may want to:</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Master Leases</td> + <td width="40%" align="right" valign="top"> Master leases</td> </tr> </table> </div> diff --git a/docs/programmer_reference/rep_twosite.html b/docs/programmer_reference/rep_twosite.html index 550404bd..c8882906 100644 --- a/docs/programmer_reference/rep_twosite.html +++ b/docs/programmer_reference/rep_twosite.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="repmgr_channels.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_partition.html">Next</a></td> </tr> </table> @@ -38,78 +36,239 @@ </div> </div> </div> - <p> + <div class="toc"> + <dl> + <dt> + <span class="sect2"> + <a href="rep_twosite.html#twosite_strict">Two-site strict configuration</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="rep_twosite.html#twosite_prefmas">Preferred master mode</a> + </span> + </dt> + <dt> + <span class="sect2"> + <a href="rep_twosite.html#twosite_other">Other alternatives</a> + </span> + </dt> + </dl> + </div> + <p> One of the benefits of replication is that it helps your - application remain available for writes even when a site crashes. - Another benefit is the added durability achieved by storing multiple - copies of your application data at different sites. However, if - your replication group contains only two sites, you must prioritize - which of these benefits is more important to your - application. + application remain available for writes even when a site + crashes. Another benefit is the added durability achieved by + storing multiple copies of your application data at different + sites. However, if your replication group contains only two + sites, you must prioritize which of these benefits is more + important to your application. </p> - <p> + <p> A two-site replication group is particularly vulnerable to - duplicate masters if there is a loss of communication between the - sites. The original master continues to accept new transactions. - If the original client detects the loss of the master and elects - itself master, it also starts accepting new transactions. When - communications are restored, there are duplicate masters and one - site's new transactions will be rolled back. - </p> - <p> - If it is unacceptable to your application for any new transactions - to be rolled back, the alternative in a two-site replication group - is to require both sites to be present in order to elect a master. - This stops a client from electing itself master when it loses - contact with the master and prevents creation of parallel sets of - transactions, one of which must be rolled back. + duplicate masters when there is a disruption to communications + between the sites. The original master continues to accept new + transactions. If the original client detects the loss of the + master and elects itself master, it also starts accepting new + transactions. When communications are restored, there are + duplicate masters and one site's new transactions will be + rolled back. </p> <p> - However, requiring both sites to be present to elect a master - results in a loss of write availability when the master crashes. - The client cannot take over as master and the replication group - exists in a read-only state until the original master site rejoins - the replication group. + If it is unacceptable to your application for any new + transactions to be rolled back, the alternative in a two-site + replication group is to require both sites to be present in + order to elect a master. This stops a client from electing + itself master when it loses contact with the master and + prevents creation of parallel sets of transactions, one of + which must be rolled back. + However, requiring both sites to be present to elect a + master results in a loss of write availability when the master + crashes. The client cannot take over as master and the + replication group exists in a read-only state until the + original master site rejoins the replication group. </p> - <p> + <div class="sect2" lang="en" xml:lang="en"> + <div class="titlepage"> + <div> + <div> + <h3 class="title"><a id="twosite_strict"></a>Two-site strict configuration</h3> + </div> + </div> + </div> + <p> Replication Manager applications use the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method - <a href="../api_reference/C/repconfig.html#config_DB_REPMGR_CONF_2SITE_STRICT" class="olink">DB_REPMGR_CONF_2SITE_STRICT</a> flag to make this tradeoff between - write availability and transaction durability. When this flag is - turned on, Replication Manager favors transaction durability. When - it is turned off, Replication Manager favors write - availability. + <a href="../api_reference/C/repconfig.html#config_DB_REPMGR_CONF_2SITE_STRICT" class="olink">DB_REPMGR_CONF_2SITE_STRICT</a> flag to make this tradeoff + between write availability and transaction durability. When + this flag is turned on, Replication Manager favors transaction + durability. When it is turned off, Replication Manager favors + write availability. </p> - <p> - A two-site Replication Manager application that uses heartbeats in - an environment with frequent communications disruptions generally - should operate with the <a href="../api_reference/C/repconfig.html#config_DB_REPMGR_CONF_2SITE_STRICT" class="olink">DB_REPMGR_CONF_2SITE_STRICT</a> flag turned - on. Otherwise, frequent heartbeat failures will cause frequent - duplicate masters and the resulting elections and client + <p> + A two-site Replication Manager application that uses + heartbeats in an environment with frequent communications + disruptions generally should operate with the + <a href="../api_reference/C/repconfig.html#config_DB_REPMGR_CONF_2SITE_STRICT" class="olink">DB_REPMGR_CONF_2SITE_STRICT</a> flag turned on. Otherwise, + frequent heartbeat failures will cause frequent duplicate + masters and the resulting elections and client synchronizations will make one or both sites unavailable for extended periods of time. </p> - <p> - Base API applications use the values of the - <span class="bold"><strong>nvotes</strong></span> and - <span class="bold"><strong>nsites</strong></span> parameters in calls to the - <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> method to make this tradeoff. For more information, see - <a class="xref" href="rep_elect.html" title="Elections">Elections</a>.</p> - <p> - A replication group containing only two electable sites is subject - to duplicate masters and rollback of one site's new transactions - even when it contains additional unelectable sites. The - <a href="../api_reference/C/repconfig.html#config_DB_REPMGR_CONF_2SITE_STRICT" class="olink">DB_REPMGR_CONF_2SITE_STRICT</a> does not apply in this case because - the replication group is larger than two sites. + <p> + A replication group containing only two electable sites is + subject to duplicate masters and rollback of one site's new + transactions even when it contains additional unelectable + sites. The <a href="../api_reference/C/repconfig.html#config_DB_REPMGR_CONF_2SITE_STRICT" class="olink">DB_REPMGR_CONF_2SITE_STRICT</a> flag does not apply in + this case because the replication group is larger than two + sites. </p> - <p> - If both write availability and transaction durability are important - to your application, you should strongly consider having three or - more electable sites in your replication group. You should also - carefully choose an acknowledgement policy that requires at least a - quorum of sites. It is best to have an odd number of electable - sites to provide a clear majority in the event of a network - partition. + <p> + Base API applications using two sites use the values of the <span class="bold"><strong>nvotes</strong></span> and <span class="bold"><strong>nsites</strong></span> parameters in calls to the <a href="../api_reference/C/repelect.html" class="olink">DB_ENV->rep_elect()</a> + method to make this durability vs. availability tradeoff. For more + information, see <a class="xref" href="rep_elect.html" title="Elections">Elections</a>. + </p> + </div> + <div class="sect2" lang="en" xml:lang="en"> + <div class="titlepage"> + <div> + <div> + <h3 class="title"><a id="twosite_prefmas"></a>Preferred master mode</h3> + </div> + </div> + </div> + <p> + In some two-site replication groups, it is desirable for one + particular site to function as the master whenever possible. This + might be due to hardware differences, geographical location, or + other reasons. Replication Manager preferred master mode provides + this behavior. + </p> + <p> + A preferred master replication group contains two sites where one + site is the <span class="bold"><strong>preferred master site</strong></span> + and the other site is the <span class="bold"><strong>client + site</strong></span>. The preferred master site operates as + the master as much of the time as its availability permits. + The client site takes over as <span class="bold"><strong>temporary + master</strong></span> when the preferred + master is unavailable, providing write availability when the + preferred master site is down. Transactions committed on the preferred + master site are never rolled back, providing more predictable + durability than turning off the <a href="../api_reference/C/repconfig.html#config_DB_REPMGR_CONF_2SITE_STRICT" class="olink">DB_REPMGR_CONF_2SITE_STRICT</a> flag. + </p> + <p> + In a preferred master replication group where the client site is + operating as the temporary master, when the preferred master site + again becomes available it synchronizes with the temporary master + and then automatically takes over as master. The preferred master + synchronization preserves temporary master transactions when + they do not conflict with any new preferred master transactions. If + there are conflicting transactions, the temporary master transactions + are always the transactions that are rolled back. + </p> + <p> + To configure a preferred master replication group, you use the + <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method to specify the <a href="../api_reference/C/repconfig.html#config_DB_REPMGR_CONF_PREFMAS_MASTER" class="olink">DB_REPMGR_CONF_PREFMAS_MASTER</a> + flag on the preferred master site and to specify the + <a href="../api_reference/C/repconfig.html#config_DB_REPMGR_CONF_PREFMAS_CLIENT" class="olink">DB_REPMGR_CONF_PREFMAS_CLIENT</a> flag on the client site. These + flags must be specified before calling the <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a> method + and cannot be changed after it is called. Both sites should call + the <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a> method using the <a href="../api_reference/C/repmgrstart.html#repmgrstart_DB_REP_CLIENT" class="olink">DB_REP_CLIENT</a> flags value. + </p> + <p> + Some configuration items are automatically set in preferred master + mode and cannot be changed: the <a href="../api_reference/C/repconfig.html#config_DB_REPMGR_CONF_2SITE_STRICT" class="olink">DB_REPMGR_CONF_2SITE_STRICT</a> and + <a href="../api_reference/C/repconfig.html#config_DB_REPMGR_CONF_ELECTIONS" class="olink">DB_REPMGR_CONF_ELECTIONS</a> flags are turned on, and + each site's priority value is set. The following timeout values + have different defaults in preferred master mode which can be + changed: <a href="../api_reference/C/repset_timeout.html#set_timeout_DB_REP_HEARTBEAT_SEND" class="olink">DB_REP_HEARTBEAT_SEND</a>, <a href="../api_reference/C/repset_timeout.html#set_timeout_DB_REP_HEARTBEAT_MONITOR" class="olink">DB_REP_HEARTBEAT_MONITOR</a>, + <a href="../api_reference/C/repset_timeout.html#set_timeout_DB_REP_ELECTION_RETRY" class="olink">DB_REP_ELECTION_RETRY</a>. Heartbeats cannot be turned off in + preferred master mode because they are required for automatic + takeovers. Preferred master mode + is not supported with master leases, in-memory databases, in-memory + logs, in-memory replication files or private environments. + </p> + <p> + When restarting a preferred master replication group after both + sites were down, it is best to restart the client site first in + order to preserve as many temporary master transactions as possible. + If the preferred master site is started first and then commits new + transactions, these new preferred master transactions would conflict + with any recent temporary master transactions and the temporary + master transactions would be rolled back. + </p> + <p> + If the preferred master site can no longer be used (for example, due + to a hardware failure), you can reconfigure the client site to become + the new preferred master site using the following steps: </p> + <div class="itemizedlist"> + <ul type="disc"> + <li> + <p> + Shut down the old preferred master site if it is still running. + Use the old client site (now operating as temporary master) to + remove the old preferred master site from the replication group, + then close the old client's environment. + </p> + </li> + <li> + <p> + If you are using the <a href="../api_reference/C/configuration_reference.html" class="olink">DB_CONFIG</a> file to configure preferred master + mode, edit it to replace the <a href="../api_reference/C/rep_set_config_parameter.html" class="olink">rep_set_config</a> method + <code class="literal">db_repmgr_conf_prefmas_client</code> parameter + with the <code class="literal">db_repmgr_conf_prefmas_master</code> + parameter and adjust any other configuration values as needed. + </p> + </li> + <li> + <p> + Perform a recovery on the new preferred master site's environment + and reopen it. + </p> + </li> + <li> + <p> + If you are using the <a href="../api_reference/C/repconfig.html" class="olink">DB_ENV->rep_set_config()</a> method to configure preferred + master mode, invoke it to configure the + <a href="../api_reference/C/repconfig.html#config_DB_REPMGR_CONF_PREFMAS_MASTER" class="olink">DB_REPMGR_CONF_PREFMAS_MASTER</a> flag and adjust any other + configuration values as needed. + </p> + </li> + <li> + <p> + Call the <a href="../api_reference/C/repmgrstart.html" class="olink">DB_ENV->repmgr_start()</a> method with the <a href="../api_reference/C/repmgrstart.html#repmgrstart_DB_REP_CLIENT" class="olink">DB_REP_CLIENT</a> flags + value to restart this site as the new preferred master. + </p> + </li> + </ul> + </div> + <p> + Preferred master applications are more likely to have + <code class="literal">DB_LOCK_DEADLOCK</code>, + <code class="literal">DB_REP_HANDLE_DEAD</code> and <code class="literal">EACCES</code> + errors on either site during automatic takeovers and should be + prepared to handle these errors appropriately. + </p> + </div> + <div class="sect2" lang="en" xml:lang="en"> + <div class="titlepage"> + <div> + <div> + <h3 class="title"><a id="twosite_other"></a>Other alternatives</h3> + </div> + </div> + </div> + <p> + If both write availability and transaction durability are + important to your application, you should strongly consider + having three or more electable sites in your replication + group. You should also carefully choose an acknowledgement + policy that requires at least a quorum of sites. It is best to + have an odd number of electable sites to provide a clear + majority in the event of a network partition. + </p> + </div> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/repmgr_channels.html b/docs/programmer_reference/repmgr_channels.html index 6897d45c..09c0bde5 100644 --- a/docs/programmer_reference/repmgr_channels.html +++ b/docs/programmer_reference/repmgr_channels.html @@ -8,13 +8,13 @@ <meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /> <link rel="start" href="index.html" title="Berkeley DB Programmer's Reference Guide" /> <link rel="up" href="rep.html" title="Chapter 12. Berkeley DB Replication" /> - <link rel="prev" href="rep_clock_skew.html" title="Clock Skew" /> + <link rel="prev" href="rep_clock_skew.html" title="Clock skew" /> <link rel="next" href="rep_twosite.html" title="Special considerations for two-site replication groups" /> </head> <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_clock_skew.html">Prev</a> </td> - <th width="60%" align="center">Chapter 12. - Berkeley DB Replication - </th> + <th width="60%" align="center">Chapter 12. Berkeley DB Replication </th> <td width="20%" align="right"> <a accesskey="n" href="rep_twosite.html">Next</a></td> </tr> </table> @@ -57,26 +55,24 @@ </dt> </dl> </div> - <p> - The various sites comprising a replication group frequently need to - communicate with one another. Mostly, these messages are handled - for you internally by the Replication Manager. However, your - application may have a requirement to pass messages beyond what the - Replication Manager requires in order to satisfy its own internal - workings. + <p> + The various sites comprising a replication group frequently + need to communicate with one another. Mostly, these messages + are handled for you internally by the Replication Manager. + However, your application may have a requirement to pass + messages beyond what the Replication Manager requires in order + to satisfy its own internal workings. </p> <p> - For this reason, you can access and use the Replication Manager's - internal message channels. You do this by using the - <code class="literal">DB_CHANNEL</code> class, and by implementing a message - handling function on each of your sites. + For this reason, you can access and use the Replication + Manager's internal message channels. You do this by using the + <code class="literal">DB_CHANNEL</code> class, and by implementing a + message handling function on each of your sites. </p> - <p> - Note that an example of using Replication Manager message channels - is available in the distribution. See - <a class="xref" href="rep_ex_chan.html" title="Ex_rep_chan: a Replication Manager channel example">Ex_rep_chan: a Replication Manager -channel example</a> - for more information. + <p> + Note that an example of using Replication Manager message + channels is available in the distribution. See <a class="xref" href="rep_ex_chan.html" title="Ex_rep_chan: a Replication Manager channel example">Ex_rep_chan: a Replication Manager channel example</a> for + more information. </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> @@ -87,44 +83,47 @@ channel example</a> </div> </div> <p> - The <code class="literal">DB_CHANNEL</code> class provides a series of - methods which allow you to send messages to the other sites in - your replication group. You create a <code class="literal">DB_CHANNEL</code> - handle using the <a href="../api_reference/C/repmgr_channel.html" class="olink">DB_ENV->repmgr_channel()</a> method. When you are - done with the handle, close it using the <a href="../api_reference/C/dbchannel_close.html" class="olink">DB_CHANNEL->close()</a> - method. A closed handle must never be accessed again. Note that - all channel handles should be closed before the associated + The <code class="literal">DB_CHANNEL</code> class provides a + series of methods which allow you to send messages to the + other sites in your replication group. You create a + <code class="literal">DB_CHANNEL</code> handle using the + <a href="../api_reference/C/repmgr_channel.html" class="olink">DB_ENV->repmgr_channel()</a> method. When you are done with the + handle, close it using the <a href="../api_reference/C/dbchannel_close.html" class="olink">DB_CHANNEL->close()</a> method. A + closed handle must never be accessed again. Note that all + channel handles should be closed before the associated environment handle is closed. Also, allow all message operations to complete on the channel before closing the - handle. + handle. </p> <p> - When you create a <code class="literal">DB_CHANNEL</code> handle, you - indicate what channel you want to use. Possibilities are: + When you create a <code class="literal">DB_CHANNEL</code> handle, + you indicate what channel you want to use. Possibilities + are: </p> <div class="itemizedlist"> <ul type="disc"> <li> <p> - The numerical env ID of a remote site in the - replication group. - </p> + The numerical env ID of a remote site in the + replication group. + </p> </li> <li> <p> - <code class="literal">DB_EID_MASTER</code> - </p> - <p> - Messages sent on this channel are sent only to the - master site. Note that messages are always sent to - the current master, even if the master has changed - since the channel was opened. - </p> - <p> - If the local site is the master, then sending messages - on this channel will result in the local site - receiving those messages echoed back to itself. - </p> + <code class="literal">DB_EID_MASTER</code> + </p> + <p> + Messages sent on this channel are sent only to + the master site. Note that messages are always + sent to the current master, even if the master has + changed since the channel was opened. + </p> + <p> + If the local site is the master, then sending + messages on this channel will result in the local + site receiving those messages echoed back to + itself. + </p> </li> </ul> </div> @@ -137,33 +136,31 @@ channel example</a> </div> </div> </div> - <p> - You can send any message you want over a message channel. The - message can be as simple as a character string and as complex - as a large data structure. However, before you can send the - message, you must encapsulate it within one or more <a href="../api_reference/C/dbt.html" class="olink">DBT</a>s. - This means <a class="link" href="am_misc_struct.html" title="Storing C/C++ structures/objects">marshaling the message</a> - if it is contained within a complex data structure. + <p> + You can send any message you want over a message + channel. The message can be as simple as a character + string and as complex as a large data structure. However, + before you can send the message, you must encapsulate it + within one or more <a href="../api_reference/C/dbt.html" class="olink">DBT</a>s. This means <a class="link" href="am_misc_struct.html" title="Storing C/C++ structures/objects">marshaling the message</a> + if it is contained within a complex data structure. </p> + <p> + The methods that you use to send messages all accept an + array of <a href="../api_reference/C/dbt.html" class="olink">DBT</a>s. This means that in most circumstances it + is perfectly acceptable to send multi-part messages. </p> <p> - The methods that you use to send messages all accept an array of - <a href="../api_reference/C/dbt.html" class="olink">DBT</a>s. This means that in most circumstances it is perfectly - acceptable to send multi-part messages. - </p> - <p> - Messages may be sent either asynchronously or synchronously. - To send a message asynchronously, use the <a href="../api_reference/C/dbchannel_send_msg.html" class="olink">DB_CHANNEL->send_msg()</a> - method. This method sends its message and then immediately - returns without waiting for any sort of a response. - + Messages may be sent either asynchronously or + synchronously. To send a message asynchronously, use the + <a href="../api_reference/C/dbchannel_send_msg.html" class="olink">DB_CHANNEL->send_msg()</a> method. This method sends its message + and then immediately returns without waiting for any sort + of a response. </p> <p> To send a message synchronously, use the - <a href="../api_reference/C/dbchannel_send_request.html" class="olink">DB_CHANNEL->send_request()</a> method. This method blocks until it - receives a response from the site to which it sent the message - (or until a timeout threshold is reached). - + <a href="../api_reference/C/dbchannel_send_request.html" class="olink">DB_CHANNEL->send_request()</a> method. This method blocks until + it receives a response from the site to which it sent the + message (or until a timeout threshold is reached). </p> <div class="sect3" lang="en" xml:lang="en"> @@ -175,29 +172,29 @@ channel example</a> </div> </div> <p> - Message responses are required if a message is sent on a - channel using the <a href="../api_reference/C/dbchannel_send_request.html" class="olink">DB_CHANNEL->send_request()</a> method. That - method accepts the address of a single <a href="../api_reference/C/dbt.html" class="olink">DBT</a> which is used - to receive the response from the remote site. + Message responses are required if a message is sent + on a channel using the <a href="../api_reference/C/dbchannel_send_request.html" class="olink">DB_CHANNEL->send_request()</a> + method. That method accepts the address of a single + <a href="../api_reference/C/dbt.html" class="olink">DBT</a> which is used to receive the response from the + remote site. </p> <p> - Message responses are encapsulated in a single <a href="../api_reference/C/dbt.html" class="olink">DBT</a>. The - response can be anything from a complex data structure, to - a string, to a simple type, to no information at all. In - the latter case, receipt of the <a href="../api_reference/C/dbt.html" class="olink">DBT</a> is sufficient to - indicate that the request was received at the remote site. + Message responses are encapsulated in a single + <a href="../api_reference/C/dbt.html" class="olink">DBT</a>. The response can be anything from a complex + data structure, to a string, to a simple type, to no + information at all. In the latter case, receipt of the + <a href="../api_reference/C/dbt.html" class="olink">DBT</a> is sufficient to indicate that the request was + received at the remote site. </p> - <p> - Responses are sent back from the remote system using its - message handling function. Usually that function calls - <a href="../api_reference/C/dbchannel_send_msg.html" class="olink">DB_CHANNEL->send_msg()</a> to send a single response. - </p> - <p> - The response must be contained in a single <a href="../api_reference/C/dbt.html" class="olink">DBT</a>. If a - multi-part response is required by the application, you can - configure the response <a href="../api_reference/C/dbt.html" class="olink">DBT</a> that you provide to - <a href="../api_reference/C/dbchannel_send_request.html" class="olink">DB_CHANNEL->send_request()</a> for - <a class="link" href="am_misc_bulk.html" title="Retrieving and updating records in bulk">bulk operations</a>. + <p> Responses are sent back from the remote system + using its message handling function. Usually that + function calls <a href="../api_reference/C/dbchannel_send_msg.html" class="olink">DB_CHANNEL->send_msg()</a> to send a single + response. </p> + <p> + The response must be contained in a single <a href="../api_reference/C/dbt.html" class="olink">DBT</a>. + If a multi-part response is required by the + application, you can configure the response <a href="../api_reference/C/dbt.html" class="olink">DBT</a> that + you provide to <a href="../api_reference/C/dbchannel_send_request.html" class="olink">DB_CHANNEL->send_request()</a> for <a class="link" href="am_misc_bulk.html" title="Retrieving and updating records in bulk">bulk operations</a>. </p> </div> </div> @@ -210,58 +207,60 @@ channel example</a> </div> </div> <p> - Messages received at a remote site are handled using a callback - function. This function is configured for the local environment - using the <a href="../api_reference/C/repmgr_msg_dispatch.html" class="olink">DB_ENV->repmgr_msg_dispatch()</a> method. For best results, - the message dispatch function should be configured for the - local environment before replication is started. In this way, - you do not run the risk of missing messages sent after - replication has started but before the message dispatch - function is configured for the environment. + Messages received at a remote site are handled using a + callback function. This function is configured for the + local environment using the <a href="../api_reference/C/repmgr_msg_dispatch.html" class="olink">DB_ENV->repmgr_msg_dispatch()</a> + method. For best results, the message dispatch function + should be configured for the local environment before + replication is started. In this way, you do not run the + risk of missing messages sent after replication has + started but before the message dispatch function is + configured for the environment. </p> <p> - The callback configured by <a href="../api_reference/C/repmgr_msg_dispatch.html" class="olink">DB_ENV->repmgr_msg_dispatch()</a> accepts - four parameters of note: + The callback configured by <a href="../api_reference/C/repmgr_msg_dispatch.html" class="olink">DB_ENV->repmgr_msg_dispatch()</a> + accepts four parameters of note: </p> <div class="itemizedlist"> <ul type="disc"> <li> - <p> - A response channel. This is the channel the function - will use to response to the message, if a response is - required. To respond to the message, the function uses - the <a href="../api_reference/C/dbchannel_send_msg.html" class="olink">DB_CHANNEL->send_msg()</a> method. + <p> + A response channel. This is the channel the + function will use to respond to the message, if a + response is required. To respond to the message, + the function uses the <a href="../api_reference/C/dbchannel_send_msg.html" class="olink">DB_CHANNEL->send_msg()</a> method. </p> </li> <li> <p> - An array of <a href="../api_reference/C/dbt.html" class="olink">DBT</a>s. These hold the message that this - function must handle. + An array of <a href="../api_reference/C/dbt.html" class="olink">DBT</a>s. These hold the message that + this function must handle. </p> </li> <li> <p> - A numerical value that indicates how many elements the - previously described array holds. + A numerical value that indicates how many + elements the previously described array holds. </p> </li> <li> - <p> - A flag that indicates whether the message requires a - response. If the flag is set to + <p> + A flag that indicates whether the message + requires a response. If the flag is set to <code class="literal">DB_REPMGR_NEED_RESPONSE</code>, then the function should send a single <a href="../api_reference/C/dbt.html" class="olink">DBT</a> in - response using the channel provided to this function, - and the <a href="../api_reference/C/dbchannel_send_msg.html" class="olink">DB_CHANNEL->send_msg()</a> method. + response using the channel provided to this + function, and the <a href="../api_reference/C/dbchannel_send_msg.html" class="olink">DB_CHANNEL->send_msg()</a> method. </p> </li> </ul> </div> - <p> + <p> For an example of using this callback, see the - <code class="literal">operation_dispatch()</code> function, which is - available with the <a class="link" href="rep_ex_chan.html" title="Ex_rep_chan: a Replication Manager channel example">ex_rep_chan example</a> - in your product distribution. + <code class="literal">operation_dispatch()</code> function, + which is available with the <a class="link" href="rep_ex_chan.html" title="Ex_rep_chan: a Replication Manager channel example"> + ex_rep_chan example</a> in your product + distribution. </p> </div> </div> @@ -276,7 +275,7 @@ channel example</a> <td width="40%" align="right"> <a accesskey="n" href="rep_twosite.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Clock Skew </td> + <td width="40%" align="left" valign="top">Clock skew </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/rq_conf.html b/docs/programmer_reference/rq_conf.html index ab9c302d..a7379d26 100644 --- a/docs/programmer_reference/rq_conf.html +++ b/docs/programmer_reference/rq_conf.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="heap_conf.html">Prev</a> </td> - <th width="60%" align="center">Chapter 2. - Access Method Configuration - </th> + <th width="60%" align="center">Chapter 2. Access Method Configuration </th> <td width="20%" align="right"> <a accesskey="n" href="am.html">Next</a></td> </tr> </table> @@ -42,265 +40,334 @@ <dl> <dt> <span class="sect2"> - <a href="rq_conf.html#am_conf_recno">Managing record-based databases</a> + <a href="rq_conf.html#am_conf_recno">Managing record-based + databases</a> </span> </dt> <dt> <span class="sect2"> - <a href="rq_conf.html#am_conf_extentsize">Selecting a Queue extent size</a> + <a href="rq_conf.html#am_conf_extentsize">Selecting a Queue extent + size</a> </span> </dt> <dt> <span class="sect2"> - <a href="rq_conf.html#am_conf_re_source">Flat-text backing files</a> + <a href="rq_conf.html#am_conf_re_source">Flat-text backing + files</a> </span> </dt> <dt> <span class="sect2"> - <a href="rq_conf.html#am_conf_renumber">Logically renumbering records</a> + <a href="rq_conf.html#am_conf_renumber">Logically renumbering + records</a> </span> </dt> </dl> </div> - <p> - There are a series of configuration tasks which you can perform when - using the Queue and Recno access methods. They are described in the following sections. -</p> + <p> + There are a series of configuration tasks which you can + perform when using the Queue and Recno access methods. They + are described in the following sections. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="am_conf_recno"></a>Managing record-based databases</h3> + <h3 class="title"><a id="am_conf_recno"></a>Managing record-based + databases</h3> </div> </div> </div> - <p>When using fixed- or variable-length record-based databases, particularly -with flat-text backing files, there are several items that the user can -control. The Recno access method can be used to store either variable- -or fixed-length data items. By default, the Recno access method stores -variable-length data items. The Queue access method can only store -fixed-length data items.</p> + <p> + When using fixed- or variable-length record-based databases, + particularly with flat-text backing files, there are several + items that the user can control. The Recno access method can + be used to store either variable- or fixed-length data items. + By default, the Recno access method stores variable-length + data items. The Queue access method can only store + fixed-length data items. + </p> <div class="sect3" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idp704256"></a>Record Delimiters</h4> + <h4 class="title"><a id="idm2664864"></a>Record Delimiters</h4> </div> </div> </div> - <p>When using the Recno access method to store variable-length records, -records read from any backing source file are separated by a specific -byte value which marks the end of one record and the beginning of the -next. This delimiting value is ignored except when reading records from -a backing source file, that is, records may be stored into the database -that include the delimiter byte. However, if such records are written -out to the backing source file and the backing source file is -subsequently read into a database, the records will be split where -delimiting bytes were found.</p> - <p>For example, UNIX text files can usually be interpreted as a sequence of -variable-length records separated by ASCII newline characters. This byte -value (ASCII 0x0a) is the default delimiter. Applications may specify a -different delimiting byte using the <a href="../api_reference/C/dbset_re_delim.html" class="olink">DB->set_re_delim()</a> method. If no -backing source file is being used, there is no reason to set the -delimiting byte value.</p> + <p> + When using the Recno access method to store + variable-length records, records read from any backing + source file are separated by a specific byte value which + marks the end of one record and the beginning of the next. + This delimiting value is ignored except when reading + records from a backing source file, that is, records may + be stored into the database that include the delimiter + byte. However, if such records are written out to the + backing source file and the backing source file is + subsequently read into a database, the records will be + split where delimiting bytes were found. + </p> + <p> + For example, UNIX text files can usually be interpreted + as a sequence of variable-length records separated by + ASCII newline characters. This byte value (ASCII 0x0a) is + the default delimiter. Applications may specify a + different delimiting byte using the <a href="../api_reference/C/dbset_re_delim.html" class="olink">DB->set_re_delim()</a> + method. If no backing source file is being used, there is + no reason to set the delimiting byte value. + </p> </div> <div class="sect3" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idp719456"></a>Record Length</h4> + <h4 class="title"><a id="idm2240344"></a>Record Length</h4> </div> </div> </div> - <p>When using the Recno or Queue access methods to store fixed-length -records, the record length must be specified. Since the Queue access -method always uses fixed-length records, the user must always set the -record length prior to creating the database. Setting the record length -is what causes the Recno access method to store fixed-length, not -variable-length, records.</p> - <p>The length of the records is specified by calling the -<a href="../api_reference/C/dbset_re_len.html" class="olink">DB->set_re_len()</a> method. The default length of the records is 0 bytes. -Any record read from a backing source file or otherwise stored in the -database that is shorter than the declared length will automatically be -padded as described for the <a href="../api_reference/C/dbset_re_pad.html" class="olink">DB->set_re_pad()</a> method. Any record stored -that is longer than the declared length results in an error. For -further information on backing source files, see -<a class="xref" href="rq_conf.html#am_conf_re_source" title="Flat-text backing files">Flat-text backing files</a>.</p> + <p> + When using the Recno or Queue access methods to store + fixed-length records, the record length must be specified. + Since the Queue access method always uses fixed-length + records, the user must always set the record length prior + to creating the database. Setting the record length is + what causes the Recno access method to store fixed-length, + not variable-length, records. + </p> + <p> + The length of the records is specified by calling the + <a href="../api_reference/C/dbset_re_len.html" class="olink">DB->set_re_len()</a> method. The default length of the records + is 0 bytes. Any record read from a backing source file or + otherwise stored in the database that is shorter than the + declared length will automatically be padded as described + for the <a href="../api_reference/C/dbset_re_pad.html" class="olink">DB->set_re_pad()</a> method. Any record stored that is + longer than the declared length results in an error. For + further information on backing source files, see <a class="xref" href="rq_conf.html#am_conf_re_source" title="Flat-text backing files">Flat-text backing + files</a>. + </p> </div> <div class="sect3" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idm2290720"></a>Record Padding Byte Value</h4> + <h4 class="title"><a id="idm2071320"></a>Record Padding Byte Value</h4> </div> </div> </div> - <p>When storing fixed-length records in a Queue or Recno database, a pad -character may be specified by calling the <a href="../api_reference/C/dbset_re_pad.html" class="olink">DB->set_re_pad()</a> method. Any -record read from the backing source file or otherwise stored in the -database that is shorter than the expected length will automatically be -padded with this byte value. If fixed-length records are specified but -no pad value is specified, a space character (0x20 in the ASCII -character set) will be used. For further information on backing source -files, see <a class="xref" href="rq_conf.html#am_conf_re_source" title="Flat-text backing files">Flat-text backing files</a>.</p> + <p> + When storing fixed-length records in a Queue or Recno + database, a pad character may be specified by calling the + <a href="../api_reference/C/dbset_re_pad.html" class="olink">DB->set_re_pad()</a> method. Any record read from the backing + source file or otherwise stored in the database that is + shorter than the expected length will automatically be + padded with this byte value. If fixed-length records are + specified but no pad value is specified, a space character + (0x20 in the ASCII character set) will be used. For + further information on backing source files, see <a class="xref" href="rq_conf.html#am_conf_re_source" title="Flat-text backing files">Flat-text backing + files</a>. + </p> </div> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="am_conf_extentsize"></a>Selecting a Queue extent size</h3> + <h3 class="title"><a id="am_conf_extentsize"></a>Selecting a Queue extent + size</h3> </div> </div> </div> <p> - In Queue databases, records are allocated sequentially and directly - mapped to an offset within the file storage for the database. As - records are deleted from the Queue, pages will become empty and will - not be reused in normal queue operations. To facilitate the - reclamation of disk space a Queue may be partitioned into extents. - Each extent is kept in a separate physical file. + In Queue databases, records are allocated sequentially and + directly mapped to an offset within the file storage for the + database. As records are deleted from the Queue, pages will + become empty and will not be reused in normal queue + operations. To facilitate the reclamation of disk space a + Queue may be partitioned into extents. Each extent is kept in + a separate physical file. </p> - <p> - Extent files are automatically created as needed and marked for - deletion when the head of the queue moves off the extent. The extent - will not be deleted until all processes close the extent. In addition, - Berkeley DB caches a small number of extents that have been recently - used; this may delay when an extent will be deleted. The number of - extents left open depends on queue activity. + <p> + Extent files are automatically created as needed and marked + for deletion when the head of the queue moves off the extent. + The extent will not be deleted until all processes close the + extent. In addition, Berkeley DB caches a small number of + extents that have been recently used; this may delay when an + extent will be deleted. The number of extents left open + depends on queue activity. </p> <p> - The extent size specifies the number of pages that make up each extent. - By default, if no extent size is specified, the Queue resides in a - single file and disk space is not reclaimed. In choosing an extent - size there is a tradeoff between the amount of disk space used and the - overhead of creating and deleting files. If the extent size is too - small, the system will pay a performance penalty, creating and deleting - files frequently. In addition, if the active part of the queue spans - many files, all those files will need to be open at the same time, - consuming system and process file resources. + The extent size specifies the number of pages that make up + each extent. By default, if no extent size is specified, the + Queue resides in a single file and disk space is not + reclaimed. In choosing an extent size there is a tradeoff + between the amount of disk space used and the overhead of + creating and deleting files. If the extent size is too small, + the system will pay a performance penalty, creating and + deleting files frequently. In addition, if the active part of + the queue spans many files, all those files will need to be + open at the same time, consuming system and process file + resources. </p> <p> - You can set the Queue extent size using the <a href="../api_reference/C/dbset_q_extentsize.html" class="olink">DB->set_q_extentsize()</a> - method. You can see the current extent size using the - <a href="../api_reference/C/dbget_q_extentsize.html" class="olink">DB->get_q_extentsize()</a> method. + You can set the Queue extent size using the + <a href="../api_reference/C/dbset_q_extentsize.html" class="olink">DB->set_q_extentsize()</a> method. You can see the current extent + size using the <a href="../api_reference/C/dbget_q_extentsize.html" class="olink">DB->get_q_extentsize()</a> method. </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="am_conf_re_source"></a>Flat-text backing files</h3> + <h3 class="title"><a id="am_conf_re_source"></a>Flat-text backing + files</h3> </div> </div> </div> - <p>It is possible to back any Recno database (either fixed or variable -length) with a flat-text source file. This provides fast read (and -potentially write) access to databases that are normally created and -stored as flat-text files. The backing source file may be specified by -calling the <a href="../api_reference/C/dbset_re_source.html" class="olink">DB->set_re_source()</a> method.</p> - <p>The backing source file will be read to initialize the database. In the -case of variable length records, the records are assumed to be separated -as described for the <a href="../api_reference/C/dbset_re_delim.html" class="olink">DB->set_re_delim()</a> method. For example, standard -UNIX byte stream files can be interpreted as a sequence of variable -length records separated by ASCII newline characters. This is the -default.</p> - <p>When cached data would normally be written back to the underlying -database file (for example, when the <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> or -<a href="../api_reference/C/dbsync.html" class="olink">DB->sync()</a> methods are called), the in-memory copy of the database will -be written back to the backing source file.</p> - <p>The backing source file must already exist (but may be zero-length) when -<a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> is called. By default, the backing source file is read -lazily, that is, records are not read from the backing source file until -they are requested by the application. If multiple processes (not -threads) are accessing a Recno database concurrently and either -inserting or deleting records, the backing source file must be read in -its entirety before more than a single process accesses the database, -and only that process should specify the backing source file as part of -the <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> call. This can be accomplished by calling the -<a href="../api_reference/C/dbset_flags.html" class="olink">DB->set_flags()</a> method with the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_SNAPSHOT" class="olink">DB_SNAPSHOT</a> flag.</p> - <p>Reading and writing the backing source file cannot be transactionally -protected because it involves filesystem operations that are not part of -the Berkeley DB transaction methodology. For this reason, if a temporary -database is used to hold the records (a NULL was specified as the file -argument to <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a>), <span class="bold"><strong>it is possible to lose the -contents of the backing source file if the system crashes at the right -instant</strong></span>. If a permanent file is used to hold the database (a filename -was specified as the file argument to <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a>), normal database -recovery on that file can be used to prevent information loss. It is -still possible that the contents of the backing source file itself will -be corrupted or lost if the system crashes.</p> - <p>For all of the above reasons, the backing source file is generally used -to specify databases that are read-only for Berkeley DB applications, and that -are either generated on the fly by software tools, or modified using a -different mechanism such as a text editor.</p> + <p> + It is possible to back any Recno database (either fixed or + variable length) with a flat-text source file. This provides + fast read (and potentially write) access to databases that are + normally created and stored as flat-text files. The backing + source file may be specified by calling the <a href="../api_reference/C/dbset_re_source.html" class="olink">DB->set_re_source()</a> + method. + </p> + <p> + The backing source file will be read to initialize the + database. In the case of variable length records, the records + are assumed to be separated as described for the + <a href="../api_reference/C/dbset_re_delim.html" class="olink">DB->set_re_delim()</a> method. For example, standard UNIX byte + stream files can be interpreted as a sequence of variable + length records separated by ASCII newline characters. This is + the default. + </p> + <p> + When cached data would normally be written back to the + underlying database file (for example, when the <a href="../api_reference/C/dbclose.html" class="olink">DB->close()</a> or + <a href="../api_reference/C/dbsync.html" class="olink">DB->sync()</a> methods are called), the in-memory copy of the + database will be written back to the backing source + file. + </p> + <p> + The backing source file must already exist (but may be + zero-length) when <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> is called. By default, the backing + source file is read lazily, that is, records are not read from + the backing source file until they are requested by the + application. If multiple processes (not threads) are accessing + a Recno database concurrently and either inserting or deleting + records, the backing source file must be read in its entirety + before more than a single process accesses the database, and + only that process should specify the backing source file as + part of the <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> call. This can be accomplished by calling + the <a href="../api_reference/C/dbset_flags.html" class="olink">DB->set_flags()</a> method with the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_SNAPSHOT" class="olink">DB_SNAPSHOT</a> flag. + </p> + <p> + Reading and writing the backing source file cannot be + transactionally protected because it involves filesystem + operations that are not part of the Berkeley DB transaction + methodology. For this reason, if a temporary database is used + to hold the records (a NULL was specified as the file argument + to <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a>), <span class="bold"><strong>it is possible to lose the + contents of the backing source file if the system crashes + at the right instant</strong></span>. If a permanent file is + used to hold the database (a filename was specified as the + file argument to <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a>), normal database recovery on that + file can be used to prevent information loss. It is still + possible that the contents of the backing source file itself + will be corrupted or lost if the system crashes. + </p> + <p> + For all of the above reasons, the backing source file is + generally used to specify databases that are read-only for + Berkeley DB applications, and that are either generated on the + fly by software tools, or modified using a different mechanism + such as a text editor. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="am_conf_renumber"></a>Logically renumbering records</h3> + <h3 class="title"><a id="am_conf_renumber"></a>Logically renumbering + records</h3> </div> </div> </div> - <p>Records stored in the Queue and Recno access methods are accessed by -logical record number. In all cases in Btree databases, and optionally -in Recno databases (see the <a href="../api_reference/C/dbset_flags.html" class="olink">DB->set_flags()</a> method and the -<a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> flag for more information), record numbers are -mutable. This means that the record numbers may change as records are -added to and deleted from the database. The deletion of record number -4 causes any records numbered 5 and higher to be renumbered downward by -1; the addition of a new record after record number 4 causes any -records numbered 5 and higher to be renumbered upward by 1. In all -cases in Queue databases, and by default in Recno databases, record -numbers are not mutable, and the addition or deletion of records to the -database will not cause already-existing record numbers to change. For -this reason, new records cannot be inserted between already-existing -records in databases with immutable record numbers.</p> - <p>Cursors pointing into a Btree database or a Recno database with mutable -record numbers maintain a reference to a specific record, rather than -a record number, that is, the record they reference does not change as -other records are added or deleted. For example, if a database contains -three records with the record numbers 1, 2, and 3, and the data items -"A", "B", and "C", respectively, the deletion of record number 2 ("B") -will cause the record "C" to be renumbered downward to record number 2. -A cursor positioned at record number 3 ("C") will be adjusted and -continue to point to "C" after the deletion. Similarly, a cursor -previously referring to the now deleted record number 2 will be -positioned between the new record numbers 1 and 2, and an insertion -using that cursor will appear between those records. In this manner -records can be added and deleted to a database without disrupting the -sequential traversal of the database by a cursor.</p> - <p>Only cursors created using a single <a href="../api_reference/C/db.html" class="olink">DB</a> handle can adjust each -other's position in this way, however. If multiple <a href="../api_reference/C/db.html" class="olink">DB</a> handles -have a renumbering Recno database open simultaneously (as when multiple -processes share a single database environment), a record referred to by -one cursor could change underfoot if a cursor created using another -<a href="../api_reference/C/db.html" class="olink">DB</a> handle inserts or deletes records into the database. For -this reason, applications using Recno databases with mutable record -numbers will usually make all accesses to the database using a single -<a href="../api_reference/C/db.html" class="olink">DB</a> handle and cursors created from that handle, or will -otherwise single-thread access to the database, for example, by using -the Berkeley DB Concurrent Data Store product.</p> - <p>In any Queue or Recno databases, creating new records will cause the -creation of multiple records if the record number being created is more -than one greater than the largest record currently in the database. For -example, creating record number 28, when record 25 was previously the -last record in the database, will implicitly create records 26 and 27 -as well as 28. All first, last, next and previous cursor operations -will automatically skip over these implicitly created records. So, if -record number 5 is the only record the application has created, -implicitly creating records 1 through 4, the <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> method with the -<a href="../api_reference/C/dbcget.html#dbcget_DB_FIRST" class="olink">DB_FIRST</a> flag will return record number 5, not record number 1. -Attempts to explicitly retrieve implicitly created records by their -record number will result in a special error return, -<a class="link" href="program_errorret.html#program_errorret.DB_KEYEMPTY">DB_KEYEMPTY</a>.</p> - <p>In any Berkeley DB database, attempting to retrieve a deleted record, using -a cursor positioned on the record, results in a special error return, -<a class="link" href="program_errorret.html#program_errorret.DB_KEYEMPTY">DB_KEYEMPTY</a>. In addition, when using Queue databases or Recno -databases with immutable record numbers, attempting to retrieve a deleted -record by its record number will also result in the <a class="link" href="program_errorret.html#program_errorret.DB_KEYEMPTY">DB_KEYEMPTY</a> -return.</p> + <p> + Records stored in the Queue and Recno access methods are + accessed by logical record number. In all cases in Btree + databases, and optionally in Recno databases (see the + <a href="../api_reference/C/dbset_flags.html" class="olink">DB->set_flags()</a> method and the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> flag for more + information), record numbers are mutable. This means that the + record numbers may change as records are added to and deleted + from the database. The deletion of record number 4 causes any + records numbered 5 and higher to be renumbered downward by 1; + the addition of a new record after record number 4 causes any + records numbered 5 and higher to be renumbered upward by 1. In + all cases in Queue databases, and by default in Recno + databases, record numbers are not mutable, and the addition or + deletion of records to the database will not cause + already-existing record numbers to change. For this reason, + new records cannot be inserted between already-existing + records in databases with immutable record numbers. + </p> + <p> + Cursors pointing into a Btree database or a Recno database + with mutable record numbers maintain a reference to a specific + record, rather than a record number, that is, the record they + reference does not change as other records are added or + deleted. For example, if a database contains three records + with the record numbers 1, 2, and 3, and the data items "A", + "B", and "C", respectively, the deletion of record number 2 + ("B") will cause the record "C" to be renumbered downward to + record number 2. A cursor positioned at record number 3 ("C") + will be adjusted and continue to point to "C" after the + deletion. Similarly, a cursor previously referring to the now + deleted record number 2 will be positioned between the new + record numbers 1 and 2, and an insertion using that cursor + will appear between those records. In this manner records can + be added and deleted to a database without disrupting the + sequential traversal of the database by a cursor. + </p> + <p> + Only cursors created using a single <a href="../api_reference/C/db.html" class="olink">DB</a> handle can adjust + each other's position in this way, however. If multiple <a href="../api_reference/C/db.html" class="olink">DB</a> + handles have a renumbering Recno database open simultaneously + (as when multiple processes share a single database + environment), a record referred to by one cursor could change + underfoot if a cursor created using another <a href="../api_reference/C/db.html" class="olink">DB</a> handle + inserts or deletes records into the database. For this reason, + applications using Recno databases with mutable record numbers + will usually make all accesses to the database using a single + <a href="../api_reference/C/db.html" class="olink">DB</a> handle and cursors created from that handle, or will + otherwise single-thread access to the database, for example, + by using the Berkeley DB Concurrent Data Store product. + </p> + <p> + In any Queue or Recno databases, creating new records will + cause the creation of multiple records if the record number + being created is more than one greater than the largest record + currently in the database. For example, creating record number + 28, when record 25 was previously the last record in the + database, will implicitly create records 26 and 27 as well as + 28. All first, last, next and previous cursor operations will + automatically skip over these implicitly created records. So, + if record number 5 is the only record the application has + created, implicitly creating records 1 through 4, the <a href="../api_reference/C/dbcget.html" class="olink">DBC->get()</a> + method with the <a href="../api_reference/C/dbcget.html#dbcget_DB_FIRST" class="olink">DB_FIRST</a> flag will return record number 5, + not record number 1. Attempts to explicitly retrieve + implicitly created records by their record number will result + in a special error return, <a class="link" href="program_errorret.html#program_errorret.DB_KEYEMPTY">DB_KEYEMPTY</a>. + </p> + <p> + In any Berkeley DB database, attempting to retrieve a + deleted record, using a cursor positioned on the record, + results in a special error return, <a class="link" href="program_errorret.html#program_errorret.DB_KEYEMPTY">DB_KEYEMPTY</a>. + In addition, when using Queue databases or Recno databases + with immutable record numbers, attempting to retrieve a + deleted record by its record number will also result in the + <a class="link" href="program_errorret.html#program_errorret.DB_KEYEMPTY">DB_KEYEMPTY</a> return. + </p> </div> </div> <div class="navfooter"> @@ -314,13 +381,12 @@ return.</p> <td width="40%" align="right"> <a accesskey="n" href="am.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Heap access method specific configuration </td> + <td width="40%" align="left" valign="top">Heap access method specific + configuration </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 3. - Access Method Operations - </td> + <td width="40%" align="right" valign="top"> Chapter 3. Access Method Operations </td> </tr> </table> </div> diff --git a/docs/programmer_reference/second.javas b/docs/programmer_reference/second.javas index 1e544773..a2f536f3 100644 --- a/docs/programmer_reference/second.javas +++ b/docs/programmer_reference/second.javas @@ -1,5 +1,5 @@ /*- - * Copyright (c) 2002,2009 Oracle. All rights reserved. + * Copyright (c) 2002, 2014 Oracle. All rights reserved. * */ package db; diff --git a/docs/programmer_reference/sequence.html b/docs/programmer_reference/sequence.html index ed259fb4..8d51075d 100644 --- a/docs/programmer_reference/sequence.html +++ b/docs/programmer_reference/sequence.html @@ -14,13 +14,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 20. - Sequences - </th> + <th colspan="3" align="center">Chapter 20. Sequences </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="txn_limits.html">Prev</a> </td> @@ -34,36 +32,50 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="sequence"></a>Chapter 20. - Sequences - </h2> + <h2 class="title"><a id="sequence"></a>Chapter 20. Sequences </h2> </div> </div> </div> - <p>Sequences provide an arbitrary number of persistent objects that return -an increasing or decreasing sequence of integers. Opening a sequence -handle associates it with a record in a database. The handle can -maintain a cache of values from the database so that a database update -is not needed as the application allocates a value.</p> - <p>A sequence is stored as a record pair in a database. The database may -be of any type, but may not have been configured to support duplicate -data items. The sequence is referenced by the key used when the -sequence is created, therefore the key must be compatible with the -underlying access method. If the database stores fixed-length records, -the record size must be at least 64 bytes long.</p> - <p>Since a sequence handle is opened using a database handle, the use of -transactions with the sequence must follow how the database handle was -opened. In other words, if the database handle was opened within a -transaction, operations on the sequence handle must use transactions. -Of course, if sequences are cached, not all operations will actually -trigger a transaction.</p> - <p>For the highest concurrency, caching should be used and the -<a href="../api_reference/C/envset_flags.html#envset_flags_DB_AUTO_COMMIT" class="olink">DB_AUTO_COMMIT</a> and <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> flags should be -specified to the <a href="../api_reference/C/seqget.html" class="olink">DB_SEQUENCE->get()</a> method call. If the allocation of the -sequence value must be part of a transaction, and rolled back if the -transaction aborts, then no caching should be specified and the -transaction handle must be passed to the <a href="../api_reference/C/seqget.html" class="olink">DB_SEQUENCE->get()</a> method.</p> - <p>For more information on the operations supported by the sequence handle, see the <a href="../api_reference/C/seq.html#seqlist" class="olink">Sequences and Related Methods</a> section in the <em class="citetitle">Berkeley DB C API Reference Guide.</em> </p> + <p> + Sequences provide an arbitrary number of persistent objects + that return an increasing or decreasing sequence of integers. + Opening a sequence handle associates it with a record in a + database. The handle can maintain a cache of values from the + database so that a database update is not needed as the + application allocates a value. + </p> + <p> + A sequence is stored as a record pair in a database. The + database may be of any type, but may not have been configured + to support duplicate data items. The sequence is referenced by + the key used when the sequence is created, therefore the key + must be compatible with the underlying access method. If the + database stores fixed-length records, the record size must be + at least 64 bytes long. + </p> + <p> + Since a sequence handle is opened using a database handle, + the use of transactions with the sequence must follow how the + database handle was opened. In other words, if the database + handle was opened within a transaction, operations on the + sequence handle must use transactions. Of course, if sequences + are cached, not all operations will actually trigger a + transaction. + </p> + <p> + For the highest concurrency, caching should be used and the + <a href="../api_reference/C/envset_flags.html#envset_flags_DB_AUTO_COMMIT" class="olink">DB_AUTO_COMMIT</a> and <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> flags should be specified + to the <a href="../api_reference/C/seqget.html" class="olink">DB_SEQUENCE->get()</a> method call. If the allocation of the sequence + value must be part of a transaction, and rolled back if the + transaction aborts, then no caching should be specified and + the transaction handle must be passed to the <a href="../api_reference/C/seqget.html" class="olink">DB_SEQUENCE->get()</a> + method. + </p> + <p> + For more information on the operations supported by the + sequence handle, see the <a href="../api_reference/C/seq.html#seqlist" class="olink">Sequences and Related Methods</a> + section in the <em class="citetitle">Berkeley DB C API Reference Guide.</em> + </p> </div> <div class="navfooter"> <hr /> @@ -78,9 +90,7 @@ transaction handle must be passed to the <a href="../api_reference/C/seqget.html <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 21. - Berkeley DB Extensions: Tcl - </td> + <td width="40%" align="right" valign="top"> Chapter 21. Berkeley DB Extensions: Tcl </td> </tr> </table> </div> diff --git a/docs/programmer_reference/stl.html b/docs/programmer_reference/stl.html index 220d1b23..f4f14488 100644 --- a/docs/programmer_reference/stl.html +++ b/docs/programmer_reference/stl.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -55,7 +55,8 @@ </dt> <dt> <span class="sect2"> - <a href="stl.html#stl_intro_performance">Performance overhead</a> + <a href="stl.html#stl_intro_performance">Performance + overhead</a> </span> </dt> <dt> @@ -84,51 +85,52 @@ <dl> <dt> <span class="sect2"> - <a href="stl_db_usage.html#idp1202384">Registering database and environment handles</a> + <a href="stl_db_usage.html#idp756272">Registering database and environment handles</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_db_usage.html#idp1225928">Truncate requirements</a> + <a href="stl_db_usage.html#idp772680">Truncate requirements</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_db_usage.html#idp1236168">Auto commit support</a> + <a href="stl_db_usage.html#idp773680">Auto commit support</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_db_usage.html#idp1239792">Database and environment identity checks</a> + <a href="stl_db_usage.html#idp792728">Database and environment identity checks</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_db_usage.html#idp1236512">Products, constructors and configurations</a> + <a href="stl_db_usage.html#idp795408">Products, constructors and configurations</a> </span> </dt> </dl> </dd> <dt> <span class="sect1"> - <a href="stl_db_advanced_usage.html">Using advanced Berkeley DB features with dbstl</a> + <a href="stl_db_advanced_usage.html">Using advanced Berkeley DB + features with dbstl</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="stl_db_advanced_usage.html#idp1232296">Using bulk retrieval iterators</a> + <a href="stl_db_advanced_usage.html#idp672128">Using bulk retrieval iterators</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_db_advanced_usage.html#idp1232520">Using the DB_RMW flag</a> + <a href="stl_db_advanced_usage.html#idp794736">Using the DB_RMW flag</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_db_advanced_usage.html#idp1199288">Using secondary index database and secondary containers</a> + <a href="stl_db_advanced_usage.html#idp773392">Using secondary index database and secondary containers</a> </span> </dt> </dl> @@ -140,7 +142,8 @@ </dt> <dt> <span class="sect1"> - <a href="stl_mt_usage.html">Using dbstl in multithreaded applications</a> + <a href="stl_mt_usage.html">Using dbstl in multithreaded + applications</a> </span> </dt> <dt> @@ -152,31 +155,32 @@ <dl> <dt> <span class="sect2"> - <a href="stl_primitive_rw.html#idp1288424">Storing strings</a> + <a href="stl_primitive_rw.html#idp849056">Storing strings</a> </span> </dt> </dl> </dd> <dt> <span class="sect1"> - <a href="stl_complex_rw.html">Store and Retrieve data or objects of complex types </a> + <a href="stl_complex_rw.html">Store and Retrieve data or + objects of complex types </a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="stl_complex_rw.html#idp1279008">Storing varying length objects</a> + <a href="stl_complex_rw.html#idp808056">Storing varying length objects</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_complex_rw.html#idp1278616">Storing arbitrary sequences</a> + <a href="stl_complex_rw.html#idp859120">Storing arbitrary sequences</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_complex_rw.html#idp1344912">Notes</a> + <a href="stl_complex_rw.html#idp909008">Notes</a> </span> </dt> </dl> @@ -207,38 +211,40 @@ </dd> <dt> <span class="sect1"> - <a href="stl_container_specific.html">Dbstl container specific notes</a> + <a href="stl_container_specific.html">Dbstl container specific + notes</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="stl_container_specific.html#idp1313840">db_vector specific notes</a> + <a href="stl_container_specific.html#idp873704">db_vector specific notes</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_container_specific.html#idp1381768">Associative container specific notes</a> + <a href="stl_container_specific.html#idp948648">Associative container specific notes</a> </span> </dt> </dl> </dd> <dt> <span class="sect1"> - <a href="stl_efficienct_use.html">Using dbstl efficiently</a> + <a href="stl_efficienct_use.html">Using dbstl + efficiently</a> </span> </dt> <dd> <dl> <dt> <span class="sect2"> - <a href="stl_efficienct_use.html#idp1350664">Using iterators efficiently</a> + <a href="stl_efficienct_use.html#idp915608">Using iterators efficiently</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_efficienct_use.html#idp1350448">Using containers efficiently</a> + <a href="stl_efficienct_use.html#idp957584">Using containers efficiently</a> </span> </dt> </dl> @@ -252,12 +258,12 @@ <dl> <dt> <span class="sect2"> - <a href="stl_memory_mgmt.html#idp1384984">Freeing memory</a> + <a href="stl_memory_mgmt.html#idp952296">Freeing memory</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_memory_mgmt.html#idp1389512">Type specific notes</a> + <a href="stl_memory_mgmt.html#idp994336">Type specific notes</a> </span> </dt> </dl> @@ -271,12 +277,13 @@ <dl> <dt> <span class="sect2"> - <a href="stl_misc.html#idp1407848">Special notes about trivial methods</a> + <a href="stl_misc.html#idp953048">Special notes about trivial methods</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_misc.html#idp1421568">Using correct container and iterator public types</a> + <a href="stl_misc.html#idp993208">Using correct container and iterator public + types</a> </span> </dt> </dl> @@ -305,7 +312,8 @@ </dt> <dt> <span class="sect2"> - <a href="stl.html#stl_intro_performance">Performance overhead</a> + <a href="stl.html#stl_intro_performance">Performance + overhead</a> </span> </dt> <dt> @@ -315,23 +323,26 @@ </dt> </dl> </div> + <p> + Dbstl is a C++ STL style API that provides for Berkeley DB + usage. It allows for the storage and retrieval of data/objects + of any type using Berkeley DB databases, but with an interface + that mimics that of C++ STL containers. Dbstl provides access + to all of the functionality of Berkeley DB available via this + STL-style API. + </p> <p> - Dbstl is a C++ STL style API that provides for Berkeley DB usage. It - allows for the storage and retrieval of data/objects of any type using - Berkeley DB databases, but with an interface that mimics that of C++ - STL containers. Dbstl provides access to all of the functionality of - Berkeley DB available via this STL-style API. -</p> - <p> - With proper configuration, dbstl is able to store/retrieve any complex - data types. There is no need to perform repetitive marshalling and - unmarshalling of data. Dbstl also properly manages the life-cycle of - all Berkeley DB structures and objects. All example methods referred to in this chapter -can be found in the StlAdvancedFeaturesExample class in the $DbSrc/examples_stl/StlAdvancedFeatures.cpp file, and you can -build the example in $DbSrc/build_unix directory like this: -make exstl_advancedfeatures, -where DbSrc is the source directory for Berkeley DB. -</p> + With proper configuration, dbstl is able to store/retrieve + any complex data types. There is no need to perform repetitive + marshalling and unmarshalling of data. Dbstl also properly + manages the life-cycle of all Berkeley DB structures and + objects. All example methods referred to in this chapter can + be found in the StlAdvancedFeaturesExample class in the + $DbSrc/examples/stl/StlAdvancedFeatures.cpp file, and you can + build the example in $DbSrc/build_unix directory like this: + make exstl_advancedfeatures, where DbSrc is the source + directory for Berkeley DB. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> @@ -341,33 +352,37 @@ where DbSrc is the source directory for Berkeley DB. </div> </div> <p> - Dbstl is composed of many container and iterator class templates. These - containers and iterators correspond exactly to each container and - iterator available in the C++ STL API, including identical sets of - methods. This allows existing algorithms, functions and - container-adapters for C++ STL to use dbstl containers through its - standard iterators. This means that existing STL code can manipulate - Berkeley DB databases. As a result, existing C++ STL code can very - easily use dbstl to gain persistence and transaction guarantees. -</p> + Dbstl is composed of many container and iterator class + templates. These containers and iterators correspond + exactly to each container and iterator available in the + C++ STL API, including identical sets of methods. This + allows existing algorithms, functions and + container-adapters for C++ STL to use dbstl containers + through its standard iterators. This means that existing + STL code can manipulate Berkeley DB databases. As a + result, existing C++ STL code can very easily use dbstl to + gain persistence and transaction guarantees. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="stl_intro_performance"></a>Performance overhead</h3> + <h3 class="title"><a id="stl_intro_performance"></a>Performance + overhead</h3> </div> </div> </div> <p> - Because dbstl uses C++ template technologies, its performance overhead - is minimal. -</p> + Because dbstl uses C++ template technologies, its + performance overhead is minimal. + </p> <p> - The dbstl API performs almost equally to the C API, as measured by two - different implementations of the TPC-B benchmark: - <code class="literal">ex_tpcb</code> and <code class="literal">exstl_tpcb</code>. -</p> + The dbstl API performs almost equally to the C API, as + measured by two different implementations of the TPC-B + benchmark: <code class="literal">ex_tpcb</code> and + <code class="literal">exstl_tpcb</code>. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> @@ -378,86 +393,89 @@ where DbSrc is the source directory for Berkeley DB. </div> </div> <p> - The degree to which dbstl is portable to a new platform is determined - by whether Berkeley DB is available on the platform, as well as whether - an appropriate C++ compiler is available on the platform. -</p> - <p> - For information on porting Berkeley DB to new platforms, see the - <em class="citetitle">Berkeley DB Porting Guide</em>. -</p> - <p> - Almost all the advanced C++ template features are used in dbstl, including: -</p> + The degree to which dbstl is portable to a new platform + is determined by whether Berkeley DB is available on the + platform, as well as whether an appropriate C++ compiler + is available on the platform. + </p> + <p> + For information on porting Berkeley DB to new + platforms, see the <em class="citetitle">Berkeley DB Porting Guide</em>. + </p> + <p> + Almost all the advanced C++ template features are used + in dbstl, including: + </p> <div class="itemizedlist"> <ul type="disc"> <li> <p> - member function templates - </p> + member function templates + </p> </li> <li> - <p> - member function template overloads - </p> + <p> + member function template overloads + </p> </li> <li> - <p> - partial specialization - </p> + <p> + partial specialization + </p> </li> <li> <p> - default template parameters. - </p> + default template parameters. + </p> </li> </ul> </div> - <p> - For this reason, you need a standards-compatible C++ compiler to - build dbstl. As of this writing, the following compilers are known - to build dbstl successfully: - </p> + <p> + For this reason, you need a standards-compatible C++ + compiler to build dbstl. As of this writing, the following + compilers are known to build dbstl successfully: + </p> <div class="itemizedlist"> <ul type="disc"> <li> - <p> - MSVC8 - </p> + <p> + MSVC8 + </p> </li> <li> - <p> - gcc3.4.4 and above - </p> + <p> + gcc3.4.4 and above + </p> </li> <li> - <p> - Intel C++ 9 and above - </p> + <p> + Intel C++ 9 and above + </p> </li> </ul> </div> <p> - For *nix platforms, if you can successfully configure your Berkeley - DB build script with <code class="literal">--enable-stl</code>, then you - should be able to successfully build dbstl library and application - code using it. - </p> + For *nix platforms, if you can successfully configure + your Berkeley DB build script with + <code class="literal">--enable-stl</code>, then you should be + able to successfully build dbstl library and application + code using it. + </p> <p> - Besides its own test suite, dbstl has also been tested against, - and passes, the following test suites: - </p> + Besides its own test suite, dbstl has also been tested + against, and passes, the following test suites: + </p> <div class="itemizedlist"> <ul type="disc"> <li> <p> - MS STL test suite - </p> + MS STL test suite + </p> </li> <li> <p> - SGI STL test suite - </p> + SGI STL test suite + </p> </li> </ul> </div> diff --git a/docs/programmer_reference/stl_complex_rw.html b/docs/programmer_reference/stl_complex_rw.html index d173b63f..353bbbdd 100644 --- a/docs/programmer_reference/stl_complex_rw.html +++ b/docs/programmer_reference/stl_complex_rw.html @@ -14,11 +14,12 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Store and Retrieve data or objects of complex types </th> + <th colspan="3" align="center">Store and Retrieve data or + objects of complex types </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="stl_primitive_rw.html">Prev</a> </td> @@ -32,7 +33,8 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="stl_complex_rw"></a>Store and Retrieve data or objects of complex types </h2> + <h2 class="title" style="clear: both"><a id="stl_complex_rw"></a>Store and Retrieve data or + objects of complex types </h2> </div> </div> </div> @@ -40,17 +42,17 @@ <dl> <dt> <span class="sect2"> - <a href="stl_complex_rw.html#idp1279008">Storing varying length objects</a> + <a href="stl_complex_rw.html#idp808056">Storing varying length objects</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_complex_rw.html#idp1278616">Storing arbitrary sequences</a> + <a href="stl_complex_rw.html#idp859120">Storing arbitrary sequences</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_complex_rw.html#idp1344912">Notes</a> + <a href="stl_complex_rw.html#idp909008">Notes</a> </span> </dt> </dl> @@ -59,46 +61,51 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1279008"></a>Storing varying length objects</h3> + <h3 class="title"><a id="idp808056"></a>Storing varying length objects</h3> </div> </div> </div> <p> - A structure like this: -</p> + A structure like this: + </p> <pre class="programlisting">class SMSMsg { public: - size_t mysize; - time_t when; - size_t szmsg; - int to; - char msg[1]; + size_t mysize; + time_t when; + size_t szmsg; + int to; + char msg[1]; }; </pre> <p> - with a varying length string in <code class="literal">msg</code> cannot simply be - stored in a <code class="literal">db_vector<SMSMsg></code> without some - configuration on your part. This is because, by default, dbstl uses the - <span class="bold"><strong>sizeof()</strong></span> operator to get the size of - an object and then <code class="function">memcpy()</code> to copy the object. This - process is not suitable for this use-case as it will fail to capture the - variable length string contained in <code class="literal">msg</code>. -</p> - <p> - There are currently two ways to store these kind of objects: -</p> + with a varying length string in <code class="literal">msg</code> + cannot simply be stored in a + <code class="literal">db_vector<SMSMsg></code> without + some configuration on your part. This is because, by + default, dbstl uses the <span class="bold"><strong>sizeof()</strong></span> operator + to get the size of an object and then <code class="function">memcpy()</code> to copy the + object. This process is not suitable for this use-case as + it will fail to capture the variable length string + contained in <code class="literal">msg</code>. + </p> + <p> + There are currently two ways to store these kind of + objects: + </p> <div class="orderedlist"> <ol type="1"> <li> <p> - Register callback functions with dbstl that are used to measure an object's size, - and then marshal/unmarshal the object. - </p> + Register callback functions with dbstl that are + used to measure an object's size, and then + marshal/unmarshal the object. + </p> </li> <li> <p> - Use a <code class="classname">DbstlDbt</code> wrapper object. - </p> + Use a <code class="classname">DbstlDbt</code> wrapper + object. + </p> </li> </ol> </div> @@ -106,280 +113,380 @@ public: <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idp1264432"></a>Storing by marshaling objects</h4> + <h4 class="title"><a id="idp838336"></a>Storing by marshaling objects</h4> </div> </div> </div> - <p> - One way to store an object that contains variable-sized fields is - to marshall all of the object's data into a single contiguous area - in memory, and then store the contents of that buffer. This means that upon retrieval, the - contents of the buffer must be unmarshalled. To do these things, - you must register three callback functions: - </p> + <p> + One way to store an object that contains + variable-sized fields is to marshall all of the + object's data into a single contiguous area in memory, + and then store the contents of that buffer. This means + that upon retrieval, the contents of the buffer must + be unmarshalled. To do these things, you must register + three callback functions: + </p> <div class="itemizedlist"> <ul type="disc"> <li> <p> - <code class="function">typedef void (*ElemRstoreFunct)(T& dest, const void *srcdata);</code> - </p> + <code class="function">typedef void + (*ElemRstoreFunct)(T& dest, const void + *srcdata);</code> + </p> <p> - This callback is used to unmarshal an object, updating <span class="bold"><strong>dest</strong></span> - using data found in <span class="bold"><strong>srcdata</strong></span>. The data in - <span class="bold"><strong>srcdata </strong></span> contains the chunk of memory into which the - object was originally marshalled. The default unmarshalling function simply performs a cast - (for example, <code class="literal">dest = *((T*)srcdata)</code>), which assumes the - <span class="bold"><strong>srcdata</strong></span> simply points to the memory layout of the object. - </p> + This callback is used to unmarshal an + object, updating <span class="bold"><strong>dest</strong></span> using + data found in <span class="bold"><strong>srcdata</strong></span>. + The data in <span class="bold"><strong>srcdata + </strong></span> contains the chunk of memory into + which the object was originally marshalled. + The default unmarshalling function simply + performs a cast (for example, <code class="literal">dest = + *((T*)srcdata)</code>), which assumes + the <span class="bold"><strong>srcdata</strong></span> + simply points to the memory layout of the + object. + </p> </li> <li> <p> - <code class="function">typedef size_t (*ElemSizeFunct)(const T& elem);</code> - </p> - <p> - This callback returns the size in bytes needed to store the - <span class="bold"><strong>elem</strong></span> object. By default this function - simply uses <span class="bold"><strong>sizeof(elem)</strong></span> to determine - the size of <span class="bold"><strong>elem</strong></span>. - </p> + <code class="function">typedef size_t + (*ElemSizeFunct)(const T& + elem);</code> + </p> + <p> + This callback returns the size in bytes + needed to store the <span class="bold"><strong>elem</strong></span> object. + By default this function simply uses <span class="bold"><strong>sizeof(elem)</strong></span> + to determine the size of <span class="bold"><strong>elem</strong></span>. + </p> </li> <li> <p> - <code class="function">typedef void (*ElemCopyFunct)(void *dest, const T&elem);</code> - </p> - <p> - This callback is used to arrange all data contained by <span class="bold"><strong>elem</strong></span> - into the chunk of memory to which <span class="bold"><strong>dest</strong></span> refers. The size of - <span class="bold"><strong>dest</strong></span> is set by the <code class="function">ElemSizeFunct</code> - function, discussed above. The default marshalling function simply uses - <code class="function">memcpy()</code> to copy <span class="bold"><strong>elem</strong></span> to - <span class="bold"><strong>dest</strong></span>. - </p> + <code class="function">typedef void (*ElemCopyFunct)(void + *dest, const T&elem);</code> + </p> + <p> + This callback is used to arrange all data + contained by <span class="bold"><strong>elem</strong></span> into + the chunk of memory to which <span class="bold"><strong>dest</strong></span> + refers. The size of <span class="bold"><strong>dest</strong></span> is set + by the <code class="function">ElemSizeFunct</code> + function, discussed above. The default + marshalling function simply uses + <code class="function">memcpy()</code> to copy + <span class="bold"><strong>elem</strong></span> to + <span class="bold"><strong>dest</strong></span>. + </p> </li> </ul> </div> - <p> - The <code class="function">DbstlElemTraits<SMSMsg>::instance()->set_size_function()</code>, - <code class="function">set_copy_function()</code> and <code class="function">set_restore_function()</code> methods - are used to register these callback functions. If a callback is not registered, its - default function is used. -</p> - <p> - By providing non-default implementations of the callbacks described here, you can store objects - of varying length and/or objects which do not reside in a continuous memory chunk — for - example, objects containing a pointer which refers another object, or a string, and so forth. - As a result, containers/iterators can manage variable length objects in the same as they would - manage objects that reside in continuous chunks of memory and are of identical size. -</p> + <p> + The + <code class="function">DbstlElemTraits<SMSMsg>::instance()->set_size_function()</code>, + <code class="function">set_copy_function()</code> and + <code class="function">set_restore_function()</code> + methods are used to register these callback functions. + If a callback is not registered, its default function + is used. + </p> + <p> + By providing non-default implementations of the + callbacks described here, you can store objects of + varying length and/or objects which do not reside in a + continuous memory chunk — for example, objects + containing a pointer which refers another object, or a + string, and so forth. As a result, + containers/iterators can manage variable length + objects in the same as they would manage objects that + reside in continuous chunks of memory and are of + identical size. + </p> </div> <div class="sect3" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idp1266088"></a>Using a <code class="classname">DbstlDbt</code> wrapper object</h4> + <h4 class="title"><a id="idp880480"></a>Using a <code class="classname">DbstlDbt</code> wrapper + object</h4> </div> </div> </div> <p> - To use a <code class="classname">DbstlDbt</code> wrapper object to store objects of variable length, a - <code class="literal">db_vector<DbstlDbt></code> container is used to store complex objects in a - <code class="classname">db_vector</code>. <code class="classname">DbstlDbt</code> derives from DB C++ API's - <code class="classname">Dbt</code>class, but can manage its referenced memory properly and release it - upon destruction. The memory referenced by <code class="classname">DbstlDbt</code> objects is required - to be allocated using the <code class="function">malloc()</code>/<code class="function">realloc()</code> functions - from the standard C library. -</p> - <p> - Note that the use of <code class="classname">DbstlDbt</code> wrapper class is not ideal. It exists only - to allow raw bytes of no specific type to be stored in a container. -</p> - <p> - To store an <code class="classname">SMSMsg</code> object into a <code class="literal">db_vector<DbstlDbt></code> - container using a <code class="classname">DbstlDbt</code> object: -</p> + To use a <code class="classname">DbstlDbt</code> wrapper + object to store objects of variable length, a + <code class="literal">db_vector<DbstlDbt></code> + container is used to store complex objects in a + <code class="classname">db_vector</code>. + <code class="classname">DbstlDbt</code> derives from DB + C++ API's <code class="classname">Dbt</code>class, but can + manage its referenced memory properly and release it + upon destruction. The memory referenced by + <code class="classname">DbstlDbt</code> objects is + required to be allocated using the + <code class="function">malloc()</code>/<code class="function">realloc()</code> + functions from the standard C library. + </p> + <p> + Note that the use of + <code class="classname">DbstlDbt</code> wrapper class is + not ideal. It exists only to allow raw bytes of no + specific type to be stored in a container. + </p> + <p> + To store an <code class="classname">SMSMsg</code> object + into a <code class="literal">db_vector<DbstlDbt></code> + container using a <code class="classname">DbstlDbt</code> + object: + </p> <div class="orderedlist"> <ol type="1"> - <li> - Wrap the <code class="classname">SMSMSg</code> object into a <code class="classname">DbstlDbt</code> object, - then marshal the SMSMsg object properly into the memory chunk referenced by - <code class="methodname">DbstlDbt::data</code>. -</li> - <li> - Store the <code class="classname">DbstlDbt</code> object into a <code class="literal">db_vector<DbstlDbt></code> - container. The bytes in the memory chunk referenced by the <code class="classname">DbstlDbt</code> object's - <span class="bold"><strong>data</strong></span> member are stored in the - <code class="literal">db_vector<DbstlDbt></code> container. -</li> - <li> - Reading from the container returns a <code class="classname">DbstlDbt</code> object whose - <span class="bold"><strong>data</strong></span> field points to the <code class="classname">SMSMsg</code> object - located in a continuous chunk of memory. The application needs to perform its own unmarshalling. -</li> - <li> - The memory referenced by <code class="literal">DbstlDbt::data</code> is freed automatically, - and so the application should not attempt to free the memory. -</li> + <li> + Wrap the <code class="classname">SMSMSg</code> + object into a <code class="classname">DbstlDbt</code> + object, then marshal the SMSMsg object properly + into the memory chunk referenced by + <code class="methodname">DbstlDbt::data</code>. + </li> + <li> + Store the <code class="classname">DbstlDbt</code> + object into a + <code class="literal">db_vector<DbstlDbt></code> + container. The bytes in the memory chunk + referenced by the <code class="classname">DbstlDbt</code> + object's <span class="bold"><strong>data</strong></span> + member are stored in the + <code class="literal">db_vector<DbstlDbt></code> + container. + </li> + <li> + Reading from the container returns a + <code class="classname">DbstlDbt</code> object whose + <span class="bold"><strong>data</strong></span> field + points to the <code class="classname">SMSMsg</code> object + located in a continuous chunk of memory. The + application needs to perform its own + unmarshalling. + </li> + <li> + The memory referenced by + <code class="literal">DbstlDbt::data</code> is freed + automatically, and so the application should not + attempt to free the memory. + </li> </ol> </div> <p> - <code class="classname">ElementHolder</code> should not be used to store objects of a class because it - doesn't support access to object members using <span class="bold"><strong>(*iter).member</strong></span> - or <span class="bold"><strong>iter->member</strong></span> expressions. In this case, the default - <code class="literal">ElementRef<ddt></code> is used automatically. -</p> - <p> - <code class="classname">ElementRef</code> inherits from <code class="classname">ddt</code>, which allows - <span class="bold"><strong>*iter</strong></span> to return the object stored in the container. - (Technically it is an <code class="classname">ElementRef<ddt> object</code>, whose "base class" - part is the object you stored). There are a few data members and member functions in - <code class="classname">ElementRef</code>, which all start with <code class="literal">_DB_STL_</code>. To avoid - potential name clashes, applications should not use names prefixing <code class="literal">_DB_STL_</code> - in classes whose instances may be stored into dbstl containers. -</p> + <code class="classname">ElementHolder</code> should not be + used to store objects of a class because it doesn't + support access to object members using <span class="bold"><strong>(*iter).member</strong></span> or <span class="bold"><strong>iter->member</strong></span> + expressions. In this case, the default + <code class="literal">ElementRef<ddt></code> is used + automatically. + </p> <p> - Example code demonstrating this feature can be found in the - <code class="methodname">StlAdvancedFeaturesExample::arbitrary_object_storage</code> method. -</p> + <code class="classname">ElementRef</code> inherits from + <code class="classname">ddt</code>, which allows <span class="bold"><strong>*iter</strong></span> to return the object + stored in the container. (Technically it is an + <code class="classname">ElementRef<ddt> + object</code>, whose "base class" part is the + object you stored). There are a few data members and + member functions in <code class="classname">ElementRef</code>, + which all start with <code class="literal">_DB_STL_</code>. To + avoid potential name clashes, applications should not + use names prefixing <code class="literal">_DB_STL_</code> in + classes whose instances may be stored into dbstl + containers. + </p> + <p> + Example code demonstrating this feature can be + found in the + <code class="methodname">StlAdvancedFeaturesExample::arbitrary_object_storage</code> + method. + </p> </div> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1278616"></a>Storing arbitrary sequences</h3> + <h3 class="title"><a id="idp859120"></a>Storing arbitrary sequences</h3> </div> </div> </div> + <p> + A sequence is a group of related objects, such as an + array, a string, and so forth. You can store sequences of + any structure using dbstl, so long as you implement and + register the proper callback functions. By using these + callbacks, each object in the sequence can be a complex + object with data members that are all not stored in a + continuous memory chunk. + </p> <p> - A sequence is a group of related objects, such as an array, a string, and so forth. - You can store sequences of any structure using dbstl, so long as you implement and register the - proper callback functions. By using these callbacks, each object in the sequence can be a - complex object with data members that are all not stored in a continuous memory chunk. -</p> - <p> - Note that when using these callbacks, when you retrieve a stored sequence from the database, the - entire sequence will reside in a single continuous block of memory with the same layout as that - constructed by your sequence copy function. -</p> - <p> - For example, given a type RGB: -</p> - <pre class="programlisting"> -struct RGB{char r, g, b, bright;}; </pre> + Note that when using these callbacks, when you retrieve + a stored sequence from the database, the entire sequence + will reside in a single continuous block of memory with + the same layout as that constructed by your sequence copy + function. + </p> <p> - and an array of RGB objects, the following steps describe how to store an array into one - key/data pair of a <code class="classname">db_map</code> container. -</p> + For example, given a type RGB: + </p> + <pre class="programlisting">struct RGB{char r, g, b, bright;}; </pre> + <p> + and an array of RGB objects, the following steps + describe how to store an array into one key/data pair of a + <code class="classname">db_map</code> container. + </p> <div class="orderedlist"> <ol type="1"> - <li> - Use a <code class="classname">db_map<int, RGB *, ElementHolder<RGB *> ></code> container. -</li> + <li> + Use a <code class="classname">db_map<int, RGB *, + ElementHolder<RGB *> ></code> + container. + </li> <li> <p> - Define two functions. The first returns the number of objects in a sequence, the second that - copies objects from a sequence to a defined destination in memory: - </p> + Define two functions. The first returns the + number of objects in a sequence, the second that + copies objects from a sequence to a defined + destination in memory: + </p> <pre class="programlisting">typedef size_t (*SequenceLenFunct)(const RGB*); </pre> <p> - and -</p> + and + </p> <pre class="programlisting"> typedef void (*SequenceCopyFunct)(RGB*dest, const RGB*src); </pre> </li> - <li> -Call DbstlElemTraits<RGB>::set_sequence_len_function()/set_sequence_copy_function() -to register them as callbacks. -</li> + <li> + Call + DbstlElemTraits<RGB>::set_sequence_len_function()/set_sequence_copy_function() + to register them as callbacks. + </li> </ol> </div> <div class="sect3" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idp1308520"></a>The <code class="function">SequenceLenFunct</code> function</h4> + <h4 class="title"><a id="idp869312"></a>The <code class="function">SequenceLenFunct</code> + function</h4> </div> </div> </div> <pre class="programlisting">typedef size_t (*SequenceLenFunct)(const RGB*); </pre> <p> - A <code class="function">SequenceLenFunct</code> function returns the number of objects in a sequence. It - is called when inserting into or reading from the database, so there must be enough information - in the sequence itself to enable the <code class="function">SequenceLenFunct</code> function to tell how many - objects the sequence contains. The <code class="literal">char*</code> and <code class="literal">wchar_t*</code> - strings use a <code class="literal">'\0'</code> special character to do this. For example, RGB(0, 0, 0, 0) - could be used to denote the end of the sequence. Note that for your implementation of this - callback, you are not required to use a - trailing object with a special value like <code class="literal">'\0'</code> or - <code class="literal">RGB(0, 0, 0, 0)</code> to denote the end of the sequence. You are free to use - what mechanism you want in your - <code class="function">SequenceLenFunct</code> function implementation to figure out the length of the sequence. -</p> + A <code class="function">SequenceLenFunct</code> function + returns the number of objects in a sequence. It is + called when inserting into or reading from the + database, so there must be enough information in the + sequence itself to enable the + <code class="function">SequenceLenFunct</code> function to + tell how many objects the sequence contains. The + <code class="literal">char*</code> and + <code class="literal">wchar_t*</code> strings use a + <code class="literal">'\0'</code> special character to do + this. For example, RGB(0, 0, 0, 0) could be used to + denote the end of the sequence. Note that for your + implementation of this callback, you are not required + to use a trailing object with a special value like + <code class="literal">'\0'</code> or <code class="literal">RGB(0, 0, 0, + 0)</code> to denote the end of the sequence. + You are free to use what mechanism you want in your + <code class="function">SequenceLenFunct</code> function + implementation to figure out the length of the + sequence. + </p> </div> <div class="sect3" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idp1340280"></a>The <code class="function">SequenceCopyFunct</code> function</h4> + <h4 class="title"><a id="idp903872"></a>The <code class="function">SequenceCopyFunct</code> function</h4> </div> </div> </div> <pre class="programlisting"> typedef void (*SequenceCopyFunct)(RGB*dest, const RGB*src); </pre> <p> - <code class="function">SequenceCopyFunct</code> copies objects from the sequence - <span class="bold"><strong>src</strong></span> into memory chunk <span class="bold"><strong>dest</strong></span>. - If the objects in the sequence do not reside in a continuous memory chunk, this function must - marshal each object in the sequence into the <span class="bold"><strong>dest</strong></span> memory chunk. -</p> - <p> - The sequence objects will reside in the continuous memory chunk referred to by <span class="bold"><strong>dest</strong></span>, which has been sized by <code class="classname">SequenceLenFunct</code> - and <code class="classname">ElemSizeFunct</code> if available (which is when objects in the sequence are - of varying lengths). <code class="classname">ElemSizeFunct</code> function is not needed in this example - because <span class="bold"><strong>RGB</strong></span> is a simple fixed length type, the - <code class="literal">sizeof()</code> operator is sufficient to return the size of the sequence. -</p> + <code class="function">SequenceCopyFunct</code> copies objects + from the sequence <span class="bold"><strong>src</strong></span> + into memory chunk <span class="bold"><strong>dest</strong></span>. If the + objects in the sequence do not reside in a continuous memory chunk, this + function must marshal each object in the sequence into + the <span class="bold"><strong>dest</strong></span> memory + chunk. + </p> + <p> + The sequence objects will reside in the continuous + memory chunk referred to by <span class="bold"><strong>dest</strong></span>, which + has been sized by + <code class="classname">SequenceLenFunct</code> and + <code class="classname">ElemSizeFunct</code> if available + (which is when objects in the sequence are of varying + lengths). <code class="classname">ElemSizeFunct</code> + function is not needed in this example because + <span class="bold"><strong>RGB</strong></span> is a simple + fixed length type, the <code class="literal">sizeof()</code> + operator is sufficient to return the size of the + sequence. + </p> </div> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1344912"></a>Notes</h3> + <h3 class="title"><a id="idp909008"></a>Notes</h3> </div> </div> </div> <div class="itemizedlist"> <ul type="disc"> <li> - <p> - The get and set functions of this class are not protected by any mutexes. When using - multiple threads to access the function pointers, the callback functions must be - registered to the singleton of this class before any retrieval of the callback function - pointers. Isolation may also be required among multiple threads. The best way is to - register all callback function pointers in a single thread before making use of the any - containers. - </p> + <p> + The get and set functions of this class are not + protected by any mutexes. When using multiple + threads to access the function pointers, the + callback functions must be registered to the + singleton of this class before any retrieval of + the callback function pointers. Isolation may also + be required among multiple threads. The best way + is to register all callback function pointers in a + single thread before making use of the any + containers. + </p> </li> <li> <p> - If objects in a sequence are not of identical sizes, or are not located in a consecutive - chunk of memory, you also need to implement and register the - <code class="function">DbstlElemTraits<>::ElemSizeFunct</code> callback function to measure - the size of each object. When this function is registered, it is also used when - allocating memory space. - </p> + If objects in a sequence are not of identical + sizes, or are not located in a consecutive chunk + of memory, you also need to implement and register + the + <code class="function">DbstlElemTraits<>::ElemSizeFunct</code> + callback function to measure the size of each + object. When this function is registered, it is + also used when allocating memory space. </p> <p> - There is example code demonstrating the use this feature in the - <code class="methodname">StlAdvancedFeaturesExample::arbitray_sequence_storage()</code> method. - </p> + There is example code demonstrating the use + this feature in the + <code class="methodname">StlAdvancedFeaturesExample::arbitray_sequence_storage()</code> + method. + </p> </li> <li> <p> - A consequence of this dbstl feature is that you can not store a pointer value directly - because dbstl will think it is a sequence head pointer. Instead, you need to convert the - pointer into a <code class="literal">long</code> and then store it into a <code class="literal">long</code> - container. And please note that pointer values are probably meaningless if the stored - value is to be used across different application run times. - </p> + A consequence of this dbstl feature is that you + can not store a pointer value directly because + dbstl will think it is a sequence head pointer. + Instead, you need to convert the pointer into a + <code class="literal">long</code> and then store it into + a <code class="literal">long</code> container. And please + note that pointer values are probably meaningless + if the stored value is to be used across different + application run times. + </p> </li> </ul> </div> diff --git a/docs/programmer_reference/stl_container_specific.html b/docs/programmer_reference/stl_container_specific.html index bfe4907e..189871ee 100644 --- a/docs/programmer_reference/stl_container_specific.html +++ b/docs/programmer_reference/stl_container_specific.html @@ -14,11 +14,12 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Dbstl container specific notes</th> + <th colspan="3" align="center">Dbstl container specific + notes</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="stl_persistence.html">Prev</a> </td> @@ -32,7 +33,8 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="stl_container_specific"></a>Dbstl container specific notes</h2> + <h2 class="title" style="clear: both"><a id="stl_container_specific"></a>Dbstl container specific + notes</h2> </div> </div> </div> @@ -40,12 +42,12 @@ <dl> <dt> <span class="sect2"> - <a href="stl_container_specific.html#idp1313840">db_vector specific notes</a> + <a href="stl_container_specific.html#idp873704">db_vector specific notes</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_container_specific.html#idp1381768">Associative container specific notes</a> + <a href="stl_container_specific.html#idp948648">Associative container specific notes</a> </span> </dt> </dl> @@ -54,69 +56,99 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1313840"></a>db_vector specific notes</h3> + <h3 class="title"><a id="idp873704"></a>db_vector specific notes</h3> </div> </div> </div> <div class="itemizedlist"> <ul type="disc"> <li> + <p> + Set the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> flag in the database + handle if you want + <code class="classname">db_vector<></code> to + work like <code class="classname">std::vector</code> or + <code class="classname">std::deque</code>. Do not set + <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> if you want + <code class="classname">db_vector<></code> to + work like <code class="classname">std::list</code>. Note + that without <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> set, + <code class="classname">db_vector<></code> can + work faster. + </p> <p> - Set the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> flag in the database handle if you want - <code class="classname">db_vector<></code> to work like - <code class="classname">std::vector</code> or <code class="classname">std::deque</code>. Do not set - <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> if you want <code class="classname">db_vector<></code> to work like - <code class="classname">std::list</code>. Note that without <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> set, - <code class="classname">db_vector<></code> can work faster. - </p> + For example, to construct a fast + std::queue/std::stack object, you only need a + <code class="classname">db_vector<></code> + object whose database handle does not have + <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> set. Of course, if the database + handle has <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> set, it still works for + this kind of scenario, just not as fast. + </p> <p> - For example, to construct a fast std::queue/std::stack object, you only need a - <code class="classname">db_vector<></code> object whose database handle does not have - <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> set. Of course, if the database handle has <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> set, it - still works for this kind of scenario, just not as fast. - </p> + <code class="classname">db_vector</code> does not check + whether <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> is set. If you do not set + it, <code class="classname">db_vector<></code> will + not work like + std::vector<>/std::deque<> with regard + to operator[], because the indices are not + maintained in that case. + </p> <p> - <code class="classname">db_vector</code> does not check whether <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> is set. If - you do not set it, <code class="classname">db_vector<></code> will not work like - std::vector<>/std::deque<> with regard to operator[], because the - indices are not maintained in that case. - </p> - <p> - You can find example code showing how to use this feature in the - <code class="methodname">StlAdvancedFeaturesExample::queue_stack()</code> method. - </p> + You can find example code showing how to use + this feature in the + <code class="methodname">StlAdvancedFeaturesExample::queue_stack()</code> + method. + </p> </li> <li> + <p> + Just as is the case with + <code class="classname">std::vector</code>, + inserting/deleting in the middle of a + <code class="classname">db_vector</code> is slower + than doing the same action at the end of the + sequence. This is because the underlying DB_RECNO + DB (with the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> flag set) is relatively + slow when inserting/deleting in the middle or the + head — it has to update the index numbers of + all the records following the one that was + inserted/deleted. If you do not need to keep the + index ordered on insert/delete, you can use + <code class="classname">db_map</code> instead. + </p> <p> - Just as is the case with <code class="classname">std::vector</code>, inserting/deleting in - the middle of a <code class="classname">db_vector</code> is slower than doing the same - action at the end of the sequence. This is because the underlying DB_RECNO DB (with - the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> flag set) is relatively slow when inserting/deleting in the middle - or the head — it has to update the index numbers of all the records following - the one that was inserted/deleted. If you do not need to keep the index ordered on - insert/delete, you can use <code class="classname">db_map</code> instead. - </p> - <p> - <code class="classname">db_vector</code> also contains methods inherited from - <code class="classname">std::list</code> and <code class="classname">std::deque</code>, - including <code class="classname">std::list<>'s</code> unique methods - <code class="methodname">remove()</code>, <code class="methodname">remove_if()</code>, - <code class="methodname">unique()</code>, <code class="methodname">merge()</code>, - <code class="methodname">sort()</code>, <code class="methodname">reverse()</code>, and - <code class="methodname">splice()</code>. These use the identical semantics/behaviors - of the <code class="classname">std::list<></code> methods, although - pushing/deleting at the head is slower than the - <code class="methodname">std::deque</code> and <code class="methodname">std::list</code> - equivalent when there are quite a lot of elements in the database. - </p> + <code class="classname">db_vector</code> also contains + methods inherited from + <code class="classname">std::list</code> and + <code class="classname">std::deque</code>, including + <code class="classname">std::list<>'s</code> + unique methods <code class="methodname">remove()</code>, + <code class="methodname">remove_if()</code>, + <code class="methodname">unique()</code>, + <code class="methodname">merge()</code>, + <code class="methodname">sort()</code>, + <code class="methodname">reverse()</code>, and + <code class="methodname">splice()</code>. These use + the identical semantics/behaviors of the + <code class="classname">std::list<></code> + methods, although pushing/deleting at the head is + slower than the + <code class="methodname">std::deque</code> and + <code class="methodname">std::list</code> equivalent + when there are quite a lot of elements in the + database. + </p> </li> <li> - <p> - You can use <code class="classname">std::queue</code>, - <code class="classname">std::priority_queue</code> and <code class="classname">std::stack</code> - container adapters with <code class="classname">db_vector</code>; they work with db_vector - even without <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> set. - </p> + <p> + You can use <code class="classname">std::queue</code>, + <code class="classname">std::priority_queue</code> and + <code class="classname">std::stack</code> container + adapters with <code class="classname">db_vector</code>; + they work with db_vector even without + <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_RENUMBER" class="olink">DB_RENUMBER</a> set. + </p> </li> </ul> </div> @@ -125,24 +157,32 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1381768"></a>Associative container specific notes</h3> + <h3 class="title"><a id="idp948648"></a>Associative container specific notes</h3> </div> </div> </div> <p> - <code class="classname">db_map</code> contains the union of method set from - <code class="classname">std::map</code> and <code class="classname">hash_map</code>, but there are some - methods that can only be called on containers backed by <code class="literal">DB_BTREE</code> or - <code class="literal">DB_HASH</code> databases. You can call - <code class="function">db_map<>::is_hash()</code> to figure out the type of the backing - database. If you call unsupported methods then an InvalidFunctionCall exception is thrown. - </p> - <p> - These are the <code class="literal">DB_BTREE</code> specific methods: <code class="methodname">upper_bound()</code>, - <code class="methodname">lower_bound()</code>, <code class="methodname">key_comp()</code>, - and <code class="methodname">value_comp()</code>. The <code class="literal">DB_HASH</code> specific methods are - <code class="methodname">key_eq()</code>, <code class="methodname">hash_funct()</code>. - </p> + <code class="classname">db_map</code> contains the union of method + set from <code class="classname">std::map</code> and + <code class="classname">hash_map</code>, but there are some + methods that can only be called on containers backed by + <code class="literal">DB_BTREE</code> or + <code class="literal">DB_HASH</code> databases. You can call + <code class="function">db_map<>::is_hash()</code> to + figure out the type of the backing database. If you call + unsupported methods then an InvalidFunctionCall exception + is thrown. + </p> + <p> + These are the <code class="literal">DB_BTREE</code> specific + methods: <code class="methodname">upper_bound()</code>, + <code class="methodname">lower_bound()</code>, + <code class="methodname">key_comp()</code>, and + <code class="methodname">value_comp()</code>. The + <code class="literal">DB_HASH</code> specific methods are + <code class="methodname">key_eq()</code>, + <code class="methodname">hash_funct()</code>. + </p> </div> </div> <div class="navfooter"> @@ -160,7 +200,8 @@ <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Using dbstl efficiently</td> + <td width="40%" align="right" valign="top"> Using dbstl + efficiently</td> </tr> </table> </div> diff --git a/docs/programmer_reference/stl_db_advanced_usage.html b/docs/programmer_reference/stl_db_advanced_usage.html index 6405dded..9046bdee 100644 --- a/docs/programmer_reference/stl_db_advanced_usage.html +++ b/docs/programmer_reference/stl_db_advanced_usage.html @@ -14,11 +14,12 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Using advanced Berkeley DB features with dbstl</th> + <th colspan="3" align="center">Using advanced Berkeley DB + features with dbstl</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="stl_db_usage.html">Prev</a> </td> @@ -32,7 +33,8 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="stl_db_advanced_usage"></a>Using advanced Berkeley DB features with dbstl</h2> + <h2 class="title" style="clear: both"><a id="stl_db_advanced_usage"></a>Using advanced Berkeley DB + features with dbstl</h2> </div> </div> </div> @@ -40,158 +42,177 @@ <dl> <dt> <span class="sect2"> - <a href="stl_db_advanced_usage.html#idp1232296">Using bulk retrieval iterators</a> + <a href="stl_db_advanced_usage.html#idp672128">Using bulk retrieval iterators</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_db_advanced_usage.html#idp1232520">Using the DB_RMW flag</a> + <a href="stl_db_advanced_usage.html#idp794736">Using the DB_RMW flag</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_db_advanced_usage.html#idp1199288">Using secondary index database and secondary containers</a> + <a href="stl_db_advanced_usage.html#idp773392">Using secondary index database and secondary containers</a> </span> </dt> </dl> </div> <p> - This section describes advanced Berkeley DB features that are - available through dbstl. + This section describes advanced Berkeley DB features that + are available through dbstl. </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1232296"></a>Using bulk retrieval iterators</h3> + <h3 class="title"><a id="idp672128"></a>Using bulk retrieval iterators</h3> </div> </div> </div> <p> - Bulk retrieval is an optimization option for const iterators and - nonconst but read-only iterators. Bulk retrieval can minimize the - number of database accesses performed by your application. It does this - by reading multiple entries at a time, which reduces read overhead. - Note that non-sequential reads will benefit less from, or even be hurt - by, this behavior, because it might result in unneeded data being read - from the database. Also, non-serializable reads may read obsolete - data, because part of the data read from the bulk read buffer may have - been updated since the retrieval. -</p> + Bulk retrieval is an optimization option for const + iterators and nonconst but read-only iterators. Bulk + retrieval can minimize the number of database accesses + performed by your application. It does this by reading + multiple entries at a time, which reduces read overhead. + Note that non-sequential reads will benefit less from, or + even be hurt by, this behavior, because it might result in + unneeded data being read from the database. Also, + non-serializable reads may read obsolete data, because + part of the data read from the bulk read buffer may have + been updated since the retrieval. + </p> <p> - When using the default transaction isolation, iterators will perform - serializable reads. In this situation, the bulk-retrieved data cannot - be updated until the iterator's cursor is closed. -</p> + When using the default transaction isolation, iterators + will perform serializable reads. In this situation, the + bulk-retrieved data cannot be updated until the iterator's + cursor is closed. + </p> <p> - Iterators using a different isolation levels, such as - <a href="../api_reference/C/dbcget.html#dbcget_DB_READ_COMMITTED" class="olink">DB_READ_COMMITTED</a> or <a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a> will not perform - serializable reads. The same is true for any iterators that do not use - transactions. -</p> + Iterators using a different isolation levels, such as + <a href="../api_reference/C/dbcget.html#dbcget_DB_READ_COMMITTED" class="olink">DB_READ_COMMITTED</a> or <a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a> will not + perform serializable reads. The same is true for any + iterators that do not use transactions. + </p> + <p> + A bulk retrieval iterator can only move in a singled + direction, from beginning to end. This means that + iterators only support operator++, and reverse iterators + only support operator--. + </p> + <p> + Iterator objects that use bulk retrieval might contain + hundreds of kilobytes of data, which makes copying the + iterator object an expensive operation. If possible, use + ++iterator rather than iterator++. This can save a useless + copy construction of the iterator, as well as an + unnecessary dup/close of the cursor. + </p> + <p> + You can configure bulk retrieval for each container + using both in the const and non-const version of the + <code class="methodname">begin()</code> method. The non-const + version of <code class="methodname">begin()</code> will return a + read-only cursor. Note that read-only means something + different in C++ than it does when referring to an + iterator. The latter only means that it cannot be used to + update the database. + </p> + <p> + To configure the bulk retrieval buffer for an iterator + when calling the <code class="methodname">begin()</code> method, + use the + <code class="function">BulkRetrievelItrOpt::bulk_retrieval(u_int32_t + bulk_buffer_size)</code> function. + </p> <p> - A bulk retrieval iterator can only move in a singled direction, from - beginning to end. This means that iterators only support operator++, - and reverse iterators only support operator--. -</p> - <p> - Iterator objects that use bulk retrieval might contain hundreds of - kilobytes of data, which makes copying the iterator object an expensive - operation. If possible, use ++iterator rather than iterator++. This - can save a useless copy construction of the iterator, as well as an - unnecessary dup/close of the cursor. -</p> - <p> - You can configure bulk retrieval for each container using both in the - const and non-const version of the <code class="methodname">begin()</code> - method. The non-const version of <code class="methodname">begin()</code> will - return a read-only cursor. Note that read-only means something - different in C++ than it does when referring to an iterator. The latter - only means that it cannot be used to update the database. -</p> - <p> - To configure the bulk retrieval buffer for an iterator when calling the - <code class="methodname">begin()</code> method, use the - <code class="function">BulkRetrievelItrOpt::bulk_retrieval(u_int32_t bulk_buffer_size)</code> - function. -</p> - <p> - If you move a <code class="classname">db_vector_iterator</code> randomly rather - than sequentially, then dbstl will not perform bulk retrieval because - there is little performance gain from bulk retrieval in such an access - pattern. -</p> - <p> - You can call <code class="function">iterator::set_bulk_buffer()</code> to modify - the iterator's bulk buffer size. Note that once bulk read is enabled, - only the bulk buffer size can be modified. This means that bulk read - cannot be disabled. Also, if bulk read was not enabled when you created - the iterator, you can't enable it after creation. -</p> - <p> - Example code using this feature can be found in the - <code class="methodname">StlAdvancedFeaturesExample::bulk_retrieval_read()</code> method. -</p> + If you move a <code class="classname">db_vector_iterator</code> + randomly rather than sequentially, then dbstl will not + perform bulk retrieval because there is little performance + gain from bulk retrieval in such an access pattern. + </p> + <p> + You can call + <code class="function">iterator::set_bulk_buffer()</code> to + modify the iterator's bulk buffer size. Note that once + bulk read is enabled, only the bulk buffer size can be + modified. This means that bulk read cannot be disabled. + Also, if bulk read was not enabled when you created the + iterator, you can't enable it after creation. + </p> + <p> + Example code using this feature can be found in the + <code class="methodname">StlAdvancedFeaturesExample::bulk_retrieval_read()</code> + method. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1232520"></a>Using the DB_RMW flag</h3> + <h3 class="title"><a id="idp794736"></a>Using the DB_RMW flag</h3> </div> </div> </div> <p> - The <a href="../api_reference/C/dbcget.html#dbcget_DB_RMW" class="olink">DB_RMW</a> flag is an optimization for non-const (read-write) - iterators. This flag causes the underlying cursor to acquire a write - lock when reading so as to avoid deadlocks. Passing - <code class="function">ReadModifyWriteOption::read_modify_write()</code> to a - container's <code class="methodname">begin()</code> method creates an iterator - whose cursor has this behavior. -</p> + The <a href="../api_reference/C/dbcget.html#dbcget_DB_RMW" class="olink">DB_RMW</a> flag is an optimization for non-const + (read-write) iterators. This flag causes the underlying + cursor to acquire a write lock when reading so as to avoid + deadlocks. Passing + <code class="function">ReadModifyWriteOption::read_modify_write()</code> + to a container's <code class="methodname">begin()</code> method + creates an iterator whose cursor has this behavior. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1199288"></a>Using secondary index database and secondary containers</h3> + <h3 class="title"><a id="idp773392"></a>Using secondary index database and secondary containers</h3> </div> </div> </div> + <p> + Because duplicate keys are forbidden in primary + databases, only <code class="classname">db_map</code>, + <code class="classname">db_set</code> and + <code class="classname">db_vector</code> are allowed to use + primary databases. For this reason, they are called + <span class="bold"><strong>primary containers</strong></span>. A + secondary database that supports duplicate keys can be + used with <code class="classname">db_multimap</code> containers. + These are called <span class="bold"><strong>secondary + containers</strong></span>. Finally, a secondary database + that forbids duplicate keys can back a + <code class="classname">db_map</code> container. + </p> + <p> + The <span class="bold"><strong>data_type</strong></span> of this + <code class="classname">db_multimap</code> secondary container + is the <span class="bold"><strong>data_type</strong></span> for the + primary container. For example, a + <code class="classname">db_map<int, Person></code> + object where the <code class="classname">Person</code> class has + an <code class="literal">age</code> property of type + <code class="literal">size_t</code>, a + <code class="classname">db_multimap<size_t, + Person></code> using a secondary database + allows access to a person by age. + </p> <p> - Because duplicate keys are forbidden in primary databases, only - <code class="classname">db_map</code>, <code class="classname">db_set</code> and - <code class="classname">db_vector</code> are allowed to use primary databases. - For this reason, they are called - <span class="bold"><strong>primary containers</strong></span>. - A secondary database that supports duplicate keys can be used with - <code class="classname">db_multimap</code> containers. These are called - <span class="bold"><strong>secondary containers</strong></span>. Finally, a - secondary database that forbids duplicate keys can back a - <code class="classname">db_map</code> container. -</p> - <p> - The <span class="bold"><strong>data_type</strong></span> of this - <code class="classname">db_multimap</code> secondary container is the - <span class="bold"><strong>data_type</strong></span> for the primary container. For - example, a <code class="classname">db_map<int, Person></code> object - where the <code class="classname">Person</code> class has an - <code class="literal">age</code> property of type <code class="literal">size_t</code>, a - <code class="classname">db_multimap<size_t, Person></code> using a - secondary database allows access to a person by age. -</p> - <p> - - A container created from a secondary database can only be used to - iterate, search or delete. It can not be used to update or insert. - While dbstl does expose the update and insert operations, Berkeley DB - does not, and an exception will be thrown if attempts are made to - insert objects into or update objects of a secondary container. -</p> + A container created from a secondary database can only + be used to iterate, search or delete. It can not be used + to update or insert. While dbstl does expose the update + and insert operations, Berkeley DB does not, and an + exception will be thrown if attempts are made to insert + objects into or update objects of a secondary container. + </p> <p> - Example code demonstrating this feature is available in the - <code class="methodname">StlAdvancedFeaturesExample::secondary_containers()</code> method. -</p> + Example code demonstrating this feature is available in + the + <code class="methodname">StlAdvancedFeaturesExample::secondary_containers()</code> + method. + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/stl_db_usage.html b/docs/programmer_reference/stl_db_usage.html index 815a6ed4..af409424 100644 --- a/docs/programmer_reference/stl_db_usage.html +++ b/docs/programmer_reference/stl_db_usage.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -40,99 +40,105 @@ <dl> <dt> <span class="sect2"> - <a href="stl_db_usage.html#idp1202384">Registering database and environment handles</a> + <a href="stl_db_usage.html#idp756272">Registering database and environment handles</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_db_usage.html#idp1225928">Truncate requirements</a> + <a href="stl_db_usage.html#idp772680">Truncate requirements</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_db_usage.html#idp1236168">Auto commit support</a> + <a href="stl_db_usage.html#idp773680">Auto commit support</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_db_usage.html#idp1239792">Database and environment identity checks</a> + <a href="stl_db_usage.html#idp792728">Database and environment identity checks</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_db_usage.html#idp1236512">Products, constructors and configurations</a> + <a href="stl_db_usage.html#idp795408">Products, constructors and configurations</a> </span> </dt> </dl> </div> - <p> - While dbstl behaves like the C++ STL APIs in most situations, there - are some Berkeley DB configuration activities that you can and - should perform using dbstl. These activities are described in the - following sections. + <p> + While dbstl behaves like the C++ STL APIs in most + situations, there are some Berkeley DB configuration + activities that you can and should perform using dbstl. These + activities are described in the following sections. </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1202384"></a>Registering database and environment handles</h3> + <h3 class="title"><a id="idp756272"></a>Registering database and environment handles</h3> </div> </div> </div> - <p> - Remember the following things as you use Berkeley DB Database - and Environment handles with dbstl: + <p> + Remember the following things as you use Berkeley DB + Database and Environment handles with dbstl: </p> <div class="itemizedlist"> <ul type="disc"> <li> - <p> - If you share environment or database handles among multiple - threads, remember to specify the <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flag in the open call to - the handle. - </p> + <p> + If you share environment or database handles + among multiple threads, remember to specify the + <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flag in the open call to the handle. + </p> </li> <li> - <p> - If you create or open environment and/or database handles without - using the dbstl helper functions, - <code class="function">dbstl::open_db()</code> or - <code class="function">dbstl::open_env()</code>, remember that your - environment and database handles should be: - </p> + <p> + If you create or open environment and/or + database handles without using the dbstl helper + functions, <code class="function">dbstl::open_db()</code> + or <code class="function">dbstl::open_env()</code>, + remember that your environment and database + handles should be: + </p> <div class="orderedlist"> <ol type="1"> <li> <p> - Allocated in the heap via "new" operator. - </p> + Allocated in the heap via "new" + operator. + </p> </li> <li> - <p> - Created using the <a href="../api_reference/CXX/envcreate.html#env_DB_CXX_NO_EXCEPTIONS" class="olink">DB_CXX_NO_EXCEPTIONS</a> flag. - </p> + <p> Created using the + <a href="../api_reference/CXX/envcreate.html#env_DB_CXX_NO_EXCEPTIONS" class="olink">DB_CXX_NO_EXCEPTIONS</a> flag. </p> </li> <li> - <p> - In each thread sharing the handles, the handles are registered using - either <code class="function">dbstl::register_db()</code> or - <code class="function">dbstl::register_dbenv()</code>. - </p> + <p> + In each thread sharing the handles, the + handles are registered using either + <code class="function">dbstl::register_db()</code> + or + <code class="function">dbstl::register_dbenv()</code>. + </p> </li> </ol> </div> </li> <li> - <p> - If you opened the database or environment handle using the - <code class="function">open_db()</code> or <code class="function">open_env()</code> - functions, the thread opening the handles should not call - <code class="function">register_db()</code> or <code class="function">register_env()</code> - again. This is because they have already been registered by the - <code class="function">open_db()</code> or <code class="function">open_env()</code> - functions. However, other threads sharing these handles still must register - them locally. - </p> + <p> + If you opened the database or environment + handle using the <code class="function">open_db()</code> or + <code class="function">open_env()</code> functions, the + thread opening the handles should not call + <code class="function">register_db()</code> or + <code class="function">register_env()</code> again. + This is because they have already been registered + by the <code class="function">open_db()</code> or + <code class="function">open_env()</code> functions. + However, other threads sharing these handles still + must register them locally. + </p> </li> </ul> </div> @@ -141,55 +147,62 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1225928"></a>Truncate requirements</h3> + <h3 class="title"><a id="idp772680"></a>Truncate requirements</h3> </div> </div> </div> <p> - Some Berkeley DB operations require there to be no open cursors on the - database handle at the time the operation occurs. Dbstl is aware of - these requirements, and will attempt to close the cursors opened in the - current thread when it performs these operations. However, the scope of - dbstl's activities in this regard are limited to the current thread; it - makes no attempt to close cursors opened in other threads. So you - are required to ensure there are no open cursors on database handles - shared across threads when operations are performed that require all - cursors on that handle to be closed. -</p> - <p> - - There are only a a few operations which require all open cursors to be closed. - This include all container <code class="methodname">clear()</code> and - <code class="methodname">swap()</code> functions, and all versions of - <code class="methodname">db_vection<>::assign()</code> functions. These - functions require all cursors to be closed for the database because by default - they remove all key/data pairs from the database by truncating it. -</p> + Some Berkeley DB operations require there to be no open + cursors on the database handle at the time the operation + occurs. Dbstl is aware of these requirements, and will + attempt to close the cursors opened in the current thread + when it performs these operations. However, the scope of + dbstl's activities in this regard are limited to the + current thread; it makes no attempt to close cursors + opened in other threads. So you are required to ensure + there are no open cursors on database handles shared + across threads when operations are performed that require + all cursors on that handle to be closed. + </p> <p> - When a function removes all key/data pairs from a database, there are - two ways it can perform this activity: -</p> + There are only a a few operations which require all + open cursors to be closed. This include all container + <code class="methodname">clear()</code> and + <code class="methodname">swap()</code> functions, and all + versions of + <code class="methodname">db_vection<>::assign()</code> + functions. These functions require all cursors to be + closed for the database because by default they remove all + key/data pairs from the database by truncating it. + </p> + <p> + When a function removes all key/data pairs from a + database, there are two ways it can perform this activity: + </p> <div class="itemizedlist"> <ul type="disc"> <li> - <p> - The default method is to truncate the database, which is an - operation that requires all cursors to be closed. As mentioned - above, it is your responsibility to close cursors opened in other - threads before performing this operation. Otherwise, the operation - will fail. - </p> + <p> + The default method is to truncate the database, + which is an operation that requires all cursors to + be closed. As mentioned above, it is your + responsibility to close cursors opened in other + threads before performing this operation. + Otherwise, the operation will fail. + </p> </li> <li> <p> - Alternatively, you can specify that the database not be truncated. - Instead, you can cause dbstl to delete all key/data pairs - individually, one after another. In this situation, - open cursors in the database will not cause the delete operations to - fail. However, due to lock contention, the delete operations might not - complete until all cursors are closed, which is when all their read locks - are released. - </p> + Alternatively, you can specify that the + database not be truncated. Instead, you can cause + dbstl to delete all key/data pairs individually, + one after another. In this situation, open cursors + in the database will not cause the delete + operations to fail. However, due to lock + contention, the delete operations might not + complete until all cursors are closed, which is + when all their read locks are released. + </p> </li> </ul> </div> @@ -198,102 +211,114 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1236168"></a>Auto commit support</h3> + <h3 class="title"><a id="idp773680"></a>Auto commit support</h3> </div> </div> </div> <p> - - Dbstl supports auto commit for some of its container's operations. When a dbstl - container is created using a <code class="classname">Db</code> or <code class="classname">DbEnv</code> - object, if that object was opened using the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_AUTO_COMMIT" class="olink">DB_AUTO_COMMIT</a> flag, then - every operation subsequently performed on that object will be - automatically enclosed in a unique transaction (unless the operation is already in an - external transaction). This - is identical to how the Berkeley DB C, C++ and Java APIs behave. -</p> - <p> - Note that only a subset of a container's operations support auto - commit. This is because those operations that accept or return an - iterator have to exist in an external transactional context and so - cannot support auto commit. -</p> - <p> - The dbstl API documentation identifies when a method supports auto - commit transactions. -</p> + Dbstl supports auto commit for some of its container's + operations. When a dbstl container is created using a + <code class="classname">Db</code> or + <code class="classname">DbEnv</code> object, if that object + was opened using the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_AUTO_COMMIT" class="olink">DB_AUTO_COMMIT</a> flag, then every + operation subsequently performed on that object will be + automatically enclosed in a unique transaction (unless the + operation is already in an external transaction). This is + identical to how the Berkeley DB C, C++ and Java APIs + behave. + </p> + <p> + Note that only a subset of a container's operations + support auto commit. This is because those operations that + accept or return an iterator have to exist in an external + transactional context and so cannot support auto commit. + </p> + <p> + The dbstl API documentation identifies when a method + supports auto commit transactions. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1239792"></a>Database and environment identity checks</h3> + <h3 class="title"><a id="idp792728"></a>Database and environment identity checks</h3> </div> </div> </div> <p> - When a container member function involves another container (for example, - <code class="methodname">db_vector::swap(self& v2)</code>), the two - containers involved in the operation must not use the same database. - Further, if the function is in an external or internal transaction context, - then both containers must belong - to the same transactional database environment; Otherwise, the two containers - can belong to the same database environment, or two different ones. -</p> - <p> - For example, if <code class="methodname">db_vector::swap(self& v2)</code> - is an auto commit method or it is in an external transaction context, - then <code class="literal">v2</code> must be in the same transactional database - environment as this container, because a transaction is started - internally that must be used by both <code class="literal">v2</code> and this - container. If this container and the <code class="literal">v2</code> container - have different database environments, and either of them are using transactions, - an exception is thrown. This condition is checked in every such member - function. -</p> + When a container member function involves another + container (for example, + <code class="methodname">db_vector::swap(self& + v2)</code>), the two containers involved in the + operation must not use the same database. Further, if the + function is in an external or internal transaction + context, then both containers must belong to the same + transactional database environment; Otherwise, the two + containers can belong to the same database environment, or + two different ones. + </p> <p> - However, if the function is not in a transactional context, - then the databases used by these - containers can be in different environments because in this situation - dbstl makes no attempt to wrap container operations in a common transaction - context. -</p> + For example, if <code class="methodname">db_vector::swap(self& + v2)</code> is an auto commit method or it is in + an external transaction context, then + <code class="literal">v2</code> must be in the same + transactional database environment as this container, + because a transaction is started internally that must be + used by both <code class="literal">v2</code> and this container. If + this container and the <code class="literal">v2</code> container + have different database environments, and either of them + are using transactions, an exception is thrown. This + condition is checked in every such member function. + </p> + <p> + However, if the function is not in a transactional + context, then the databases used by these containers can + be in different environments because in this situation + dbstl makes no attempt to wrap container operations in a + common transaction context. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1236512"></a>Products, constructors and configurations</h3> + <h3 class="title"><a id="idp795408"></a>Products, constructors and configurations</h3> </div> </div> </div> <p> - You can use dbstl with all Berkeley DB products (DS, CDS, TDS, and HA). - Because dbstl is a Berkeley DB interface, all necessary configurations - for these products are performed using Berkeley DB's standard - create/open/set APIs. -</p> - <p> - As a result, the dbstl container constructors differ from those of C++ - STL because in dbstl no configuration is supported using the container - constructors. On the other hand, dbstl container constructors accept - already opened and configured environment and database handles. They - also provide functions to retrieve some handle configuration, such as - key comparison and hash functions, as required by the C++ STL - specifications. -</p> - <p> - The constructors verify that the handles passed to them are well - configured. This means they ensure that no banned settings are used, as - well as ensuring that all required setting are performed. If the - handles are not well configured, an - <code class="classname">InvalidArgumentException</code> is thrown. -</p> - <p> - If a container constructor is not passed a database or environment - handle, an internal anonymous database is created for you by dbstl. This anonymous - database does not provide data persistence. -</p> + You can use dbstl with all Berkeley DB products (DS, + CDS, TDS, and HA). Because dbstl is a Berkeley DB + interface, all necessary configurations for these products + are performed using Berkeley DB's standard create/open/set + APIs. + </p> + <p> + As a result, the dbstl container constructors differ + from those of C++ STL because in dbstl no configuration is + supported using the container constructors. On the other + hand, dbstl container constructors accept already opened + and configured environment and database handles. They also + provide functions to retrieve some handle configuration, + such as key comparison and hash functions, as required by + the C++ STL specifications. + </p> + <p> + The constructors verify that the handles passed to them + are well configured. This means they ensure that no banned + settings are used, as well as ensuring that all required + setting are performed. If the handles are not well + configured, an + <code class="classname">InvalidArgumentException</code> is + thrown. + </p> + <p> + If a container constructor is not passed a database or + environment handle, an internal anonymous database is + created for you by dbstl. This anonymous database does not + provide data persistence. + </p> </div> </div> <div class="navfooter"> @@ -311,7 +336,8 @@ <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Using advanced Berkeley DB features with dbstl</td> + <td width="40%" align="right" valign="top"> Using advanced Berkeley DB + features with dbstl</td> </tr> </table> </div> diff --git a/docs/programmer_reference/stl_efficienct_use.html b/docs/programmer_reference/stl_efficienct_use.html index 035fcd68..b7a7f005 100644 --- a/docs/programmer_reference/stl_efficienct_use.html +++ b/docs/programmer_reference/stl_efficienct_use.html @@ -14,11 +14,12 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Using dbstl efficiently</th> + <th colspan="3" align="center">Using dbstl + efficiently</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="stl_container_specific.html">Prev</a> </td> @@ -32,7 +33,8 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="stl_efficienct_use"></a>Using dbstl efficiently</h2> + <h2 class="title" style="clear: both"><a id="stl_efficienct_use"></a>Using dbstl + efficiently</h2> </div> </div> </div> @@ -40,12 +42,12 @@ <dl> <dt> <span class="sect2"> - <a href="stl_efficienct_use.html#idp1350664">Using iterators efficiently</a> + <a href="stl_efficienct_use.html#idp915608">Using iterators efficiently</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_efficienct_use.html#idp1350448">Using containers efficiently</a> + <a href="stl_efficienct_use.html#idp957584">Using containers efficiently</a> </span> </dt> </dl> @@ -54,7 +56,7 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1350664"></a>Using iterators efficiently</h3> + <h3 class="title"><a id="idp915608"></a>Using iterators efficiently</h3> </div> </div> </div> @@ -65,83 +67,115 @@ <ul type="disc"> <li> <p> - Close an iterator's cursor as soon as possible. - </p> - <p> - Each iterator has an open cursor associated with it, so when you are finished using the - iterator it is a good habit to explicitly close its cursor. This can potentially improve - performance by avoiding locking issues, which will enhanced concurrency. Dbstl will close - the cursor when the iterator is destroyed, but you can close the cursor before that time. - If the cursor is closed, the associated iterator cannot any longer be used. - </p> - <p> - In some functions of container classes, an iterator is used to access the database, and its - cursor is internally created by dbstl. So if you want to specify a non-zero flag for the - <code class="methodname">Db::cursor()</code> call, you need to call the container's - <code class="function">set_cursor_open_flag()</code> function to do so. - </p> + Close an iterator's cursor as soon as possible. + </p> + <p> + Each iterator has an open cursor associated + with it, so when you are finished using the + iterator it is a good habit to explicitly close + its cursor. This can potentially improve + performance by avoiding locking issues, which will + enhanced concurrency. Dbstl will close the cursor + when the iterator is destroyed, but you can close + the cursor before that time. If the cursor is + closed, the associated iterator cannot any longer + be used. + </p> + <p> + In some functions of container classes, an + iterator is used to access the database, and its + cursor is internally created by dbstl. So if you + want to specify a non-zero flag for the + <code class="methodname">Db::cursor()</code> call, + you need to call the container's + <code class="function">set_cursor_open_flag()</code> + function to do so. + </p> </li> <li> <p> - Use const iterators where applicable. - </p> + Use const iterators where applicable. + </p> <p> - If your data access is read only, you are strongly recommended to use a const iterator. In - order to create a const iterator, you must use a const reference to the container object. - For example, supposed we have: - </p> + If your data access is read only, you are + strongly recommended to use a const iterator. In + order to create a const iterator, you must use a + const reference to the container object. For + example, supposed we have: + </p> <pre class="programlisting">db_vector<int> intv(10);</pre> <p> - then we must use a: - </p> + then we must use a: + </p> <pre class="programlisting">const db_vector<int>& intv_ref = intv;</pre> + <p> + reference to invoke the const begin/end + functions. + <code class="methodname">intv_ref.begin()</code> will + give you a const iterator. You can use a const + iterator only to read its referenced data + elements, not update them. However, you should + have better performance with this iterator using, + for example, either + <code class="literal">iterator::operator*</code> or + <code class="literal">iterator::operator->member</code>. + Also, using array indices like + <code class="literal">intv_ref[i]</code> will also + perform better. + </p> + <p> + All functions in dbstl's containers which + return an iterator or data element reference have + two versions — one returns a const + iterator/reference, the other returns an + iterator/reference. If your access is read only, + choose the version returning const + iterators/references. + </p> <p> - reference to invoke the const begin/end functions. <code class="methodname">intv_ref.begin()</code> - will give you a const iterator. You can use a const iterator only to read its referenced - data elements, not update them. However, you should have better performance with this - iterator using, for example, either <code class="literal">iterator::operator*</code> or - <code class="literal">iterator::operator->member</code>. Also, using array indices like - <code class="literal">intv_ref[i]</code> will also perform better. - </p> - <p> - All functions in dbstl's containers which return an iterator or data element reference have - two versions — one returns a const iterator/reference, the other returns an - iterator/reference. If your access is read only, choose the version returning const - iterators/references. - </p> - <p> - Remember that you can only use a const reference to a container object to call the const - versions of <code class="literal">operator*</code> and <code class="literal">operator[]</code>. - </p> + Remember that you can only use a const + reference to a container object to call the const + versions of <code class="literal">operator*</code> and + <code class="literal">operator[]</code>. + </p> <p> - You can also use the non-const container object or its non-const reference to create a read - only iterator by passing <code class="literal">true</code> to the - <span class="bold"><strong>readonly</strong></span> parameter in the container's - <code class="methodname">begin()</code> method. - </p> + You can also use the non-const container object + or its non-const reference to create a read only + iterator by passing <code class="literal">true</code> to the + <span class="bold"><strong>readonly</strong></span> + parameter in the container's + <code class="methodname">begin()</code> method. + </p> </li> <li> <p> - Use pre-increment/pre-decrement rather than post-increment/post-decrement where possible - </p> - <p> - Pre-increment operations are more efficient because the <code class="literal">++iterator</code> avoids - two iterator copy constructions. This is true when you are using C++ standard STL iterators - as well. - </p> + Use pre-increment/pre-decrement rather than + post-increment/post-decrement where possible + </p> + <p> + Pre-increment operations are more efficient + because the <code class="literal">++iterator</code> avoids + two iterator copy constructions. This is true when + you are using C++ standard STL iterators as well. + </p> </li> <li> <p> - Use bulk retrieval in iterators - </p> - <p> - If your access pattern is to go through the entire database read only, or if you are reading - a continuous range of the database, bulk retrieval can be very useful because it returns - multiple key/data pairs in one database call. But be aware that you can only read the - returned data, you can not update it. Also, if you do a bulk retrieval and read the data, - and simultaneously some other thread of control updates that same data, then unless you are - using a serializable transaction, you will now be working with old data. - </p> + Use bulk retrieval in iterators + </p> + <p> + If your access pattern is to go through the + entire database read only, or if you are reading a + continuous range of the database, bulk retrieval + can be very useful because it returns multiple + key/data pairs in one database call. But be aware + that you can only read the returned data, you can + not update it. Also, if you do a bulk retrieval + and read the data, and simultaneously some other + thread of control updates that same data, then + unless you are using a serializable transaction, + you will now be working with old data. + </p> </li> </ul> </div> @@ -150,93 +184,115 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1350448"></a>Using containers efficiently</h3> + <h3 class="title"><a id="idp957584"></a>Using containers efficiently</h3> </div> </div> </div> <p> - To make the most efficient possible use of containers: - </p> + To make the most efficient possible use of containers: + </p> <div class="itemizedlist"> <ul type="disc"> <li> <p> - Avoid using container methods that return references. These because they are a little more - expensive. - </p> - <p> - To implement reference semantics, dbstl has to wrap the data element with the current - key/data pair, and must invoke two iterator copy constructions and two Berkeley DB cursor - duplications for each such a call. This is true of non-const versions of these functions: - </p> + Avoid using container methods that return + references. These because they are a little more + expensive. + </p> + <p> + To implement reference semantics, dbstl has to + wrap the data element with the current key/data + pair, and must invoke two iterator copy + constructions and two Berkeley DB cursor + duplications for each such a call. This is true of + non-const versions of these functions: + </p> <table class="simplelist" border="0" summary="Simple list"> <tr> <td> - <code class="methodname">db_vector<T>::operator[]()</code> - </td> + <code class="methodname">db_vector<T>::operator[]()</code> + </td> </tr> <tr> <td> - <code class="methodname">db_vector<T>::front()</code> - </td> + <code class="methodname">db_vector<T>::front()</code> + </td> </tr> <tr> <td> - <code class="methodname">db_vector<T>::back()</code> - </td> + <code class="methodname">db_vector<T>::back()</code> + </td> </tr> <tr> <td> - <code class="methodname">db_vector<T>::at()</code> - </td> + <code class="methodname">db_vector<T>::at()</code> + </td> </tr> <tr> <td> - <code class="methodname">db_map<>::operator[]()</code> - </td> + <code class="methodname">db_map<>::operator[]()</code> + </td> </tr> </table> - <p> - There are alternatives to these functions, mainly through explicit use of iterators. - </p> + <p> + There are alternatives to these functions, + mainly through explicit use of iterators. + </p> </li> <li> - <p> - Use const containers where possible. - </p> - <p> - The const versions of the functions listed above have less overhead than their non-const - counterparts. Using const containers and iterators can bring more performance when you call - the const version of the overloaded container/iterator methods. To do so, you define a const - container reference to an existing container, and then use this reference to call the - methods. For example, if you have: - </p> + <p> + Use const containers where possible. + </p> + <p> + The const versions of the functions listed + above have less overhead than their non-const + counterparts. Using const containers and iterators + can bring more performance when you call the const + version of the overloaded container/iterator + methods. To do so, you define a const container + reference to an existing container, and then use + this reference to call the methods. For example, + if you have: + </p> <pre class="programlisting">db_vector<int> container int_vec</pre> - <p> - then you can define a const reference to <code class="literal">int_vec</code>: - </p> + <p> + then you can define a const reference to + <code class="literal">int_vec</code>: + </p> <pre class="programlisting">const db_vector<int>& int_vec_ref; </pre> - <p> - Then you use <code class="methodname">int_vec_ref.begin()</code> to create a const iterator, - <code class="literal">citr</code>. You can now can use <code class="literal">int_vec_ref</code> to call the - const versions of the container's member functions, and then use <code class="literal">citr</code> to - access the data read only. By using <code class="literal">int_vec_ref</code> and - <code class="literal">citr</code>, we can gain better performance. - </p> - <p> - - It is acceptable to call the non-const versions of container functions that return non-const - iterators, and then assign these return values to const iterator objects. But if you are - using Berkeley DB concurrent data store (CDS), be sure to set the - <span class="bold"><strong>readonly</strong></span> parameter for each container method that returns an - iterator to <code class="literal">true</code>. This is because each iterator corresponds to a Berkeley - DB cursor, and so for best performance you should specify that the returned iterator be - read-only so that the underlying cursor is also read-only. Otherwise, the cursor will be a - writable cursor, and performance might be somewhat degraded. If you are not using CDS, but - instead TDS or DS or HA, there is no distinction between read-only cursors and read-write - cursors. Consequently, you do not need to specify the - <span class="bold"><strong>readonly</strong></span> parameter at all. - </p> + <p> + Then you use + <code class="methodname">int_vec_ref.begin()</code> + to create a const iterator, + <code class="literal">citr</code>. You can now can use + <code class="literal">int_vec_ref</code> to call the + const versions of the container's member + functions, and then use <code class="literal">citr</code> to + access the data read only. By using + <code class="literal">int_vec_ref</code> and + <code class="literal">citr</code>, we can gain better + performance. + </p> + <p> + It is acceptable to call the non-const versions + of container functions that return non-const + iterators, and then assign these return values to + const iterator objects. But if you are using + Berkeley DB concurrent data store (CDS), be sure + to set the <span class="bold"><strong>readonly</strong></span> + parameter for each container method that returns an iterator to + <code class="literal">true</code>. This is because each + iterator corresponds to a Berkeley DB cursor, and + so for best performance you should specify that + the returned iterator be read-only so that the + underlying cursor is also read-only. Otherwise, + the cursor will be a writable cursor, and + performance might be somewhat degraded. If you are + not using CDS, but instead TDS or DS or HA, there + is no distinction between read-only cursors and + read-write cursors. Consequently, you do not need + to specify the <span class="bold"><strong>readonly</strong></span> parameter at all. + </p> </li> </ul> </div> @@ -253,7 +309,8 @@ <td width="40%" align="right"> <a accesskey="n" href="stl_memory_mgmt.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Dbstl container specific notes </td> + <td width="40%" align="left" valign="top">Dbstl container specific + notes </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/stl_examples.html b/docs/programmer_reference/stl_examples.html index f029dadf..0918c31d 100644 --- a/docs/programmer_reference/stl_examples.html +++ b/docs/programmer_reference/stl_examples.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -36,83 +36,91 @@ </div> </div> </div> - <p> - Because dbstl is so much like C++ STL, its usage exactly mirrors that - of C++ STL, with the exception of a few optional Berkeley DB specific - configurations. In fact, the only difference between a program using - dbstl and one using C++ STL is the class names. That is, - <code class="classname">vector</code> becomes <code class="classname">db_vector</code>, - and <code class="classname">map</code> becomes <code class="classname">db_map</code>. -</p> - <p> - The typical procedure for using dbstl is: -</p> + <p> + Because dbstl is so much like C++ STL, its usage exactly + mirrors that of C++ STL, with the exception of a few optional + Berkeley DB specific configurations. In fact, the only + difference between a program using dbstl and one using C++ STL + is the class names. That is, <code class="classname">vector</code> + becomes <code class="classname">db_vector</code>, and + <code class="classname">map</code> becomes + <code class="classname">db_map</code>. + </p> + <p> + The typical procedure for using dbstl is: + </p> <div class="orderedlist"> <ol type="1"> <li> <p> - Optionally create and open your own Berkeley DB environment and - database handles using the DB C++ API. If you perform these opens - using the C++ API, make sure to perform necessary - environment and database configurations at that time. - </p> + Optionally create and open your own Berkeley DB + environment and database handles using the DB C++ API. + If you perform these opens using the C++ API, make + sure to perform necessary environment and database + configurations at that time. + </p> </li> <li> - <p> - Optionally pass environment and database handles to dbstl container - constructors when you create dbstl container objects. Note that you - can create a dbstl container without passing it an environment and - database object. When you do this, an internal anonymous database - is created for you. In this situation, dbstl provides no data - persistence guarantees. - </p> + <p> + Optionally pass environment and database handles to + dbstl container constructors when you create dbstl + container objects. Note that you can create a dbstl + container without passing it an environment and + database object. When you do this, an internal + anonymous database is created for you. In this + situation, dbstl provides no data persistence + guarantees. + </p> </li> <li> - <p> - Perform dbstl-specific configurations. For example, you can - configure cursor open flags, as well as database - access for autocommit. You can also configure callback functions. - </p> + <p> + Perform dbstl-specific configurations. For example, + you can configure cursor open flags, as well as + database access for autocommit. You can also configure + callback functions. + </p> </li> <li> - <p> - Interact with the data contained in your Berkeley DB databases using - dbstl containers and iterators. This usage of dbstl is identical to - C++ STL container and iterator usage. - </p> + <p> + Interact with the data contained in your Berkeley + DB databases using dbstl containers and iterators. + This usage of dbstl is identical to C++ STL container + and iterator usage. + </p> </li> <li> <p> - At this time, you can also use dbstl calls that are specific to Berkeley DB. - For example, you can use Berkeley DB specific calls that manage transaction - begin/commit/abort, handle registration, and so forth. While these - calls are part of dbstl, they have no equivalence in the C++ STL - APIs. - </p> + At this time, you can also use dbstl calls that are + specific to Berkeley DB. For example, you can use + Berkeley DB specific calls that manage transaction + begin/commit/abort, handle registration, and so forth. + While these calls are part of dbstl, they have no + equivalence in the C++ STL APIs. + </p> </li> <li> - <p> - When your application is done using Berkeley DB, you do not need to - explicitly close any Berkeley DB handles (environments, database, - cursors, and so forth). Dbstl automatically closes all such handles - for you. - </p> + <p> When your application is done using Berkeley DB, + you do not need to explicitly close any Berkeley DB + handles (environments, database, cursors, and so + forth). Dbstl automatically closes all such handles + for you. </p> </li> </ol> </div> <p> - For examples of dbstl usage, see the example programs in the - <code class="filename">$db/examples_stl</code> directory. -</p> - <p> - The following program listing provides two code fragments. You can - find more example code in the <code class="filename">dbstl/examples/</code> - and <code class="filename">dbstl/test</code> directories. -</p> + For examples of dbstl usage, see the example programs in + the <code class="filename">$db/examples/stl</code> directory. + </p> + <p> + The following program listing provides two code fragments. + You can find more example code in the + <code class="filename">dbstl/examples/</code> and + <code class="filename">dbstl/test</code> directories. + </p> <pre class="programlisting">//////////////// Code Snippet 1 //////////////// db_vector<int, ElementHolder<int> > vctr(100); for (int i = 0; i < 100; i++) - vctr[i] = i; + vctr[i] = i; for (int i = 0; i < 100; i++) { cout<<"\nvctr["<<i<<"] : "<<vctr[i]; @@ -127,12 +135,12 @@ strmap_t2 strmap; char str[2], str2[2]; str[1] = str2[1] = '\0'; for (char c = 0; c < 26; c++) { - str[0] = c + 'a'; - str2[0] = 'z' - c; - strmap[str] = str2; + str[0] = c + 'a'; + str2[0] = 'z' - c; + strmap[str] = str2; } for (strmap_t2::iterator itr = strmap.begin(); itr != strmap.end(); ++itr) - cout<<endl<<itr->first<<" : "<<itr->second; + cout<<endl<<itr->first<<" : "<<itr->second; using namespace dbstl; dbstl::db_map<char, int> v; @@ -156,33 +164,36 @@ vi.push_back(obj); // More person storage. for (int I = 0; I < vi.size(); I++) cout<<vi[I]; </pre> <p> - The first snippet initializes a db_vector container of 100 elements, - with an in-memory anonymous database internally created by dbstl. The - only difference between this and C++ STL is dbstl requires one more - <code class="literal">type</code> parameter: - <em class="parameter"><code>ElementHolder<int></code></em>. The - <code class="classname">ElementHolder</code> class template should be used for - every type of dbstl container that will store C++ primitive data types, - such as <code class="literal">int</code>, <code class="literal">float</code>, - <code class="literal">char *</code>, <code class="literal">wchar_t *</code>, and so forth. But these - class templates should not be used for class types for reasons that we - explain in the following chapters. -</p> - <p> - In the second code snippet, the assignment: -</p> + The first snippet initializes a db_vector container of 100 + elements, with an in-memory anonymous database internally + created by dbstl. The only difference between this and C++ STL + is dbstl requires one more <code class="literal">type</code> parameter: + <em class="parameter"><code>ElementHolder<int></code></em>. The + <code class="classname">ElementHolder</code> class template should + be used for every type of dbstl container that will store C++ + primitive data types, such as <code class="literal">int</code>, + <code class="literal">float</code>, <code class="literal">char *</code>, + <code class="literal">wchar_t *</code>, and so forth. But these + class templates should not be used for class types for reasons + that we explain in the following chapters. + </p> + <p> + In the second code snippet, the assignment: + </p> <pre class="programlisting">strmap[str] = str2;</pre> <p> - is used to store a string pair (<code class="literal">(str, str2)</code>) instead - of pointers to the underlying database. -</p> + is used to store a string pair (<code class="literal">(str, + str2)</code>) instead of pointers to the underlying + database. + </p> <p> - The rest of the code used in these snippets is identical to the code - you would use for C++ STL containers. However, by using dbstl, you are - storing data into a Berkeley DB database. If you create your own - database with backing files on disk, your data or objects can persist - and be restored when the program runs again. -</p> + The rest of the code used in these snippets is identical to + the code you would use for C++ STL containers. However, by + using dbstl, you are storing data into a Berkeley DB database. + If you create your own database with backing files on disk, + your data or objects can persist and be restored when the + program runs again. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/stl_known_issues.html b/docs/programmer_reference/stl_known_issues.html index f05483e5..5c8d9e05 100644 --- a/docs/programmer_reference/stl_known_issues.html +++ b/docs/programmer_reference/stl_known_issues.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -36,42 +36,51 @@ </div> </div> </div> + <p> + Three algorithm functions of gcc's C++ STL test suite do + not work with dbstl. They are <code class="function">find_end()</code>, + <code class="function">inplace_merge()</code> and + <code class="function">stable_sort()</code>. + </p> + <p> + The reason for the incompatibility of + <code class="function">find_end()</code> is that it assumes the + data an iterator refers to is located at a shared place (owned + by its container). This assumption is not correct in that it + is part of the C++ STL standards specification. However, this + assumption can not be true for dbstl because each dbstl + container iterator caches its referenced value. + </p> <p> - Three algorithm functions of gcc's C++ STL test suite do not work with dbstl. They are - <code class="function">find_end()</code>, <code class="function">inplace_merge()</code> and - <code class="function">stable_sort()</code>. -</p> - <p> - The reason for the incompatibility of <code class="function">find_end()</code> is that it assumes the - data an iterator refers to is located at a shared place (owned by its container). This - assumption is not correct in that it is part of the C++ STL standards specification. However, - this assumption can not be true for dbstl because each dbstl container iterator caches its - referenced value. -</p> - <p> - Consequently, please do not use <code class="function">find_end()</code> for dbstl container iterators if - you are using gcc's STL library. -</p> - <p> - The reason for the incompatibility with <code class="function">inplace_merge()</code> and - <code class="function">stable_sort()</code> is that their implementation in gcc requires the - <span class="bold"><strong>value_type</strong></span> for a container to be default constructible. This - requirement is not a part of the the C++ STL standard specification. Dbstl's value type - wrappers (such as <code class="classname">ElementHolder</code>) do not support it. -</p> - <p> - These issues do not exist for any function available with the Microsoft - Visual C++ 8 STL library. There are two algorithm functions of Microsoft - Visual C++ 10 STL library that do have an issue: - <code class="function">partial_sort()</code> and - <code class="function">partial_sort_copy()</code>. These are not compatible because they - require the dbstl <code class="literal">vector</code> iterator to create a new - element when updating the current element. Dbstl - <code class="literal">vector</code> iterator can copy the new content to the - current element, but it cannot create a new one. This requirement is - not a part of the C++ STL standard specification, and so dbstl's - <code class="literal">vector</code> iterator does not support it. -</p> + Consequently, please do not use + <code class="function">find_end()</code> for dbstl container + iterators if you are using gcc's STL library. + </p> + <p> + The reason for the incompatibility with + <code class="function">inplace_merge()</code> and + <code class="function">stable_sort()</code> is that their + implementation in gcc requires the <span class="bold"><strong> + value_type</strong></span> for a container to be default + constructible. This requirement is not a part of the the C++ + STL standard specification. Dbstl's value type wrappers (such + as <code class="classname">ElementHolder</code>) do not support it. + </p> + <p> + These issues do not exist for any function available with + the Microsoft Visual C++ 8 STL library. There are two + algorithm functions of Microsoft Visual C++ 10 STL library + that do have an issue: <code class="function">partial_sort()</code> and + <code class="function">partial_sort_copy()</code>. These are not + compatible because they require the dbstl + <code class="literal">vector</code> iterator to create a new element + when updating the current element. Dbstl + <code class="literal">vector</code> iterator can copy the new + content to the current element, but it cannot create a new + one. This requirement is not a part of the C++ STL standard + specification, and so dbstl's <code class="literal">vector</code> + iterator does not support it. + </p> </div> <div class="navfooter"> <hr /> @@ -88,9 +97,7 @@ <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 8. - Berkeley DB Architecture - </td> + <td width="40%" align="right" valign="top"> Chapter 8. Berkeley DB Architecture </td> </tr> </table> </div> diff --git a/docs/programmer_reference/stl_memory_mgmt.html b/docs/programmer_reference/stl_memory_mgmt.html index a32bbef5..f82eba9f 100644 --- a/docs/programmer_reference/stl_memory_mgmt.html +++ b/docs/programmer_reference/stl_memory_mgmt.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -40,12 +40,12 @@ <dl> <dt> <span class="sect2"> - <a href="stl_memory_mgmt.html#idp1384984">Freeing memory</a> + <a href="stl_memory_mgmt.html#idp952296">Freeing memory</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_memory_mgmt.html#idp1389512">Type specific notes</a> + <a href="stl_memory_mgmt.html#idp994336">Type specific notes</a> </span> </dt> </dl> @@ -54,25 +54,27 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1384984"></a>Freeing memory</h3> + <h3 class="title"><a id="idp952296"></a>Freeing memory</h3> </div> </div> </div> - <p> - When using dbstl, make sure memory allocated in the heap is - released after use. The rules for this are: - </p> + <p> + When using dbstl, make sure memory allocated in the + heap is released after use. The rules for this are: + </p> <div class="itemizedlist"> <ul type="disc"> <li> - <p> - dbstl will free/delete any memory allocated by dbstl itself. - </p> + <p> + dbstl will free/delete any memory allocated by + dbstl itself. + </p> </li> <li> <p> - You are responsible for freeing/deleting any memory allocated by your code outside of dbstl. - </p> + You are responsible for freeing/deleting any + memory allocated by your code outside of dbstl. + </p> </li> </ul> </div> @@ -81,7 +83,7 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1389512"></a>Type specific notes</h3> + <h3 class="title"><a id="idp994336"></a>Type specific notes</h3> </div> </div> </div> @@ -89,82 +91,126 @@ <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idp1425304"></a>DbEnv/Db</h4> + <h4 class="title"><a id="idp993688"></a>DbEnv/Db</h4> </div> </div> </div> + <p> + When you open a <code class="classname">DbEnv</code> or + <code class="classname">Db</code> object using + <code class="methodname">dbstl::open_env()</code> or + <code class="methodname">dbstl::open_db()</code>, you do + not need to delete that object. However, if you new'd + that object and then opened it without using the + <code class="methodname">dbstl::open_env()</code> or + <code class="methodname">dbstl::open_db()</code> methods, + you are responsible for deleting the object. + </p> <p> - When you open a <code class="classname">DbEnv</code> or <code class="classname">Db</code> object using - <code class="methodname">dbstl::open_env()</code> or <code class="methodname">dbstl::open_db()</code>, you - do not need to delete that object. However, if you new'd that object and then opened it - without using the <code class="methodname">dbstl::open_env()</code> or - <code class="methodname">dbstl::open_db()</code> methods, you are responsible for deleting the - object. - </p> - <p> - Note that you must <code class="literal">new</code> the <code class="classname">Db</code> or - <code class="classname">DbEnv</code> object, which allocates it on the heap. You can not allocate it - on the stack. If you do, the order of destruction is uncontrollable, which makes dbstl - unable to work properly. - </p> + Note that you must <code class="literal">new</code> the + <code class="classname">Db</code> or + <code class="classname">DbEnv</code> object, which + allocates it on the heap. You can not allocate it on + the stack. If you do, the order of destruction is + uncontrollable, which makes dbstl unable to work + properly. + </p> + <p> + You can call <code class="function">dbstl_exit()</code> + before the process exits, to release any memory + allocated by dbstl that has to live during the entire + process lifetime. Releasing the memory explicitly will + not make much difference, because the process is about + to exit and so all memory allocated on the heap is + going to be returned to the operating system anyway. + The only real difference is that your memory leak + checker will not report false memory leaks. + </p> <p> - You can call <code class="function">dbstl_exit()</code> before the process exits, to release any - memory allocated by dbstl that has to live during the entire process lifetime. Releasing the - memory explicitly will not make much difference, because the process is about to exit and so - all memory allocated on the heap is going to be returned to the operating system anyway. The - only real difference is that your memory leak checker will not report false memory leaks. - </p> + <code class="function">dbstl_exit()</code> releases any memory + allocated by dbstl on the heap. It also performs other + required shutdown operations, such as closing any + databases and environments registered to dbstl and + shared across the process. + </p> <p> - <code class="function">dbstl_exit()</code> releases any memory allocated by dbstl on the heap. It - also performs other required shutdown operations, such as closing any databases and - environments registered to dbstl and shared across the process. - </p> - <p> - If you are calling the <code class="function">dbstl_exit()</code> function, and your - <code class="classname">DbEnv</code> or <code class="classname">Db</code> objects are new'd by your code, - the <code class="function">dbstl_exit()</code> function should be called before deleting the - <code class="classname">DbEnv</code> or <code class="classname">Db</code> objects, because they need to be - closed before being deleted. Alternatively, you can call the - <code class="methodname">dbstl::close_env()</code> or <code class="methodname">dbstl::close_db()</code> - functions before deleting the <code class="classname">DbEnv</code> or <code class="classname">Db</code> - objects in order to explicitly close the databases or environments. If you do this, - can then delete these objects, and then call <code class="function">dbstl_exit()</code>. - </p> - <p> - In addition, before exiting a thread that uses dbstl API, you can call the <code class="function">dbstl_thread_exit() </code>function to release any Berkeley DB handles if they are not used by other threads. - If you do not call the <code class="function">dbstl_thread_exit() </code>function or call this function only in some threads, all open Berkeley DB handles will be closed by the <code class="function">dbstl_exit()</code>function. - You must call the <code class="function">dbstl_exit() </code>function before the process exits, to avoid memory leak and database update loss, if you do not have transactions and persistent log files. - </p> + If you are calling the + <code class="function">dbstl_exit()</code> function, and + your <code class="classname">DbEnv</code> or + <code class="classname">Db</code> objects are new'd by + your code, the <code class="function">dbstl_exit()</code> + function should be called before deleting the + <code class="classname">DbEnv</code> or + <code class="classname">Db</code> objects, because they + need to be closed before being deleted. Alternatively, + you can call the + <code class="methodname">dbstl::close_env()</code> or + <code class="methodname">dbstl::close_db()</code> + functions before deleting the + <code class="classname">DbEnv</code> or + <code class="classname">Db</code> objects in order to + explicitly close the databases or environments. If you + do this, can then delete these objects, and then call + <code class="function">dbstl_exit()</code>. </p> + <p> + In addition, before exiting a thread that uses + dbstl API, you can call the + <code class="function">dbstl_thread_exit() </code>function + to release any Berkeley DB handles if they are not + used by other threads. If you do not call the + <code class="function">dbstl_thread_exit() </code>function + or call this function only in some threads, all open + Berkeley DB handles will be closed by the + <code class="function">dbstl_exit()</code>function. You + must call the <code class="function">dbstl_exit() + </code>function before the process exits, to avoid + memory leak and database update loss, if you do not + have transactions and persistent log files. + </p> </div> <div class="sect3" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h4 class="title"><a id="idp1410936"></a>DbstlDbt</h4> + <h4 class="title"><a id="idp953376"></a>DbstlDbt</h4> </div> </div> </div> <p> - Only when you are storing raw bytes (such as a bitmap) do you have to - store and retrieve data by using the <code class="classname">DbstlDbt</code> helper class. Although you - also can do so simply by using the Berkeley DB <code class="classname">Dbt</code> class, the - <code class="classname">DbstlDbt</code> class offers more convenient memory management behavior. - </p> - <p> - When you are storing <code class="classname">DbstlDbt</code> objects (such as - <code class="classname">db_vector<DbstlDbt></code>), you <span class="emphasis"><em>must</em></span> allocate - heap memory explicitly using the <code class="function">malloc()</code> function for the - <code class="classname">DbstlDbt</code> object to reference, but you do not need to free the memory - – it is automatically freed by the <code class="classname">DbstlDbt</code> object that owns it - by calling the standard C library <code class="function">free()</code> function. - </p> - <p> - However, because dbstl supports storing any type of object or primitive data, it is rare - that you would have to store data using <code class="classname">DbstlDbt</code> objects while using - dbstl. Examples of storing <code class="classname">DbstlDbt</code> objects can be found in the - <code class="methodname">StlAdvancedFeaturesExample::arbitrary_object_storage()</code> and - <code class="methodname">StlAdvancedFeaturesExample::char_star_string_storage()</code> methods. - </p> + Only when you are storing raw bytes (such as a + bitmap) do you have to store and retrieve data by + using the <code class="classname">DbstlDbt</code> helper + class. Although you also can do so simply by using the + Berkeley DB <code class="classname">Dbt</code> class, the + <code class="classname">DbstlDbt</code> class offers more + convenient memory management behavior. + </p> + <p> + When you are storing + <code class="classname">DbstlDbt</code> objects (such as + <code class="classname">db_vector<DbstlDbt></code>), + you <span class="emphasis"><em>must</em></span> allocate heap memory + explicitly using the <code class="function">malloc()</code> + function for the <code class="classname">DbstlDbt</code> + object to reference, but you do not need to free the + memory – it is automatically freed by the + <code class="classname">DbstlDbt</code> object that owns + it by calling the standard C library + <code class="function">free()</code> function. + </p> + <p> + However, because dbstl supports storing any type of + object or primitive data, it is rare that you would + have to store data using + <code class="classname">DbstlDbt</code> objects while + using dbstl. Examples of storing + <code class="classname">DbstlDbt</code> objects can be + found in the + <code class="methodname">StlAdvancedFeaturesExample::arbitrary_object_storage()</code> + and + <code class="methodname">StlAdvancedFeaturesExample::char_star_string_storage()</code> + methods. + </p> </div> </div> </div> @@ -179,7 +225,8 @@ <td width="40%" align="right"> <a accesskey="n" href="stl_misc.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Using dbstl efficiently </td> + <td width="40%" align="left" valign="top">Using dbstl + efficiently </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/stl_misc.html b/docs/programmer_reference/stl_misc.html index 0647d6ec..90f3d452 100644 --- a/docs/programmer_reference/stl_misc.html +++ b/docs/programmer_reference/stl_misc.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -40,12 +40,13 @@ <dl> <dt> <span class="sect2"> - <a href="stl_misc.html#idp1407848">Special notes about trivial methods</a> + <a href="stl_misc.html#idp953048">Special notes about trivial methods</a> </span> </dt> <dt> <span class="sect2"> - <a href="stl_misc.html#idp1421568">Using correct container and iterator public types</a> + <a href="stl_misc.html#idp993208">Using correct container and iterator public + types</a> </span> </dt> </dl> @@ -54,76 +55,81 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1407848"></a>Special notes about trivial methods</h3> + <h3 class="title"><a id="idp953048"></a>Special notes about trivial methods</h3> </div> </div> </div> - <p> - There are some standard STL methods which are meaningless in dbstl, but they are kept in - dbstl as no-ops so as to stay consistent with the standard. These are: - </p> + <p> + There are some standard STL methods which are + meaningless in dbstl, but they are kept in dbstl as no-ops + so as to stay consistent with the standard. These are: + </p> <table class="simplelist" border="0" summary="Simple list"> <tr> <td> - <code class="methodname">db_vecter::reserve();</code> + <code class="methodname">db_vecter::reserve();</code> </td> </tr> <tr> <td> - <code class="methodname">db_vector::max_size();</code> + <code class="methodname">db_vector::max_size();</code> </td> </tr> <tr> <td> - <code class="methodname">db_vector::capacity();</code> + <code class="methodname">db_vector::capacity();</code> </td> </tr> <tr> <td> - <code class="methodname">db_map::reserve();</code> + <code class="methodname">db_map::reserve();</code> </td> </tr> <tr> <td> - <code class="methodname">db_map::max_size();</code> + <code class="methodname">db_map::max_size();</code> </td> </tr> </table> <p> - <code class="methodname">db_vector<>::max_size()</code> and - <code class="methodname">db_map<>::max_size()</code> both return 2^30. This does not mean - that Berkeley DB can only hold that much data. This value is returned to conform to some - compilers' overflow rules — if we set bigger numbers like 2^32 or 2^31, some compilers - complain that the number has overflowed. - </p> - <p> - See the Berkeley DB documentation for information about limitations on how much data a - database can store. - </p> + <code class="methodname">db_vector<>::max_size()</code> and + <code class="methodname">db_map<>::max_size()</code> + both return 2^30. This does not mean that Berkeley DB can + only hold that much data. This value is returned to + conform to some compilers' overflow rules — if we + set bigger numbers like 2^32 or 2^31, some compilers + complain that the number has overflowed. + </p> + <p> + See the Berkeley DB documentation for information about + limitations on how much data a database can store. + </p> <p> - There are also some read-only functions. You set the configuration - for these using the Berkeley DB API. You access them using the container's methods. Again, - this is to keep consistent with C++ standard STL containers, such as: - </p> + There are also some read-only functions. You set the + configuration for these using the Berkeley DB API. You + access them using the container's methods. Again, this is + to keep consistent with C++ standard STL containers, such + as: + </p> <table class="simplelist" border="0" summary="Simple list"> <tr> <td> - <code class="methodname">db_map::key_comp();</code> + <code class="methodname">db_map::key_comp();</code> </td> </tr> <tr> <td> - <code class="methodname">db_map::value_comp();</code> + <code class="methodname">db_map::value_comp();</code> </td> </tr> <tr> <td> - <code class="methodname">db_map::hash_funct();</code> + <code class="methodname">db_map::hash_funct();</code> </td> </tr> <tr> <td> - <code class="methodname">db_map::key_eq();</code> + <code class="methodname">db_map::key_eq();</code> </td> </tr> </table> @@ -132,48 +138,60 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1421568"></a>Using correct container and iterator public types</h3> + <h3 class="title"><a id="idp993208"></a>Using correct container and iterator public + types</h3> </div> </div> </div> + <p> + All public types defined by the C++ STL specification + are present in dbstl. One thing to note is the <span class="bold"><strong>value_type</strong></span>. dbstl defines the + <span class="bold"><strong>value_type</strong></span> for each + iterator and container class to be the raw type without + the + <code class="classname">ElementRef</code>/<code class="classname">ElementHolder</code> + wrapper, so this type of variable can not be used to store + data in a database. There is a <span class="bold"><strong>value_type_wrap</strong></span> + type for each container + and iterator type, with the raw type wrapped by the + <code class="classname">ElementRef</code>/<code class="classname">ElementHolder</code>. + </p> <p> - All public types defined by the C++ STL specification are present in dbstl. One thing to - note is the <span class="bold"><strong>value_type</strong></span>. dbstl defines the <span class="bold"><strong>value_type</strong></span> for each iterator and container class to be the raw - type without the <code class="classname">ElementRef</code>/<code class="classname">ElementHolder</code> - wrapper, so this type of variable can not be used to store data in a database. There is a - <span class="bold"><strong>value_type_wrap</strong></span> type for each container and iterator type, - with the raw type wrapped by the - <code class="classname">ElementRef</code>/<code class="classname">ElementHolder</code>. - </p> - <p> - For example, when type <code class="literal">int_vector_t</code> is defined as - </p> + For example, when type <code class="literal">int_vector_t</code> + is defined as + </p> <pre class="programlisting">db_vector<int, ElementHolder<int> ></pre> <p> - its <span class="bold"><strong>value_type</strong></span> is <code class="literal">int</code>, its - <span class="bold"><strong>value_type_wrap</strong></span> is <code class="literal">ElementHolder<int></code>, - and its reference and pointer types are <code class="literal">ElementHolder<int>&</code> and - <code class="literal">ElementHolder<int>*</code> respectively. If you need to store data, use - <span class="bold"><strong>value_type_wrap</strong></span> to make use of the wrapper to store data - into database. - </p> - <p> - The reason we leave <span class="bold"><strong>value_type</strong></span> as the raw type is that we - want the existing algorithms in the STL library to work with dbstl because we have seen that - without doing so, a few tests will fail. - </p> - <p> - You need to use the same type as the return type of the data element retrieval functions to - hold a value in order to properly manipulate the data element. For example, when calling - </p> + its <span class="bold"><strong>value_type</strong></span> is + <code class="literal">int</code>, its <span class="bold"><strong>value_type_wrap</strong></span> + is <code class="literal">ElementHolder<int></code>, and its + reference and pointer types are + <code class="literal">ElementHolder<int>&</code> and + <code class="literal">ElementHolder<int>*</code> + respectively. If you need to store data, use <span class="bold"><strong>value_type_wrap</strong></span> to make use of + the wrapper to store data into database. + </p> + <p> + The reason we leave <span class="bold"><strong>value_type</strong></span> + as the raw type is that we want the existing algorithms in the STL + library to work with dbstl because we have seen that without doing so, + a few tests will fail. + </p> + <p> + You need to use the same type as the return type of the + data element retrieval functions to hold a value in order + to properly manipulate the data element. For example, when + calling + </p> <pre class="programlisting">db_vector<T>::operator[]</pre> <p> - check that the return type for this function is - </p> + check that the return type for this function is + </p> <pre class="programlisting">db_vector<T>::datatype_wrap</pre> - <p> - Then, hold the return value using an object of the same type: - </p> + <p> + Then, hold the return value using an object of the same + type: + </p> <pre class="programlisting">db_vector<T>::datatype_wrap refelem = vctr[3];</pre> </div> </div> diff --git a/docs/programmer_reference/stl_mt_usage.html b/docs/programmer_reference/stl_mt_usage.html index f4f4f254..ce60d95b 100644 --- a/docs/programmer_reference/stl_mt_usage.html +++ b/docs/programmer_reference/stl_mt_usage.html @@ -14,11 +14,12 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Using dbstl in multithreaded applications</th> + <th colspan="3" align="center">Using dbstl in multithreaded + applications</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="stl_txn_usage.html">Prev</a> </td> @@ -32,168 +33,191 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="stl_mt_usage"></a>Using dbstl in multithreaded applications</h2> + <h2 class="title" style="clear: both"><a id="stl_mt_usage"></a>Using dbstl in multithreaded + applications</h2> </div> </div> </div> - <p> - Multithreaded use of dbstl must obey the following guidelines: -</p> + <p> + Multithreaded use of dbstl must obey the following + guidelines: + </p> <div class="orderedlist"> <ol type="1"> <li> <p> - For a few non-standard platforms, you must first configure dbstl for that - platform, but usually the configure script will - detect the applicable thread local storage (TLS) modifier to use, and - then use it. If no appropriate TLS is found, the pthread TLS API is used. -</p> + For a few non-standard platforms, you must first + configure dbstl for that platform, but usually the + configure script will detect the applicable thread + local storage (TLS) modifier to use, and then use it. + If no appropriate TLS is found, the pthread TLS API is + used. + </p> </li> <li> <p> - Perform all initializations in a single thread. - <code class="methodname">dbstl::dbstl_startup()</code> should be called - mutually exclusive in a single thread before using dbstl. If dbstl is - used in only a single thread, this function does not need to be called. -</p> - <p> - If necessary, callback functions for a complex type T must be - registered to the singleton of - DbstlElemTraits<T> before any container related to T (for - example, <code class="literal">db_vector<T></code>), is used, and certain - isolation may be required among multiple threads. The best way to do - this is to register all callback function pointers into the singleton - in a single thread before making use of the containers. -</p> + Perform all initializations in a single thread. + <code class="methodname">dbstl::dbstl_startup()</code> + should be called mutually exclusive in a single thread + before using dbstl. If dbstl is used in only a single + thread, this function does not need to be called. + </p> + <p> + If necessary, callback functions for a complex type + T must be registered to the singleton of + DbstlElemTraits<T> before any container related + to T (for example, + <code class="literal">db_vector<T></code>), is used, + and certain isolation may be required among multiple + threads. The best way to do this is to register all + callback function pointers into the singleton in a + single thread before making use of the containers. + </p> <p> - All container cursor open flags and auto commit transaction - begin/commit flags must be set in a single thread before storing - objects into or reading objects from the container. -</p> + All container cursor open flags and auto commit + transaction begin/commit flags must be set in a single + thread before storing objects into or reading objects + from the container. + </p> </li> <li> <p> - Environment and database handles can optionally be shared across - threads. If handles are shared, they must be registered in each thread - that is using the handle (either directly, or indirectly using the - containers that own the handles). You do this using the - <code class="function">dbstl::register_db()</code> and - <code class="function">dbstl::register_db_env()</code> functions. Note that - these functions are not necessary if the current thread called - <code class="function">dbstl::open_db()</code> or - <code class="function">dbstl::open_env()</code> for the handle that is being - shared. This is because the open functions automatically register the - handle for you. -</p> - <p> - Note that the get/set functions that provide access to container data - members are not mutex-protected because these data members are supposed - to be set only once at container object initialization. Applications - wishing to modify them after initialization must supply their own - protection. -</p> + Environment and database handles can optionally be + shared across threads. If handles are shared, they + must be registered in each thread that is using the + handle (either directly, or indirectly using the + containers that own the handles). You do this using + the <code class="function">dbstl::register_db()</code> and + <code class="function">dbstl::register_db_env()</code> + functions. Note that these functions are not necessary + if the current thread called + <code class="function">dbstl::open_db()</code> or + <code class="function">dbstl::open_env()</code> for the + handle that is being shared. This is because the open + functions automatically register the handle for you. + </p> + <p> + Note that the get/set functions that provide access + to container data members are not mutex-protected + because these data members are supposed to be set only + once at container object initialization. Applications + wishing to modify them after initialization must + supply their own protection. + </p> </li> <li> <p> - While container objects can be shared between multiple threads, - iterators and transactions can not be shared. -</p> + While container objects can be shared between + multiple threads, iterators and transactions can not + be shared. + </p> </li> <li> <p> - Set the <span class="bold"><strong>directdb_get</strong></span> parameter of the - container <code class="methodname">begin()</code> method to - <code class="literal">true</code> in order to guarantee that referenced key/data - pairs are always obtained from the database and not from an iterator's - cached value. (This is the default behavior.) You should do this - because otherwise a rare situation may occur. Given db_vector_iterator - i1 and i2 used in the same iteration, setting *i1 = new_value will not - update i2, and *i2 will return the original value. -</p> + Set the <span class="bold"><strong>directdb_get</strong></span> parameter of the + container <code class="methodname">begin()</code> method to + <code class="literal">true</code> in order to guarantee that + referenced key/data pairs are always obtained from the + database and not from an iterator's cached value. + (This is the default behavior.) You should do this + because otherwise a rare situation may occur. Given + db_vector_iterator i1 and i2 used in the same + iteration, setting *i1 = new_value will not update i2, + and *i2 will return the original value. + </p> </li> <li> - <p> - If using a CDS database, only const iterators or read-only non-const - iterators should be used for read only iterations. Otherwise, when - multiple threads try to open read-write iterators at the same time, - performance is greatly degraded because CDS only supports one write cursor - open at any moment. The use of read-only iterators is good practice - in general because dbstl contains internal optimizations for read-only - iterators. -</p> - <p> - To create a read-only iterator, do one of the following: -</p> + <p> + If using a CDS database, only const iterators or + read-only non-const iterators should be used for read + only iterations. Otherwise, when multiple threads try + to open read-write iterators at the same time, + performance is greatly degraded because CDS only + supports one write cursor open at any moment. The use + of read-only iterators is good practice in general + because dbstl contains internal optimizations for + read-only iterators. + </p> + <p> + To create a read-only iterator, do one of the + following: + </p> <div class="itemizedlist"> <ul type="disc"> <li> - <p> - Use a <code class="literal">const</code> reference to the container - object, then call the container's - <code class="methodname">begin()</code> method using the const - reference, and then store the return value from the - <code class="methodname">begin()</code> method in a - <code class="methodname">db_vector::const_iterator</code>. - </p> + <p> + Use a <code class="literal">const</code> reference to + the container object, then call the + container's <code class="methodname">begin()</code> + method using the const reference, and then + store the return value from the + <code class="methodname">begin()</code> method in + a + <code class="methodname">db_vector::const_iterator</code>. + </p> </li> <li> - <p> - If you are using a non-const container object, then simply - pass <code class="literal">true</code> to the - <span class="bold"><strong>readonly</strong></span> parameter of the - non-const <code class="methodname">begin()</code> method. - </p> + <p> + If you are using a non-const container + object, then simply pass + <code class="literal">true</code> to the <span class="bold"><strong>readonly</strong></span> parameter + of the non-const + <code class="methodname">begin()</code> method. + </p> </li> </ul> </div> </li> <li> - <p> - When using DS, CDS or TDS, enable the locking subsystem by passing the - <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOCK" class="olink">DB_INIT_LOCK</a> flag to <code class="methodname">DbEnv::open()</code>. -</p> + <p> + When using DS, CDS or TDS, enable the locking + subsystem by passing the <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOCK" class="olink">DB_INIT_LOCK</a> flag to + <code class="methodname">DbEnv::open()</code>. + </p> </li> <li> - <p> - Perform portable thread synchronization within a process by calling the - following functions. These are all global functions in the "dbstl" name - space: -</p> + <p> + Perform portable thread synchronization within a + process by calling the following functions. These are + all global functions in the "dbstl" name space: + </p> <table class="simplelist" border="0" summary="Simple list"> <tr> <td> - <code class="function">db_mutex_t alloc_mutex();</code> + <code class="function">db_mutex_t alloc_mutex();</code> </td> </tr> <tr> <td> - <code class="function">int lock_mutex(db_mutex_t);</code> + <code class="function">int lock_mutex(db_mutex_t);</code> </td> </tr> <tr> <td> - <code class="function">int unlock_mutex(db_mutex_t);</code> + <code class="function">int unlock_mutex(db_mutex_t);</code> </td> </tr> <tr> <td> - <code class="function">void free_mutex(db_mutex_t);</code> + <code class="function">void free_mutex(db_mutex_t);</code> </td> </tr> </table> - <p> - These functions use an internal dbstl environment's mutex functionality - to synchronize. As a result, the synchronization is portable across all - platforms supported by Berkeley DB. -</p> + <p> + These functions use an internal dbstl environment's + mutex functionality to synchronize. As a result, the + synchronization is portable across all platforms + supported by Berkeley DB. + </p> </li> </ol> </div> - <p> - The <code class="classname">WorkerThread</code> class provides example code - demonstrating the use of dbstl in multi-threaded applications. You can - find this class implemented in the dbstl test suite. -</p> + <p> + The <code class="classname">WorkerThread</code> class provides + example code demonstrating the use of dbstl in multi-threaded + applications. You can find this class implemented in the dbstl + test suite. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/stl_persistence.html b/docs/programmer_reference/stl_persistence.html index 343923f5..32d99b5f 100644 --- a/docs/programmer_reference/stl_persistence.html +++ b/docs/programmer_reference/stl_persistence.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -55,10 +55,10 @@ </dt> </dl> </div> - <p> - The following sections provide information on how to achieve - persistence using dbstl. -</p> + <p> + The following sections provide information on how to + achieve persistence using dbstl. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> @@ -68,112 +68,121 @@ </div> </div> <p> - Each container has a <span class="bold"><strong>begin()</strong></span> method - which produces an iterator. These - <span class="bold"><strong>begin</strong></span> methods take a boolean parameter, - <span class="bold"><strong>directdb_get</strong></span>, which controls the - caching behavior of the iterator. The default value of this parameter - is <code class="literal">true</code>. -</p> - <p> - If <span class="bold"><strong>directdb_get</strong></span> is - <code class="literal">true</code>, then the persistent object is fetched anew - from the database each time the iterator is dereferenced as a pointer - by use of the star-operator - (<span class="bold"><strong>*iterator</strong></span>) or by use of the arrow-operator - (<span class="bold"><strong>iterator->member</strong></span>). If - <span class="bold"><strong>directdb_get</strong></span> is <code class="literal">false</code>, then - the first dereferencing of the iterator fetches the object from the - database, but later dereferences can return cached data. -</p> - <p> - With <span class="bold"><strong>directdb_get</strong></span> set to <code class="literal">true</code>, if you call: -</p> + Each container has a <span class="bold"><strong>begin()</strong></span> method + which produces an iterator. + These <span class="bold"><strong>begin</strong></span> methods take + a boolean parameter, <span class="bold"><strong>directdb_get</strong></span>, which + controls the caching behavior of the iterator. The default value of this + parameter is <code class="literal">true</code>. + </p> + <p> + If <span class="bold"><strong>directdb_get</strong></span> is + <code class="literal">true</code>, then the persistent object is + fetched anew from the database each time the iterator is + dereferenced as a pointer by use of the star-operator + (<span class="bold"><strong>*iterator</strong></span>) or by use + of the arrow-operator (<span class="bold"><strong>iterator->member</strong></span>). + If <span class="bold"><strong>directdb_get</strong></span> is + <code class="literal">false</code>, then the first dereferencing + of the iterator fetches the object from the database, but + later dereferences can return cached data. + </p> + <p> + With <span class="bold"><strong>directdb_get</strong></span> set + to <code class="literal">true</code>, if you call: + </p> <pre class="programlisting">(*iterator).datamember1=new-value1; (*iterator).datamember2=new-value2; </pre> <p> - then the assignment to <code class="literal">datamember1</code> will be lost, - because the second dereferencing of the iterator would cause the cached - copy of the object to be overwritten by the object's persistent data - from the database. -</p> + then the assignment to <code class="literal">datamember1</code> + will be lost, because the second dereferencing of the + iterator would cause the cached copy of the object to be + overwritten by the object's persistent data from the + database. + </p> <p> - You also can use the arrow operator like this: -</p> + You also can use the arrow operator like this: + </p> <pre class="programlisting">iterator->datamember1=new-value1; iterator->datamember2=new-value2; </pre> <p> - This works exactly the same way as <span class="bold"><strong>iterator::operator*</strong></span>. - For this reason, the same caching rules apply to arrow operators as they do - for star operators. -</p> - <p> - One way to avoid this problem is to create a reference to the object, - and use it to access the object: -</p> + This works exactly the same way as <span class="bold"><strong>iterator::operator*</strong></span>. For this + reason, the same caching rules apply to arrow operators as + they do for star operators. + </p> + <p> + One way to avoid this problem is to create a reference + to the object, and use it to access the object: + </p> <pre class="programlisting">container::value_type &ref = *iterator; ref.datamember1=new-value1; ref.datamember2=new-value2; ...// more member function calls and datamember assignments ref._DB_STL_StoreElement(); </pre> - <p> - The above code will not lose the newly assigned value of <code class="literal">ref.datamember1</code> - in the way that the previous example did. -</p> - <p> - In order to avoid these complications, you can assign to the object - referenced by an iterator with another object of the same type like this: -</p> + <p> + The above code will not lose the newly assigned value + of <code class="literal">ref.datamember1</code> in the way that the + previous example did. + </p> + <p> + In order to avoid these complications, you can assign + to the object referenced by an iterator with another + object of the same type like this: + </p> <pre class="programlisting">container::value_type obj2; obj2.datamember1 = new-value1; obj2.datamember2 = new-value2; *itr = obj2; </pre> <p> - This code snippet causes the new values in <code class="literal">obj2</code> to - be stored into the underlying database. -</p> - <p> - If you have two iterators going through the same container like this: -</p> - <pre class="programlisting"> -for (iterator1 = v.begin(), iterator2 = v.begin(); + This code snippet causes the new values in + <code class="literal">obj2</code> to be stored into the + underlying database. + </p> + <p> + If you have two iterators going through the same + container like this: + </p> + <pre class="programlisting">for (iterator1 = v.begin(), iterator2 = v.begin(); iterator1 != v.end(); ++iterator1, ++iterator2) { *iterator1 = new_value; print(*iterator2); } </pre> + <p> + then the printed value will depend on the value of + <span class="bold"><strong>directdb_get</strong></span> with + which the iterator had been created. If <span class="bold"><strong>directdb_get</strong></span> is + <code class="literal">false</code>, then the original, + persistent value is printed; otherwise the newly assigned + value is returned from the cache when + <code class="literal">iterator2</code> is dereferenced. This + happens because each iterator has its own cached copy of + the persistent object, and the dereferencing of + <code class="literal">iterator2</code> refreshes + <code class="literal">iterator2</code>'s copy from the database, + retrieving the value stored by the assignment to + <code class="literal">*iterator1</code>. + </p> <p> - then the printed value will depend on the value of - <span class="bold"><strong>directdb_get</strong></span> with which the - iterator had been created. If <span class="bold"><strong>directdb_get</strong></span> - is <code class="literal">false</code>, then the original, persistent value is - printed; otherwise the newly assigned value is returned from the - cache when <code class="literal">iterator2</code> is dereferenced. This - happens because each iterator has its own cached copy of the - persistent object, and the dereferencing of - <code class="literal">iterator2</code> refreshes - <code class="literal">iterator2</code>'s copy from the database, retrieving - the value stored by the assignment to - <code class="literal">*iterator1</code>. -</p> - <p> - Alternatively, you can set <span class="bold"><strong>directdb_get - </strong></span> to <code class="literal">false</code> and call - <code class="methodname">iterator2->refresh()</code> immediately before - the dereferencing of <code class="literal">iterator2</code>, so that - <code class="literal">iterator2</code>'s cached value is refreshed. -</p> - <p> - If <span class="bold"><strong>directdb_get</strong></span> is - <code class="literal">false</code>, a few of the tests in dbstl's test kit - will fail. This is because the above contrived case appears in - several of C++ STL tests. Consequently, the default value of the - <span class="bold"><strong>directdb_get</strong></span> parameter in the - <code class="methodname">container::begin()</code> methods is - <code class="literal">true</code>. If your use cases avoid such bizarre usage - of iterators, you can set it to <code class="literal">false</code>, which - makes the iterator read operation faster. -</p> + Alternatively, you can set <span class="bold"><strong>directdb_get </strong></span> to + <code class="literal">false</code> and call <code class="methodname">iterator2->refresh()</code> + immediately before the dereferencing of + <code class="literal">iterator2</code>, so that + <code class="literal">iterator2</code>'s cached value is + refreshed. + </p> + <p> + If <span class="bold"><strong>directdb_get</strong></span> is + <code class="literal">false</code>, a few of the tests in + dbstl's test kit will fail. This is because the above + contrived case appears in several of C++ STL tests. + Consequently, the default value of the <span class="bold"><strong>directdb_get</strong></span> parameter in the + <code class="methodname">container::begin()</code> methods is + <code class="literal">true</code>. If your use cases avoid such + bizarre usage of iterators, you can set it to + <code class="literal">false</code>, which makes the iterator + read operation faster. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> @@ -184,46 +193,48 @@ for (iterator1 = v.begin(), iterator2 = v.begin(); </div> </div> <p> - If you modify the object to which an iterator refers by using one - of the following: -</p> + If you modify the object to which an iterator refers by + using one of the following: + </p> <pre class="programlisting">(*iterator).member_function_call()</pre> <p> - or -</p> + or + </p> <pre class="programlisting">(*iterator).data_member = new_value</pre> <p> - then you should call - <code class="methodname">iterator->_DB_STL_StoreElement()</code> to - store the change. Otherwise the change is lost after the - iterator moves on to other elements. -</p> - <p> - If you are storing a sequence, and you modified some part of it, you - should also call - <code class="methodname">iterator->_DB_STL_StoreElement()</code> - before moving the iterator. -</p> + then you should call + <code class="methodname">iterator->_DB_STL_StoreElement()</code> + to store the change. Otherwise the change is lost after + the iterator moves on to other elements. + </p> <p> - And in both cases, if <span class="bold"><strong>directdb_get</strong></span> - is <code class="literal">true</code> (this is the default value), you should - call <code class="methodname">_DB_STL_StoreElement()</code> after the - change and before the next iterator movement OR the next - dereferencing of the iterator by the star or arrow operators - (<code class="literal">iterator::operator*</code> or - <code class="literal">iterator::operator-></code>). Otherwise, you will - lose the change. -</p> + If you are storing a sequence, and you modified some + part of it, you should also call + <code class="methodname">iterator->_DB_STL_StoreElement()</code> + before moving the iterator. + </p> <p> - If you update the element by assigning to a dereferenced iterator like - this: -</p> + And in both cases, if <span class="bold"><strong>directdb_get</strong></span> is + <code class="literal">true</code> + (this is the default value), you should call + <code class="methodname">_DB_STL_StoreElement()</code> after + the change and before the next iterator movement OR the + next dereferencing of the iterator by the star or arrow + operators (<code class="literal">iterator::operator*</code> or + <code class="literal">iterator::operator-></code>). + Otherwise, you will lose the change. + </p> + <p> + If you update the element by assigning to a + dereferenced iterator like this: + </p> <pre class="programlisting">*iterator = new_element;</pre> <p> - then you never have to call - <code class="methodname">_DB_STL_StoreElement()</code> because the change - is stored in the database automatically. -</p> + then you never have to call + <code class="methodname">_DB_STL_StoreElement()</code> + because the change is stored in the database + automatically. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> @@ -233,42 +244,48 @@ for (iterator1 = v.begin(), iterator2 = v.begin(); </div> </div> </div> + <p> + Dbstl is an interface to Berkeley DB, so it is used to + store data persistently. This is really a different + purpose from that of regular C++ STL. This difference in + their goals has implications on expected object lifetime: + In standard STL, when you store an object A of type ID + into C++ stl vector V using V.push_back(A), if a proper + copy constructor is provided in A's class type, then the + copy of A (call it B) and everything in B, such as another + object C pointed to by B's data member B.c_ptr, will be + stored in V and will live as long as B is still in V and V + is alive. B will be destroyed when V is destroyed or B is + erased from V. + </p> <p> - Dbstl is an interface to Berkeley DB, so it is used to store data - persistently. This is really a different purpose from that of - regular C++ STL. This difference in their goals has implications on - expected object lifetime: In standard STL, when you store an object - A of type ID into C++ stl vector V using V.push_back(A), if a - proper copy constructor is provided in A's class type, then the - copy of A (call it B) and everything in B, such as another object C - pointed to by B's data member B.c_ptr, will be stored in V and will - live as long as B is still in V and V is alive. B will be destroyed - when V is destroyed or B is erased from V. -</p> - <p> - This is not true for dbstl, which will copy A's data and store it - in the underlying database. The copy is by default a shallow copy, - but users can register their object marshalling and unmarshalling - functions using the <code class="classname">DbstlElemTraits</code> class - template. So if A is passed to a <code class="classname">db_vector</code> - container, <code class="literal">dv</code>, by using - <code class="literal">dv.push_back(A)</code>, then dbstl copies A's data - using the registered functions, and stores data into the underlying - database. Consequently, A will be valid, even if the container is - destroyed, because it is stored into the database. -</p> - <p> - If the copy is simply a shallow copy, and A is later destroyed, then - the pointer stored in the database will become invalid. The next time - we use the retrieved object, we will be using an invalid pointer, which - probably will result in errors. To avoid this, store the referred - object C rather than the pointer member A.c_ptr itself, by registering - the right marshalling/unmarshalling function with - <code class="classname">DbstlElemTraits</code>. -</p> + This is not true for dbstl, which will copy A's data + and store it in the underlying database. The copy is by + default a shallow copy, but users can register their + object marshalling and unmarshalling functions using the + <code class="classname">DbstlElemTraits</code> class template. + So if A is passed to a <code class="classname">db_vector</code> + container, <code class="literal">dv</code>, by using + <code class="literal">dv.push_back(A)</code>, then dbstl copies + A's data using the registered functions, and stores data + into the underlying database. Consequently, A will be + valid, even if the container is destroyed, because it is + stored into the database. + </p> + <p> + If the copy is simply a shallow copy, and A is later + destroyed, then the pointer stored in the database will + become invalid. The next time we use the retrieved object, + we will be using an invalid pointer, which probably will + result in errors. To avoid this, store the referred object + C rather than the pointer member A.c_ptr itself, by + registering the right marshalling/unmarshalling function + with <code class="classname">DbstlElemTraits</code>. + </p> <p> - For example, consider the following example class declaration: -</p> + For example, consider the following example class + declaration: + </p> <pre class="programlisting">class ID { public: @@ -276,23 +293,25 @@ public: int Score; }; </pre> <p> - Here, the class ID has a data member <span class="bold"><strong>Name</strong></span>, which refers to a memory address of - the actual characters in the string. If we simply shallow copy an - object, <code class="literal">id</code>, of class ID to store it, then the - stored data, <code class="literal">idd</code>, is invalid when - <code class="literal">id</code> is destroyed. This is because - <code class="literal">idd</code> and <code class="literal">id</code> refer to a common - memory address which is the base address of the memory space storing - all characters in the string, and this memory space is released when - <code class="literal">id</code> is destroyed. So <code class="literal">idd</code> will be - referring to an invalid address. The next time we retrieve - <code class="literal">idd</code> and use it, there will probably be memory - corruption. -</p> + Here, the class ID has a data member <span class="bold"><strong>Name</strong></span>, which refers to a memory + address of the actual characters in the string. If we + simply shallow copy an object, <code class="literal">id</code>, of + class ID to store it, then the stored data, + <code class="literal">idd</code>, is invalid when + <code class="literal">id</code> is destroyed. This is because + <code class="literal">idd</code> and <code class="literal">id</code> refer + to a common memory address which is the base address of + the memory space storing all characters in the string, and + this memory space is released when <code class="literal">id</code> + is destroyed. So <code class="literal">idd</code> will be referring + to an invalid address. The next time we retrieve + <code class="literal">idd</code> and use it, there will probably + be memory corruption. + </p> <p> - The way to store <code class="literal">id</code> is to write a marshal/unmarshal - function pair like this: -</p> + The way to store <code class="literal">id</code> is to write a + marshal/unmarshal function pair like this: + </p> <pre class="programlisting">void copy_id(void *dest, const ID&elem) { memcpy(dest, &elem.Score, sizeof(elem.Score)); @@ -313,16 +332,17 @@ size_t size_id(const ID& elem) 1;// store the '\0' char. } </pre> <p> - Then register the above functions before storing any instance of - <code class="classname">ID</code>: - </p> + Then register the above functions before storing any + instance of <code class="classname">ID</code>: + </p> <pre class="programlisting">DbstlElemTraits<ID>::instance()->set_copy_function(copy_id); DbstlElemTraits<ID>::instance()->set_size_function(size_id); DbstlElemTraits<ID>::instance()->set_restore_function(restore_id); </pre> <p> - This way, the actual data of instances of ID are stored, and so the - data will persist even if the container itself is destroyed. - </p> + This way, the actual data of instances of ID are + stored, and so the data will persist even if the container + itself is destroyed. + </p> </div> </div> <div class="navfooter"> @@ -336,11 +356,13 @@ DbstlElemTraits<ID>::instance()->set_restore_function(restore_id); </p <td width="40%" align="right"> <a accesskey="n" href="stl_container_specific.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Store and Retrieve data or objects of complex types </td> + <td width="40%" align="left" valign="top">Store and Retrieve data or + objects of complex types </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Dbstl container specific notes</td> + <td width="40%" align="right" valign="top"> Dbstl container specific + notes</td> </tr> </table> </div> diff --git a/docs/programmer_reference/stl_primitive_rw.html b/docs/programmer_reference/stl_primitive_rw.html index a17d1d3f..be44c1be 100644 --- a/docs/programmer_reference/stl_primitive_rw.html +++ b/docs/programmer_reference/stl_primitive_rw.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -40,126 +40,146 @@ <dl> <dt> <span class="sect2"> - <a href="stl_primitive_rw.html#idp1288424">Storing strings</a> + <a href="stl_primitive_rw.html#idp849056">Storing strings</a> </span> </dt> </dl> </div> - <p> - To store simple primitive types such as <code class="literal">int</code>, - <code class="literal">long</code>, <code class="literal">double</code>, and so forth, an - additional type parameter for the container class - templates is needed. For example, to store an <code class="literal">int</code> - in a <code class="classname">db_vector</code>, use this container class: -</p> + <p> + To store simple primitive types such as + <code class="literal">int</code>, <code class="literal">long</code>, + <code class="literal">double</code>, and so forth, an additional + type parameter for the container class templates is needed. + For example, to store an <code class="literal">int</code> in a + <code class="classname">db_vector</code>, use this container + class: + </p> <pre class="programlisting">db_vector<int, ElementHolder<int> >;</pre> - <p> - To map integers to doubles, use this: -</p> + <p> + To map integers to doubles, use this: + </p> <pre class="programlisting">db_map<int, double, ElementHolder<double> >;</pre> - <p> - To store a <code class="literal">char*</code> string with <code class="literal">long</code> keys, - use this: -</p> + <p> + To store a <code class="literal">char*</code> string with + <code class="literal">long</code> keys, use this: + </p> <pre class="programlisting">db_map<long, char*, ElementHolder<char*> >;</pre> - <p> - Use this for <code class="literal">const char*</code> strings: -</p> + <p> + Use this for <code class="literal">const char*</code> strings: + </p> <pre class="programlisting">db_map<long, const char*, ElementHolder<const char*> >;</pre> - <p> - To map one const string to another, use this type: -</p> + <p> + To map one const string to another, use this type: + </p> <pre class="programlisting">db_map<const char*, const char*, ElementHolder<const char*> >;</pre> - <p> - The <code class="methodname">StlAdvancedFeaturesExample::primitive()</code> method demonstrates more of these examples. -</p> + <p> + The + <code class="methodname">StlAdvancedFeaturesExample::primitive()</code> + method demonstrates more of these examples. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp1288424"></a>Storing strings</h3> + <h3 class="title"><a id="idp849056"></a>Storing strings</h3> </div> </div> </div> <p> - For <code class="literal">char*</code> and <code class="literal">wchar_t*</code> strings, - <code class="methodname">_DB_STL_StoreElement()</code> must be called - following partial or total modifications before iterator movement, - <code class="literal">container::operator[]</code> or - <code class="literal">iterator::operator*/-></code> calls. Without the - <code class="methodname">_DB_STL_StoreElement()</code> call, the modified - change will be lost. If storing an new value like this: -</p> + For <code class="literal">char*</code> and + <code class="literal">wchar_t*</code> strings, + <code class="methodname">_DB_STL_StoreElement()</code> must + be called following partial or total modifications before + iterator movement, + <code class="literal">container::operator[]</code> or + <code class="literal">iterator::operator*/-></code> calls. + Without the + <code class="methodname">_DB_STL_StoreElement()</code> call, + the modified change will be lost. If storing an new value + like this: + </p> <pre class="programlisting">*iterator = new_char_star_string;</pre> <p> - the call to <code class="methodname">_DB_STL_StoreElement()</code> is not needed. -</p> - <p> - Note that passing a NULL pointer to a container of - <code class="literal">char*</code> type or passing a - <code class="classname">std::string</code> with no contents at all will insert - an empty string of zero length into the database. -</p> - <p> - The string returned from a container will not live beyond the next - iterator movement call, <code class="literal">container::operator[]</code> or - <code class="literal">iterator::operator*/-></code> call. -</p> + the call to + <code class="methodname">_DB_STL_StoreElement()</code> is not + needed. + </p> + <p> + Note that passing a NULL pointer to a container of + <code class="literal">char*</code> type or passing a + <code class="classname">std::string</code> with no contents at + all will insert an empty string of zero length into the + database. + </p> + <p> + The string returned from a container will not live + beyond the next iterator movement call, + <code class="literal">container::operator[]</code> or + <code class="literal">iterator::operator*/-></code> call. + </p> <p> - - A <span class="bold"><strong>db_map::value_type::second_type</strong></span> or - <span class="bold"><strong>db_map::datatype_wrap</strong></span> should be used - to hold a reference to a <code class="literal">container::operator[]</code> - return value. Then the reference should be used for repeated - references to that value. The *iterator is of type - <code class="literal">ElementHolder<char *></code>, which can be automatically converted to a - <code class="literal">char *</code> pointer using its type conversion operator. - Wherever an auto conversion is done by the compiler, the conversion - operator of <code class="literal">ElementHolder<T></code> is called. This - avoids almost all explicit conversions, except for two use cases: - -</p> + A <span class="bold"><strong>db_map::value_type::second_type</strong></span> or + <span class="bold"><strong>db_map::datatype_wrap</strong></span> + should be used to hold a reference to a + <code class="literal">container::operator[]</code> return value. + Then the reference should be used for repeated references + to that value. The *iterator is of type + <code class="literal">ElementHolder<char *></code>, which + can be automatically converted to a <code class="literal">char*</code> pointer + using its type conversion + operator. Wherever an auto conversion is done by the + compiler, the conversion operator of + <code class="literal">ElementHolder<T></code> is called. + This avoids almost all explicit conversions, except for + two use cases: + </p> <div class="orderedlist"> <ol type="1"> <li> - <p> - The *iterator is used as a "..." parameter like this: - </p> + <p> + The *iterator is used as a "..." parameter like + this: + </p> <pre class="programlisting">printf("this is the special case %s", *iterator);</pre> - <p> - This compiles but causes errors. Instead, an explicit cast - should be used: - </p> + <p> + This compiles but causes errors. Instead, an + explicit cast should be used: + </p> <pre class="programlisting">printf("this is the special case %s", (char *)*iterator);</pre> </li> <li> <p> - For some old compilers, such as gcc3.4.6, the *iterator cannot be - used with the ternary <code class="literal">?</code> operator, like - this: - </p> + For some old compilers, such as gcc3.4.6, the + *iterator cannot be used with the ternary + <code class="literal">?</code> operator, like this: + </p> <pre class="programlisting">expr ? *iterator : var</pre> - <p> - Even when <span class="bold"><strong>var</strong></span> is the same - type as the iterator's <code class="literal">value_type</code>, the - compiler fails to perform an auto conversion. - </p> + <p> + Even when <span class="bold"><strong>var</strong></span> + is the same type as the iterator's + <code class="literal">value_type</code>, the compiler + fails to perform an auto conversion. + </p> </li> </ol> </div> <p> - When using <code class="classname">std::string</code> or - <code class="classname">std::wstring</code> as the data type for dbstl - containers — that is, <code class="classname">db_vector<string></code>, - and <code class="classname">db_map<string, wstring></code> — the - string's content rather than the string object itself is stored in order - to maintain persistence. -</p> - <p> - You can find example code demonstrating string storage in the - <code class="methodname">StlAdvancedFeaturesExample::char_star_string_storage()</code> and - <code class="methodname">StlAdvancedFeaturesExample::storing_std_strings()</code> methods. -</p> + When using <code class="classname">std::string</code> or + <code class="classname">std::wstring</code> as the data type + for dbstl containers — that is, + <code class="classname">db_vector<string></code>, and + <code class="classname">db_map<string, wstring></code> + — the string's content rather than the string object + itself is stored in order to maintain persistence. + </p> + <p> + You can find example code demonstrating string storage + in the + <code class="methodname">StlAdvancedFeaturesExample::char_star_string_storage()</code> + and + <code class="methodname">StlAdvancedFeaturesExample::storing_std_strings()</code> + methods. + </p> </div> </div> <div class="navfooter"> @@ -173,11 +193,13 @@ <td width="40%" align="right"> <a accesskey="n" href="stl_complex_rw.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Using dbstl in multithreaded applications </td> + <td width="40%" align="left" valign="top">Using dbstl in multithreaded + applications </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Store and Retrieve data or objects of complex types </td> + <td width="40%" align="right" valign="top"> Store and Retrieve data or + objects of complex types </td> </tr> </table> </div> diff --git a/docs/programmer_reference/stl_txn_usage.html b/docs/programmer_reference/stl_txn_usage.html index 117c20a8..712f4588 100644 --- a/docs/programmer_reference/stl_txn_usage.html +++ b/docs/programmer_reference/stl_txn_usage.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -36,54 +36,60 @@ </div> </div> </div> + <p> + When using transactions with dbstl, you must call the dbstl + transaction functions instead of the corresponding methods + from the Berkeley DB C or C++ transaction API. That is, you + must use <code class="methodname">dbstl::begin_txn()</code>, + <code class="methodname">dbstl::commit_txn()</code> and + <code class="methodname">dbstl::abort_txn()</code> in order to + begin/commit/abort transactions. + </p> + <p> + A container can be configured to use auto commit by setting + the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_AUTO_COMMIT" class="olink">DB_AUTO_COMMIT</a> flag when the environment or database + handle is opened. In this case, any container method that + supports auto commit will automatically form an independent + transaction if the method is not in an external transactional + context; Otherwise, the operation will become part of that + transaction. + </p> <p> - When using transactions with dbstl, you must call the dbstl transaction - functions instead of the corresponding methods from the Berkeley DB C - or C++ transaction API. That is, you must use - <code class="methodname">dbstl::begin_txn()</code>, - <code class="methodname">dbstl::commit_txn()</code> and - <code class="methodname">dbstl::abort_txn()</code> in order to - begin/commit/abort transactions. -</p> + You can configure the flags used internally by dbstl when + it is creating and committing these independent transactions + required by auto commit. To do so, use the + <code class="methodname">db_container::set_txn_begin_flags()</code> + and/or + <code class="methodname">db_container::set_commit_flags()</code> + methods. + </p> + <p> + When a transaction is committed or aborted, dbstl will + automatically close any cursors opened for use by the + transaction. For this reason, any iterators opened within the + transaction context should not be used after the transaction + commits or aborts. + </p> + <p> + You can use nested transactions explicitly and externally, + by calling <code class="methodname">dbstl::begin_txn()</code> in a + context already operating under the protection of a + transaction. But you can not designate which transaction is + the parent transaction. The parent transaction is + automatically the most recently created and unresolved + transaction in current thread. + </p> <p> - A container can be configured to use auto commit by setting the - <a href="../api_reference/C/envset_flags.html#envset_flags_DB_AUTO_COMMIT" class="olink">DB_AUTO_COMMIT</a> flag when the environment or database handle is - opened. In this case, any container method that supports auto commit - will automatically form an independent transaction if the method - is not in an external transactional context; Otherwise, - the operation will become part of that transaction. - -</p> + It is also acceptable to use explicit transactions in a + container configured for auto commit. The operation performed + by the method will become part of the provided external + transaction. + </p> <p> - You can configure the flags used internally by dbstl when it is - creating and committing these independent transactions required by auto - commit. To do so, use the - <code class="methodname">db_container::set_txn_begin_flags()</code> and/or - <code class="methodname">db_container::set_commit_flags()</code> methods. -</p> - <p> - When a transaction is committed or aborted, dbstl will automatically - close any cursors opened for use by the transaction. For this reason, - any iterators opened within the transaction context should not be used - after the transaction commits or aborts. -</p> - <p> - You can use nested transactions explicitly and externally, by calling - <code class="methodname">dbstl::begin_txn()</code> in a context already - operating under the protection of a transaction. But you can not - designate which transaction is the parent transaction. The parent - transaction is automatically the most recently created and unresolved - transaction in current thread. -</p> - <p> - It is also acceptable to use explicit transactions in a container - configured for auto commit. The operation performed by the method will - become part of the provided external transaction. -</p> - <p> - Finally, transactions and iterators cannot be shared among multiple - threads. That is, they are not free-threaded, or thread-safe. -</p> + Finally, transactions and iterators cannot be shared among + multiple threads. That is, they are not free-threaded, or + thread-safe. + </p> </div> <div class="navfooter"> <hr /> @@ -96,11 +102,13 @@ <td width="40%" align="right"> <a accesskey="n" href="stl_mt_usage.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Using advanced Berkeley DB features with dbstl </td> + <td width="40%" align="left" valign="top">Using advanced Berkeley DB + features with dbstl </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Using dbstl in multithreaded applications</td> + <td width="40%" align="right" valign="top"> Using dbstl in multithreaded + applications</td> </tr> </table> </div> diff --git a/docs/programmer_reference/stl_usecase.html b/docs/programmer_reference/stl_usecase.html index 70d3ff0f..e52291eb 100644 --- a/docs/programmer_reference/stl_usecase.html +++ b/docs/programmer_reference/stl_usecase.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -28,33 +28,65 @@ </table> <hr /> </div> - <div class="sect1" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="stl_usecase"></a>Dbstl typical use cases</h2></div></div></div> -Among others, the following are some typical use cases where dbstl would -be prefered over C++ STL: - -<div class="itemizedlist"><ul type="disc"><li><p> -Working with a large amount of data, more than can reside in memory. -Using C++ STL would force a number of page swaps, which will degrade -performance. When using dbstl, data is stored in a database and Berkeley -DB ensures the needed data is in memory, so that the overall performance -of the machine is not slowed down. -</p></li><li><p> -Familiar Interface. dbstl provides a familiar interface to Berkeley DB, -hiding the marshalling and unmashalling details and automatically -managing Berkeley DB structures and objects. -</p></li><li><p> -Transaction semantics. dbstl provides the ACID properties (or a subset of -the ACID properties) in addition to supporting all of the STL -functionality. -</p></li><li><p> -Concurrent access. Few (if any) existing C++ STL implementations support -reading/writing to the same container concurrently, dbstl does. -</p></li><li><p> -Object persistence. dbstl allows your application to store objects in a -database, and use the objects across different runs of your application. -dbstl is capable of storing complicated objects which are not located in -a contiguous chunk of memory, with some user configurations. -</p></li></ul></div></div> + <div class="sect1" lang="en" xml:lang="en"> + <div class="titlepage"> + <div> + <div> + <h2 class="title" style="clear: both"><a id="stl_usecase"></a>Dbstl typical use cases</h2> + </div> + </div> + </div> + <p> + Among others, the following are some typical use cases where dbstl + would be prefered over C++ STL: + </p> + <div class="itemizedlist"> + <ul type="disc"> + <li> + <p> + Working with a large amount of data, more than can + reside in memory. Using C++ STL would force a number + of page swaps, which will degrade performance. When + using dbstl, data is stored in a database and Berkeley + DB ensures the needed data is in memory, so that the + overall performance of the machine is not slowed down. + </p> + </li> + <li> + <p> + Familiar Interface. dbstl provides a familiar + interface to Berkeley DB, hiding the marshalling and + unmashalling details and automatically managing + Berkeley DB structures and objects. + </p> + </li> + <li> + <p> + Transaction semantics. dbstl provides the ACID + properties (or a subset of the ACID properties) in + addition to supporting all of the STL functionality. + </p> + </li> + <li> + <p> + Concurrent access. Few (if any) existing C++ STL + implementations support reading/writing to the same + container concurrently, dbstl does. + </p> + </li> + <li> + <p> + Object persistence. dbstl allows your application + to store objects in a database, and use the objects + across different runs of your application. dbstl is + capable of storing complicated objects which are not + located in a contiguous chunk of memory, with some + user configurations. + </p> + </li> + </ul> + </div> + </div> <div class="navfooter"> <hr /> <table width="100%" summary="Navigation footer"> diff --git a/docs/programmer_reference/tcl.html b/docs/programmer_reference/tcl.html index 906cbe42..69c8cca7 100644 --- a/docs/programmer_reference/tcl.html +++ b/docs/programmer_reference/tcl.html @@ -14,13 +14,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 21. - Berkeley DB Extensions: Tcl - </th> + <th colspan="3" align="center">Chapter 21. Berkeley DB Extensions: Tcl </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="sequence.html">Prev</a> </td> @@ -34,9 +32,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="tcl"></a>Chapter 21. - Berkeley DB Extensions: Tcl - </h2> + <h2 class="title"><a id="tcl"></a>Chapter 21. Berkeley DB Extensions: Tcl </h2> </div> </div> </div> @@ -54,12 +50,12 @@ <dl> <dt> <span class="sect2"> - <a href="tcl.html#idp3187200">Installing as a Tcl Package</a> + <a href="tcl.html#idp2951744">Installing as a Tcl Package</a> </span> </dt> <dt> <span class="sect2"> - <a href="tcl.html#idp3177840">Loading Berkeley DB with Tcl</a> + <a href="tcl.html#idp2941624">Loading Berkeley DB with Tcl</a> </span> </dt> </dl> @@ -98,77 +94,113 @@ <dl> <dt> <span class="sect2"> - <a href="tcl.html#idp3187200">Installing as a Tcl Package</a> + <a href="tcl.html#idp2951744">Installing as a Tcl Package</a> </span> </dt> <dt> <span class="sect2"> - <a href="tcl.html#idp3177840">Loading Berkeley DB with Tcl</a> + <a href="tcl.html#idp2941624">Loading Berkeley DB with Tcl</a> </span> </dt> </dl> </div> - <p>Berkeley DB includes a dynamically loadable Tcl API, which requires that -Tcl/Tk 8.5 or later already be installed on your system. You can -download a copy of Tcl from the <a class="ulink" href="http://www.tcl.tk" target="_top">Tcl -Developer Xchange</a> Web site.</p> - <p>This document assumes that you already configured Berkeley DB for Tcl -support, and you have built and installed everything where you want it -to be. If you have not done so, see -<a href="../installation/build_unix_conf.html" class="olink">Configuring Berkeley DB</a> or <a href="../installation/build_win_tcl.html" class="olink">Building the Tcl API</a> in the Berkeley DB Installation and Build Guide for more -information.</p> + <p> + Berkeley DB includes a dynamically loadable Tcl API, which + requires that Tcl/Tk 8.5 or later already be installed on your + system. You can download a copy of Tcl from the <a class="ulink" href="http://www.tcl.tk" target="_top">Tcl Developer Xchange</a> Web + site. + </p> + <p> + This document assumes that you already configured Berkeley + DB for Tcl support, and you have built and installed + everything where you want it to be. If you have not done so, + see <a href="../installation/build_unix_conf.html" class="olink">Configuring Berkeley DB</a> or <a href="../installation/build_win_tcl.html" class="olink">Building the Tcl API</a> in the + Berkeley DB Installation and Build Guide for more information. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp3187200"></a>Installing as a Tcl Package</h3> + <h3 class="title"><a id="idp2951744"></a>Installing as a Tcl Package</h3> </div> </div> </div> - <p>Once enabled, the Berkeley DB shared library for Tcl is automatically installed -as part of the standard installation process. However, if you want to be -able to dynamically load it as a Tcl package into your script, there are -several steps that must be performed:</p> + <p> + Once enabled, the Berkeley DB shared library for Tcl is + automatically installed as part of the standard + installation process. However, if you want to be able to + dynamically load it as a Tcl package into your script, + there are several steps that must be performed: + </p> <div class="orderedlist"> <ol type="1"> - <li>Run the Tcl shell in the install directory.</li> - <li>Append this directory to your auto_path variable.</li> - <li>Run the pkg_mkIndex proc, giving the name of the Berkeley DB Tcl library.</li> + <li> + Run the Tcl shell in the install + directory. + </li> + <li> + Append this directory to your auto_path + variable. + </li> + <li> + Run the pkg_mkIndex proc, giving the name of the + Berkeley DB Tcl library. + </li> </ol> </div> - <p>For example:</p> + <p> + For example: + </p> <pre class="programlisting"># tclsh8.5 -% lappend auto_path /usr/local/BerkeleyDB.5.2/lib -% pkg_mkIndex /usr/local/BerkeleyDB.5.2/lib libdb_tcl-5.2.so</pre> - <p>Note that your Tcl and Berkeley DB version numbers may differ from the -example, and so your tclsh and library names may be different.</p> +% lappend auto_path /usr/local/BerkeleyDB.6.0/lib +% pkg_mkIndex /usr/local/BerkeleyDB.6.0/lib libdb_tcl-6.0.so</pre> + <p> + Note that your Tcl and Berkeley DB version numbers may + differ from the example, and so your tclsh and library + names may be different. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp3177840"></a>Loading Berkeley DB with Tcl</h3> + <h3 class="title"><a id="idp2941624"></a>Loading Berkeley DB with Tcl</h3> </div> </div> </div> - <p>The Berkeley DB package may be loaded into the user's interactive Tcl script -(or wish session) via the <span class="bold"><strong>load</strong></span> command. For example:</p> - <pre class="programlisting">load /usr/local/BerkeleyDB.5.2/lib/libdb_tcl-5.2.so</pre> - <p>Note that your Berkeley DB version numbers may differ from the example, and so -the library name may be different.</p> - <p>If you installed your library to run as a Tcl package, Tcl application -scripts should use the <span class="bold"><strong>package</strong></span> command to indicate to the Tcl -interpreter that it needs the Berkeley DB package and where to find it. For -example:</p> - <pre class="programlisting">lappend auto_path "/usr/local/BerkeleyDB.5.2/lib" + <p> + The Berkeley DB package may be loaded into the user's + interactive Tcl script (or wish session) via the <span class="bold"><strong>load</strong></span> command. For + example: + </p> + <pre class="programlisting">load /usr/local/BerkeleyDB.6.0/lib/libdb_tcl-6.0.so</pre> + <p> + Note that your Berkeley DB version numbers may differ + from the example, and so the library name may be + different. + </p> + <p> + If you installed your library to run as a Tcl package, + Tcl application scripts should use the <span class="bold"><strong>package</strong></span> command to indicate to + the Tcl interpreter that it needs the Berkeley DB package + and where to find it. For example: + </p> + <pre class="programlisting">lappend auto_path "/usr/local/BerkeleyDB.6.0/lib" package require Db_tcl</pre> - <p>No matter which way the library gets loaded, it creates a command named -<span class="bold"><strong>berkdb</strong></span>. All the Berkeley DB functionality is accessed via this -command and additional commands it creates on behalf of the application. -A simple test to determine whether everything is loaded and ready is to -display the library version, as follows:</p> + <p> + No matter which way the library gets loaded, it creates + a command named <span class="bold"><strong>berkdb</strong></span>. + All the Berkeley DB functionality is accessed via this + command and additional commands it creates on behalf of + the application. A simple test to determine whether + everything is loaded and ready is to display the library + version, as follows: + </p> <pre class="programlisting">berkdb version -string</pre> - <p>This should return you the Berkeley DB version in a string format.</p> + <p> + This should return you the Berkeley DB version in a + string format. + </p> </div> </div> </div> @@ -181,9 +213,7 @@ display the library version, as follows:</p> <td width="40%" align="right"> <a accesskey="n" href="tcl_using.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 20. - Sequences - </td> + <td width="40%" align="left" valign="top">Chapter 20. Sequences </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/tcl_error.html b/docs/programmer_reference/tcl_error.html index 4b328a96..dd1ce597 100644 --- a/docs/programmer_reference/tcl_error.html +++ b/docs/programmer_reference/tcl_error.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="tcl_program.html">Prev</a> </td> - <th width="60%" align="center">Chapter 21. - Berkeley DB Extensions: Tcl - </th> + <th width="60%" align="center">Chapter 21. Berkeley DB Extensions: Tcl </th> <td width="20%" align="right"> <a accesskey="n" href="tcl_faq.html">Next</a></td> </tr> </table> @@ -38,18 +36,23 @@ </div> </div> </div> - <p>The Tcl interfaces to Berkeley DB generally return TCL_OK on success and throw -a Tcl error on failure, using the appropriate Tcl interfaces to provide -the user with an informative error message. There are some "expected" -failures, however, for which no Tcl error will be thrown and for which -Tcl commands will return TCL_OK. These failures include times when a -searched-for key is not found, a requested key/data pair was previously -deleted, or a key/data pair cannot be written because the key already -exists.</p> - <p>These failures can be detected by searching the Berkeley DB error message that -is returned. For example, use the following to detect that an attempt -to put a record into the database failed because the key already -existed:</p> + <p> + The Tcl interfaces to Berkeley DB generally return TCL_OK on + success and throw a Tcl error on failure, using the + appropriate Tcl interfaces to provide the user with an + informative error message. There are some "expected" failures, + however, for which no Tcl error will be thrown and for which + Tcl commands will return TCL_OK. These failures include times + when a searched-for key is not found, a requested key/data + pair was previously deleted, or a key/data pair cannot be + written because the key already exists. + </p> + <p> + These failures can be detected by searching the Berkeley DB + error message that is returned. For example, use the following + to detect that an attempt to put a record into the database + failed because the key already existed: + </p> <pre class="programlisting">% berkdb open -create -btree a.db db0 % db0 put dog cat @@ -63,30 +66,46 @@ This was an error; the key existed % db0 close 0 % exit</pre> - <p>To simplify parsing, it is recommended that the initial Berkeley DB error name -be checked; for example, <a href="../api_reference/C/dbcput.html#dbcput_DB_KEYEXIST" class="olink">DB_MULTIPLE</a> in the previous example. -To ensure that Tcl scripts are not broken by upgrading to new releases -of Berkeley DB, these values will not change in future releases of Berkeley DB. -There are currently only three such "expected" error returns:</p> + <p> + To simplify parsing, it is recommended that the initial + Berkeley DB error name be checked; for example, <a href="../api_reference/C/dbcput.html#dbcput_DB_KEYEXIST" class="olink">DB_MULTIPLE</a> + in the previous example. To ensure that Tcl scripts are not + broken by upgrading to new releases of Berkeley DB, these + values will not change in future releases of Berkeley DB. + There are currently only three such "expected" error + returns: + </p> <pre class="programlisting">DB_NOTFOUND: No matching key/data pair found DB_KEYEMPTY: Nonexistent key/data pair DB_KEYEXIST: Key/data pair already exists</pre> - <p>Finally, sometimes Berkeley DB will output additional error information when -a Berkeley DB error occurs. By default, all Berkeley DB error messages will be -prefixed with the created command in whose context the error occurred -(for example, "env0", "db2", and so on). There are several ways to -capture and access this information.</p> - <p>First, if Berkeley DB invokes the error callback function, the additional -information will be placed in the error result returned from the command -and in the errorInfo backtrace variable in Tcl.</p> - <p>Also, the two calls to open an environment and open a database take an -option, <span class="bold"><strong>-errfile filename</strong></span>, which sets an output file to which -these additional error messages should be written.</p> - <p>Additionally, the two calls to open an environment and open a database -take an option, <span class="bold"><strong>-errpfx string</strong></span>, which sets the error prefix to -the given string. This option may be useful in circumstances where a -more descriptive prefix is desired or where a constant prefix indicating -an error is desired.</p> + <p> + Finally, sometimes Berkeley DB will output additional error + information when a Berkeley DB error occurs. By default, all + Berkeley DB error messages will be prefixed with the created + command in whose context the error occurred (for example, + "env0", "db2", and so on). There are several ways to capture + and access this information. + </p> + <p> + First, if Berkeley DB invokes the error callback function, + the additional information will be placed in the error result + returned from the command and in the errorInfo backtrace + variable in Tcl. + </p> + <p> + Also, the two calls to open an environment and open a + database take an option, <span class="bold"><strong>-errfile + filename</strong></span>, which sets an output file to which + these additional error messages should be written. + </p> + <p> + Additionally, the two calls to open an environment and open + a database take an option, <span class="bold"><strong>-errpfx + string</strong></span>, which sets the error prefix to the + given string. This option may be useful in circumstances where + a more descriptive prefix is desired or where a constant + prefix indicating an error is desired. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/tcl_faq.html b/docs/programmer_reference/tcl_faq.html index cf3355ff..40f06c3c 100644 --- a/docs/programmer_reference/tcl_faq.html +++ b/docs/programmer_reference/tcl_faq.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="tcl_error.html">Prev</a> </td> - <th width="60%" align="center">Chapter 21. - Berkeley DB Extensions: Tcl - </th> + <th width="60%" align="center">Chapter 21. Berkeley DB Extensions: Tcl </th> <td width="20%" align="right"> <a accesskey="n" href="ext.html">Next</a></td> </tr> </table> @@ -42,50 +40,71 @@ <ol type="1"> <li> <span class="bold"> - <strong>I have several versions of Tcl installed. How do I configure -Berkeley DB to use a particular version?</strong> + <strong>I have several versions of Tcl + installed. How do I configure Berkeley DB to use a + particular version?</strong> </span> <p> - To compile the Tcl interface with a particular version of Tcl, use the - --with-tcl option to specify the Tcl installation directory that - contains the tclConfig.sh file. See - the <a href="../installation/build_unix_flags.html" class="olink">Changing compile or load options</a> section in the Berkeley DB Installation and Build Guide for more information. -</p> + To compile the Tcl interface with a particular + version of Tcl, use the --with-tcl option to specify + the Tcl installation directory that contains the + tclConfig.sh file. See the <a href="../installation/build_unix_flags.html" class="olink">Changing compile or load options</a> section + in the Berkeley DB Installation and Build Guide for more information. + </p> </li> <li> <span class="bold"> - <strong>Berkeley DB was configured using --enable-tcl or --with-tcl and fails -to build.</strong> + <strong>Berkeley DB was configured using + --enable-tcl or --with-tcl and fails to + build.</strong> </span> - <p>The Berkeley DB Tcl interface requires Tcl version 8.5 or greater.</p> + <p> + The Berkeley DB Tcl interface requires Tcl version + 8.5 or greater. + </p> </li> <li> <span class="bold"> - <strong>Berkeley DB was configured using --enable-tcl or --with-tcl and fails -to build.</strong> + <strong>Berkeley DB was configured using + --enable-tcl or --with-tcl and fails to + build.</strong> </span> - <p>If the Tcl installation was moved after it was configured and installed, -try reconfiguring and reinstalling Tcl.</p> - <p>Also, some systems do not search for shared libraries by default, or do -not search for shared libraries named the way the Tcl installation names -them, or are searching for a different kind of library than those in -your Tcl installation. For example, Linux systems often require linking -"libtcl.a" to "libtcl#.#.a", whereas AIX systems often require adding the -"-brtl" flag to the linker. A simpler solution that almost always works -on all systems is to create a link from "libtcl.#.#.a" or "libtcl.so" -(or whatever you happen to have) to "libtcl.a" and reconfigure.</p> + <p> + If the Tcl installation was moved after it was + configured and installed, try reconfiguring and + reinstalling Tcl. + </p> + <p> + Also, some systems do not search for shared + libraries by default, or do not search for shared + libraries named the way the Tcl installation names + them, or are searching for a different kind of library + than those in your Tcl installation. For example, + Linux systems often require linking "libtcl.a" to + "libtcl#.#.a", whereas AIX systems often require + adding the "-brtl" flag to the linker. A simpler + solution that almost always works on all systems is to + create a link from "libtcl.#.#.a" or "libtcl.so" (or + whatever you happen to have) to "libtcl.a" and + reconfigure. + </p> </li> <li> <span class="bold"> - <strong>Loading the Berkeley DB library into Tcl on AIX causes a core dump.</strong> + <strong>Loading the Berkeley DB library into + Tcl on AIX causes a core dump.</strong> </span> - <p>In some versions of Tcl, the "tclConfig.sh" autoconfiguration script -created by the Tcl installation does not work properly under AIX, and -you may have to modify values in the tclConfig.sh file to in order to -load the Berkeley DB library into Tcl. Specifically, the TCL_LIB_SPEC -variable should contain sufficient linker flags to find and link against -the installed libtcl library. In some circumstances, the tclConfig.sh -file built by Tcl does not.</p> + <p> + In some versions of Tcl, the "tclConfig.sh" + autoconfiguration script created by the Tcl + installation does not work properly under AIX, and you + may have to modify values in the tclConfig.sh file to + in order to load the Berkeley DB library into Tcl. + Specifically, the TCL_LIB_SPEC variable should contain + sufficient linker flags to find and link against the + installed libtcl library. In some circumstances, the + tclConfig.sh file built by Tcl does not. + </p> </li> </ol> </div> @@ -105,9 +124,7 @@ file built by Tcl does not.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 22. - Berkeley DB Extensions - </td> + <td width="40%" align="right" valign="top"> Chapter 22. Berkeley DB Extensions </td> </tr> </table> </div> diff --git a/docs/programmer_reference/tcl_program.html b/docs/programmer_reference/tcl_program.html index c3c636e8..e34c64a0 100644 --- a/docs/programmer_reference/tcl_program.html +++ b/docs/programmer_reference/tcl_program.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="tcl_using.html">Prev</a> </td> - <th width="60%" align="center">Chapter 21. - Berkeley DB Extensions: Tcl - </th> + <th width="60%" align="center">Chapter 21. Berkeley DB Extensions: Tcl </th> <td width="20%" align="right"> <a accesskey="n" href="tcl_error.html">Next</a></td> </tr> </table> @@ -38,25 +36,36 @@ </div> </div> </div> - <p>The Berkeley DB Tcl API does not attempt to avoid evaluating input as Tcl -commands. For this reason, it may be dangerous to pass unreviewed user -input through the Berkeley DB Tcl API, as the input may subsequently be -evaluated as a Tcl command. Additionally, the Berkeley DB Tcl API -initialization routine resets process' effective user and group IDs to -the real user and group IDs, to minimize the effectiveness of a Tcl -injection attack.</p> - <p>The Tcl API closely parallels the Berkeley DB programmatic interfaces. If you -are already familiar with one of those interfaces, there will not be many -surprises in the Tcl API.</p> - <p>The Tcl API currently does not support multithreading although -it could be made to do so. The Tcl shell itself is -not multithreaded and the Berkeley DB extensions use global data unprotected -from multiple threads.</p> - <p>Several pieces of Berkeley DB functionality are not available in the Tcl API. -Any of the functions that require a user-provided function are not -supported via the Tcl API. For example, there is no equivalent to the -<a href="../api_reference/C/dbset_dup_compare.html" class="olink">DB->set_dup_compare()</a> or <a href="../api_reference/C/envset_errcall.html" class="olink">DB_ENV->set_errcall()</a> methods. -Additionally, the heap access method is not available.</p> + <p> + The Berkeley DB Tcl API does not attempt to avoid evaluating + input as Tcl commands. For this reason, it may be dangerous to + pass unreviewed user input through the Berkeley DB Tcl API, as + the input may subsequently be evaluated as a Tcl command. + Additionally, the Berkeley DB Tcl API initialization routine + resets process' effective user and group IDs to the real user + and group IDs, to minimize the effectiveness of a Tcl + injection attack. + </p> + <p> + The Tcl API closely parallels the Berkeley DB programmatic + interfaces. If you are already familiar with one of those + interfaces, there will not be many surprises in the Tcl + API. + </p> + <p> + The Tcl API currently does not support multithreading + although it could be made to do so. The Tcl shell itself is + not multithreaded and the Berkeley DB extensions use global + data unprotected from multiple threads. + </p> + <p> + Several pieces of Berkeley DB functionality are not + available in the Tcl API. Any of the functions that require a + user-provided function are not supported via the Tcl API. For + example, there is no equivalent to the <a href="../api_reference/C/dbset_dup_compare.html" class="olink">DB->set_dup_compare()</a> or + <a href="../api_reference/C/envset_errcall.html" class="olink">DB_ENV->set_errcall()</a> methods. Additionally, the heap access method + is not available. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/tcl_using.html b/docs/programmer_reference/tcl_using.html index a668f7d9..4940975f 100644 --- a/docs/programmer_reference/tcl_using.html +++ b/docs/programmer_reference/tcl_using.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="tcl.html">Prev</a> </td> - <th width="60%" align="center">Chapter 21. - Berkeley DB Extensions: Tcl - </th> + <th width="60%" align="center">Chapter 21. Berkeley DB Extensions: Tcl </th> <td width="20%" align="right"> <a accesskey="n" href="tcl_program.html">Next</a></td> </tr> </table> @@ -38,22 +36,33 @@ </div> </div> </div> - <p>All commands in the Berkeley DB Tcl interface are in the following form:</p> + <p> + All commands in the Berkeley DB Tcl interface are in the + following form: + </p> <pre class="programlisting">command_handle operation options</pre> - <p>The <span class="emphasis"><em>command handle</em></span> is <span class="bold"><strong>berkdb</strong></span> or one of the additional -commands that may be created. The <span class="emphasis"><em>operation</em></span> is what you want -to do to that handle, and the <span class="emphasis"><em>options</em></span> apply to the operation. -Commands that get created on behalf of the application have their own sets -of operations. Generally, any calls in DB that result in new object -handles will translate into a new command handle in Tcl. Then, the user -can access the operations of the handle via the new Tcl command handle.</p> - <p>Newly created commands are named with an abbreviated form of their -objects, followed by a number. Some created commands are subcommands of -other created commands and will be the first command, followed by a -period (.), and then followed by the new subcommand. For example, -suppose that you have a database already existing called my_data.db. -The following example shows the commands created when you open the -database and when you open a cursor:</p> + <p> + The <span class="emphasis"><em>command handle</em></span> is <span class="bold"><strong>berkdb</strong></span> or one of the additional + commands that may be created. The + <span class="emphasis"><em>operation</em></span> is what you want to do to + that handle, and the <span class="emphasis"><em>options</em></span> apply to the + operation. Commands that get created on behalf of the + application have their own sets of operations. Generally, any + calls in DB that result in new object handles will translate + into a new command handle in Tcl. Then, the user can access + the operations of the handle via the new Tcl command + handle. + </p> + <p> + Newly created commands are named with an abbreviated form of + their objects, followed by a number. Some created commands are + subcommands of other created commands and will be the first + command, followed by a period (.), and then followed by the + new subcommand. For example, suppose that you have a database + already existing called my_data.db. The following example + shows the commands created when you open the database and when + you open a cursor: + </p> <pre class="programlisting"># First open the database and get a database command handle % berkdb open my_data.db db0 @@ -66,10 +75,16 @@ db0.c0 #Get the first data from the cursor % db0.c0 get -first {{first_key first_data}}</pre> - <p>All commands in the library support a special option <span class="bold"><strong>-?</strong></span> that will -list the correct operations for a command or the correct options.</p> - <p>A list of commands and operations can be found in the - <a href="../api_reference/TCL/index.html" class="olink">Tcl API</a> documentation.</p> + <p> + All commands in the library support a special option + <span class="bold"><strong>-?</strong></span> that will list the + correct operations for a command or the correct + options. + </p> + <p> + A list of commands and operations can be found in the <a href="../api_reference/TCL/index.html" class="olink">Tcl API</a> + documentation. + </p> </div> <div class="navfooter"> <hr /> @@ -82,9 +97,7 @@ list the correct operations for a command or the correct options.</p> <td width="40%" align="right"> <a accesskey="n" href="tcl_program.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 21. - Berkeley DB Extensions: Tcl - </td> + <td width="40%" align="left" valign="top">Chapter 21. Berkeley DB Extensions: Tcl </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/transapp.html b/docs/programmer_reference/transapp.html index 27dff98a..6ce1f7e8 100644 --- a/docs/programmer_reference/transapp.html +++ b/docs/programmer_reference/transapp.html @@ -14,13 +14,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th colspan="3" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="cam_app.html">Prev</a> </td> @@ -34,9 +32,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="transapp"></a>Chapter 11. - Berkeley DB Transactional Data Store Applications - </h2> + <h2 class="title"><a id="transapp"></a>Chapter 11. Berkeley DB Transactional Data Store Applications </h2> </div> </div> </div> @@ -67,7 +63,8 @@ </dt> <dt> <span class="sect1"> - <a href="transapp_app.html">Architecting Transactional Data Store applications</a> + <a href="transapp_app.html">Architecting Transactional Data + Store applications</a> </span> </dt> <dt> @@ -121,7 +118,8 @@ </dt> <dt> <span class="sect1"> - <a href="transapp_admin.html">Environment infrastructure</a> + <a href="transapp_admin.html">Environment + infrastructure</a> </span> </dt> <dt> @@ -136,7 +134,8 @@ </dt> <dt> <span class="sect1"> - <a href="transapp_archival.html">Database and log file archival</a> + <a href="transapp_archival.html">Database and log file + archival</a> </span> </dt> <dt> @@ -176,7 +175,8 @@ </dt> <dt> <span class="sect1"> - <a href="transapp_throughput.html">Transaction throughput</a> + <a href="transapp_throughput.html">Transaction + throughput</a> </span> </dt> <dt> @@ -194,25 +194,33 @@ </div> </div> </div> - <p>It is difficult to write a useful transactional tutorial and still keep -within reasonable bounds of documentation; that is, without writing a -book on transactional programming. We have two goals in this section: -to familiarize readers with the transactional interfaces of Berkeley DB and -to provide code building blocks that will be useful for creating -applications.</p> - <p>We have not attempted to present this information using a real-world -application. First, transactional applications are often complex and -time-consuming to explain. Also, one of our goals is to give you an -understanding of the wide variety of tools Berkeley DB makes available to you, -and no single application would use most of the interfaces included in -the Berkeley DB library. For these reasons, we have chosen to simply present -the Berkeley DB data structures and programming solutions, using examples that -differ from page to page. All the examples are included in a standalone -program you can examine, modify, and run; and from which you will be able -to extract code blocks for your own applications. Fragments of the -program will be presented throughout this chapter, and the complete text -of the <a class="ulink" href="transapp.cs" target="_top">example program</a> for IEEE/ANSI Std 1003.1 (POSIX) -standard systems is included in the Berkeley DB distribution.</p> + <p> + It is difficult to write a useful transactional tutorial and + still keep within reasonable bounds of documentation; that is, + without writing a book on transactional programming. We have + two goals in this section: to familiarize readers with the + transactional interfaces of Berkeley DB and to provide code + building blocks that will be useful for creating + applications. + </p> + <p> + We have not attempted to present this information using a + real-world application. First, transactional applications are + often complex and time-consuming to explain. Also, one of our + goals is to give you an understanding of the wide variety of + tools Berkeley DB makes available to you, and no single + application would use most of the interfaces included in the + Berkeley DB library. For these reasons, we have chosen to + simply present the Berkeley DB data structures and programming + solutions, using examples that differ from page to page. All + the examples are included in a standalone program you can + examine, modify, and run; and from which you will be able to + extract code blocks for your own applications. Fragments of + the program will be presented throughout this chapter, and the + complete text of the <a class="ulink" href="transapp.cs" target="_top">example + program</a> for IEEE/ANSI Std 1003.1 (POSIX) standard + systems is included in the Berkeley DB distribution. + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/transapp_admin.html b/docs/programmer_reference/transapp_admin.html index f115930f..dfa1b83f 100644 --- a/docs/programmer_reference/transapp_admin.html +++ b/docs/programmer_reference/transapp_admin.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Environment infrastructure</th> + <th colspan="3" align="center">Environment + infrastructure</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_nested.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_deadlock.html">Next</a></td> </tr> </table> @@ -34,35 +33,53 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="transapp_admin"></a>Environment infrastructure</h2> + <h2 class="title" style="clear: both"><a id="transapp_admin"></a>Environment + infrastructure</h2> </div> </div> </div> - <p>When building transactional applications, it is usually necessary to -build an administrative infrastructure around the database environment. -There are five components to this infrastructure, and each is -supported by the Berkeley DB package in two different ways: a standalone -utility and one or more library interfaces.</p> + <p> + When building transactional applications, it is usually + necessary to build an administrative infrastructure around the + database environment. There are five components to this + infrastructure, and each is supported by the Berkeley DB + package in two different ways: a standalone utility and one or + more library interfaces. + </p> <div class="itemizedlist"> <ul type="disc"> - <li>Deadlock detection: <a href="../api_reference/C/db_deadlock.html" class="olink">db_deadlock</a> utility, <a href="../api_reference/C/lockdetect.html" class="olink">DB_ENV->lock_detect()</a>, <a href="../api_reference/C/envset_lk_detect.html" class="olink">DB_ENV->set_lk_detect()</a></li> - <li>Checkpoints: the <a href="../api_reference/C/db_checkpoint.html" class="olink">db_checkpoint</a> utility, <a href="../api_reference/C/txncheckpoint.html" class="olink">DB_ENV->txn_checkpoint()</a></li> - <li>Database and log file archival: the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility, <a href="../api_reference/C/logarchive.html" class="olink">DB_ENV->log_archive()</a></li> - <li>Log file removal: <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility, <a href="../api_reference/C/logarchive.html" class="olink">DB_ENV->log_archive()</a></li> - <li>Recovery procedures: <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility, <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a></li> + <li> + Deadlock detection: <a href="../api_reference/C/db_deadlock.html" class="olink">db_deadlock</a> utility, <a href="../api_reference/C/lockdetect.html" class="olink">DB_ENV->lock_detect()</a>, + <a href="../api_reference/C/envset_lk_detect.html" class="olink">DB_ENV->set_lk_detect()</a></li> + <li> + Checkpoints: the <a href="../api_reference/C/db_checkpoint.html" class="olink">db_checkpoint</a> utility, + <a href="../api_reference/C/txncheckpoint.html" class="olink">DB_ENV->txn_checkpoint()</a></li> + <li> + Database and log file archival: the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility, + <a href="../api_reference/C/logarchive.html" class="olink">DB_ENV->log_archive()</a></li> + <li> + Log file removal: <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility, + <a href="../api_reference/C/logarchive.html" class="olink">DB_ENV->log_archive()</a></li> + <li> + Recovery procedures: <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility, + <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a></li> </ul> </div> - <p>When writing multithreaded server applications and/or applications -intended for download from the Web, it is usually simpler to create -local threads that are responsible for administration of the database -environment as scheduling is often simpler in a single-process model, -and only a single binary need be installed and run. However, the -supplied utilities can be generally useful tools even when the -application is responsible for doing its own administration because -applications rarely offer external interfaces to database -administration. The utilities are required when programming to a Berkeley DB -scripting interface because the scripting APIs do not always offer -interfaces to the administrative functionality.</p> + <p> + When writing multithreaded server applications and/or + applications intended for download from the Web, it is usually + simpler to create local threads that are responsible for + administration of the database environment as scheduling is + often simpler in a single-process model, and only a single + binary need be installed and run. However, the supplied + utilities can be generally useful tools even when the + application is responsible for doing its own administration + because applications rarely offer external interfaces to + database administration. The utilities are required when + programming to a Berkeley DB scripting interface because the + scripting APIs do not always offer interfaces to the + administrative functionality. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/transapp_app.html b/docs/programmer_reference/transapp_app.html index abab1363..b5b09e82 100644 --- a/docs/programmer_reference/transapp_app.html +++ b/docs/programmer_reference/transapp_app.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Architecting Transactional Data Store applications</th> + <th colspan="3" align="center">Architecting Transactional Data + Store applications</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_fail.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_env_open.html">Next</a></td> </tr> </table> @@ -34,350 +33,473 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="transapp_app"></a>Architecting Transactional Data Store applications</h2> + <h2 class="title" style="clear: both"><a id="transapp_app"></a>Architecting Transactional Data + Store applications</h2> </div> </div> </div> + <p> + When building Transactional Data Store applications, the + architecture decisions involve application startup (running + recovery) and handling system or application failure. For + details on performing recovery, see the <a class="xref" href="transapp_recovery.html" title="Recovery procedures">Recovery procedures</a>. + </p> + <p> + Recovery in a database environment is a single-threaded + procedure, that is, one thread of control or process must + complete database environment recovery before any other thread + of control or process operates in the Berkeley DB environment. + </p> + <p> + Performing recovery first marks any existing database + environment as "failed" and then removes it, causing threads + of control running in the database environment to fail and + return to the application. This feature allows applications to + recover environments without concern for threads of control + that might still be running in the removed environment. The + subsequent re-creation of the database environment is + serialized, so multiple threads of control attempting to + create a database environment will serialize behind a single + creating thread. + </p> + <p> + One consideration in removing (as part of recovering) a + database environment which may be in use by another thread, is + the type of mutex being used by the Berkeley DB library. In + the case of database environment failure when using + test-and-set mutexes, threads of control waiting on a mutex + when the environment is marked "failed" will quickly notice + the failure and will return an error from the Berkeley DB API. + In the case of environment failure when using blocking + mutexes, where the underlying system mutex implementation does + not unblock mutex waiters after the thread of control holding + the mutex dies, threads waiting on a mutex when an environment + is recovered might hang forever. Applications blocked on + events (for example, an application blocked on a network + socket, or a GUI event) may also fail to notice environment + recovery within a reasonable amount of time. Systems with such + mutex implementations are rare, but do exist; applications on + such systems should use an application architecture where the + thread recovering the database environment can explicitly + terminate any process using the failed environment, or + configure Berkeley DB for test-and-set mutexes, or incorporate + some form of long-running timer or watchdog process to wake or + kill blocked processes should they block for too long. + </p> <p> - When building Transactional Data Store applications, the architecture - decisions involve application startup (running recovery) and handling - system or application failure. For details on performing recovery, see - the <a class="xref" href="transapp_recovery.html" title="Recovery procedures">Recovery procedures</a>. -</p> + Regardless, it makes little sense for multiple threads of + control to simultaneously attempt recovery of a database + environment, since the last one to run will remove all + database environments created by the threads of control that + ran before it. However, for some applications, it may make + sense for applications to have a single thread of control that + performs recovery and then removes the database environment, + after which the application launches a number of processes, + any of which will create the database environment and continue + forward. + </p> <p> - Recovery in a database environment is a single-threaded procedure, that - is, one thread of control or process must complete database environment - recovery before any other thread of control or process operates in the - Berkeley DB environment. -</p> - <p> - Performing recovery first marks any existing database environment as - "failed" and then removes it, causing threads of control running in the - database environment to fail and return to the application. This - feature allows applications to recover environments without concern for - threads of control that might still be running in the removed - environment. The subsequent re-creation of the database environment is - serialized, so multiple threads of control attempting to create a - database environment will serialize behind a single creating thread. -</p> - <p> - One consideration in removing (as part of recovering) a database - environment which may be in use by another thread, is the type of mutex - being used by the Berkeley DB library. In the case of database - environment failure when using test-and-set mutexes, threads of control - waiting on a mutex when the environment is marked "failed" will quickly - notice the failure and will return an error from the Berkeley DB API. - In the case of environment failure when using blocking mutexes, where - the underlying system mutex implementation does not unblock mutex - waiters after the thread of control holding the mutex dies, threads - waiting on a mutex when an environment is recovered might hang forever. - Applications blocked on events (for example, an application blocked on - a network socket, or a GUI event) may also fail to notice environment - recovery within a reasonable amount of time. Systems with such mutex - implementations are rare, but do exist; applications on such systems - should use an application architecture where the thread recovering the - database environment can explicitly terminate any process using the - failed environment, or configure Berkeley DB for test-and-set mutexes, - or incorporate some form of long-running timer or watchdog process to - wake or kill blocked processes should they block for too long. -</p> - <p> - Regardless, it makes little sense for multiple threads of control to - simultaneously attempt recovery of a database environment, since the - last one to run will remove all database environments created by the - threads of control that ran before it. However, for some applications, - it may make sense for applications to have a single thread of control - that performs recovery and then removes the database environment, after - which the application launches a number of processes, any of which will - create the database environment and continue forward. -</p> - <p> - There are three common ways to architect Berkeley DB Transactional Data - Store applications. The one chosen is usually based on whether or not - the application is comprised of a single process or group of processes - descended from a single process (for example, a server started when the - system first boots), or if the application is comprised of unrelated - processes (for example, processes started by web connections or users - logged into the system). -</p> + There are four ways to architect Berkeley DB + Transactional Data Store applications. The one chosen is + usually based on whether or not the application is comprised + of a single process or group of processes descended from a + single process (for example, a server started when the system + first boots), or if the application is comprised of unrelated + processes (for example, processes started by web connections + or users logged into the system). + </p> <div class="orderedlist"> <ol type="1"> <li> <p> - The first way to architect Transactional Data Store - applications is as a single process (the process may or may not - be multithreaded.) - </p> + The first way to architect Transactional Data Store + applications is as a single process (the process may + or may not be multithreaded.) + </p> + <p> + When this process starts, it runs recovery on the + database environment and then opens its databases. The + application can subsequently create new threads as it + chooses. Those threads can either share already open + Berkeley DB <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> and <a href="../api_reference/C/db.html" class="olink">DB</a> handles, or create their + own. In this architecture, databases are rarely opened + or closed when more than a single thread of control is + running; that is, they are opened when only a single + thread is running, and closed after all threads but + one have exited. The last thread of control to exit + closes the databases and the database environment. + </p> + <p> + This architecture is simplest to implement because + thread serialization is easy and failure detection + does not require monitoring multiple processes. + </p> <p> - When this process starts, it runs recovery on the database - environment and then opens its databases. The application can - subsequently create new threads as it chooses. Those threads - can either share already open Berkeley DB <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> and <a href="../api_reference/C/db.html" class="olink">DB</a> - handles, or create their own. In this architecture, databases - are rarely opened or closed when more than a single thread of - control is running; that is, they are opened when only a single - thread is running, and closed after all threads but one have - exited. The last thread of control to exit closes the - databases and the database environment. - </p> + If the application's thread model allows processes + to continue after thread failure, the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> + method can be used to determine if the database + environment is usable after thread failure. If the + application does not call <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a>, or + <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> returns <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, + the application must + behave as if there has been a system failure, + performing recovery and re-creating the database + environment. Once these actions have been taken, other + threads of control can continue (as long as all + existing Berkeley DB handles are first discarded). + </p> <p> - This architecture is simplest to implement because thread - serialization is easy and failure detection does not require - monitoring multiple processes. - </p> - <p> - If the application's thread model allows processes to continue - after thread failure, the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method can be used to - determine if the database environment is usable after thread - failure. If the application does not call <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a>, or - <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> returns - <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, - the application must - behave as if there has been a system failure, performing - recovery and re-creating the database environment. Once these - actions have been taken, other threads of control can continue - (as long as all existing Berkeley DB handles are first - discarded). - </p> + Note that by default <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> will only notify the + calling thread that the database environment is unusable. + However, you can optionally cause <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> to broadcast + this to other threads of control by using the + <code class="literal">--enable-failchk_broadcast</code> flag when you + compile your Berkeley DB library. If this option is turned + on, then all threads of control using the database + environment will return + <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a> + when they attempt to obtain a mutex lock. In this + situation, a <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_FAILCHK_PANIC" class="olink">DB_EVENT_FAILCHK_PANIC</a> or + <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_MUTEX_DIED" class="olink">DB_EVENT_MUTEX_DIED</a> event will also be + raised. (You use <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> to examine events). + </p> </li> <li> + <p> + The second way to architect Transactional Data + Store applications is as a group of related processes + (the processes may or may not be multithreaded). + </p> <p> - The second way to architect Transactional Data Store - applications is as a group of related processes (the processes - may or may not be multithreaded). - </p> - <p> - This architecture requires the order in which threads of control are - created be controlled to serialize database environment recovery. - </p> - <p> - In addition, this architecture requires that threads of control - be monitored. If any thread of control exits with open - Berkeley DB handles, the application may call the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> - method to detect lost mutexes and locks and determine if the - application can continue. If the application does not call - <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a>, or <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> returns that the database - environment can no longer be used, the application must behave - as if there has been a system failure, performing recovery and - creating a new database environment. Once these actions have - been taken, other threads of control can be continued (as long - as all existing Berkeley DB handles are first discarded), or - - </p> + This architecture requires the order in which + threads of control are created be controlled to + serialize database environment recovery. + </p> <p> - The easiest way to structure groups of related processes is to - first create a single "watcher" process (often a script) that - starts when the system first boots, runs recovery on the - database environment and then creates the processes or threads - that will actually perform work. The initial thread has no - further responsibilities other than to wait on the threads of - control it has started, to ensure none of them unexpectedly - exit. If a thread of control exits, the watcher process - optionally calls the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method. If the application - does not call <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> or if <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> returns that the - environment can no longer be used, the watcher kills all of the - threads of control using the failed environment, runs recovery, - and starts new threads of control to perform work. - </p> + In addition, this architecture requires that + threads of control be monitored. If any thread of + control exits with open Berkeley DB handles, the + application may call the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method to detect + lost mutexes and locks and determine if the + application can continue. If the application does not + call <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a>, or <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> returns that the + database environment can no longer be used, the + application must behave as if there has been a system + failure, performing recovery and creating a new + database environment. Once these actions have been + taken, other threads of control can be continued (as + long as all existing Berkeley DB handles are first + discarded). + </p> + <p> + The easiest way to structure groups of related + processes is to first create a single "watcher" + process (often a script) that starts when the system + first boots, runs recovery on the database environment + and then creates the processes or threads that will + actually perform work. The initial thread has no + further responsibilities other than to wait on the + threads of control it has started, to ensure none of + them unexpectedly exit. If a thread of control exits, + the watcher process optionally calls the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> + method. If the application does not call <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> + or if <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> returns that the environment can no + longer be used, the watcher kills all of the threads + of control using the failed environment, runs + recovery, and starts new threads of control to perform + work. + </p> </li> <li> <p> - The third way to architect Transactional Data Store - applications is as a group of unrelated processes (the - processes may or may not be multithreaded). This is the most - difficult architecture to implement because of the level of - difficulty in some systems of finding and monitoring unrelated - processes. There are several possible techniques to implement - this architecture. - </p> + The third way to architect Transactional Data Store + applications is as a group of related processes that rely + on <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> broadcasting to inform other threads and + processes that recovery is required. <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> + broadcasting is not enabled by default for the DB + library, but using broadcasting means that a watcher + process is not required. Instead, if <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> fails + then all other threads and processes operating in that + environment will also be notified of that failure so that + they can know to run recovery. + </p> <p> - One solution is to log a thread of control ID when a new - Berkeley DB handle is opened. For example, an initial - "watcher" process could run recovery on the database - environment and then create a sentinel file. Any "worker" - process wanting to use the environment would check for the - sentinel file. If the sentinel file does not exist, the worker - would fail or wait for the sentinel file to be created. Once - the sentinel file exists, the worker would register its process - ID with the watcher (via shared memory, IPC or some other - registry mechanism), and then the worker would open its - <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handles and proceed. When the worker finishes - using the environment, it would unregister its process ID with - the watcher. The watcher periodically checks to ensure that no - worker has failed while using the environment. If a worker - fails while using the environment, the watcher removes the - sentinel file, kills all of the workers currently using the - environment, runs recovery on the environment, and finally - creates a new sentinel file. - </p> + To enable <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> broadcasting use the + <code class="literal">--enable-failchk_broadcast</code> flag when you + configure the library. On Windows, use + <code class="literal">HAVE_FAILCHK_BROADCAST</code> when you compile + the library. + </p> <p> - The weakness of this approach is that, on some systems, it is - difficult to determine if an unrelated process is still - running. For example, POSIX systems generally disallow sending - signals to unrelated processes. The trick to monitoring - unrelated processes is to find a system resource held by the - process that will be modified if the process dies. On POSIX - systems, flock- or fcntl-style locking will work, as will - LockFile on Windows systems. Other systems may have to use - other process-related information such as file reference counts - or modification times. In the worst case, threads of control - can be required to periodically re-register with the watcher - process: if the watcher has not heard from a thread of control - in a specified period of time, the watcher will take action, - recovering the environment. - </p> - <p> - The Berkeley DB library includes one built-in implementation of this approach, - the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method's <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> flag: - </p> - <p> - If the <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> flag is set, each process opening the - database environment first checks to see if recovery needs to - be performed. If recovery needs to be performed for any reason - (including the initial creation of the database environment), - and <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> is also specified, recovery will be performed - and then the open will proceed normally. If recovery needs to - be performed and <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> is not specified, - <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a> - will be returned. If recovery does not need to be performed, - <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> will be ignored. - </p> - <p> - Prior to the actual recovery beginning, the <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REG_PANIC" class="olink">DB_EVENT_REG_PANIC</a> - event is set for the environment. Processes in the application using - the <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> method will be notified when they do their next - operations in the environment. Processes receiving this event should - exit the environment. Also, the <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REG_ALIVE" class="olink">DB_EVENT_REG_ALIVE</a> event will be - triggered if there are other processes currently attached to the - environment. Only the process doing the recovery will receive this - event notification. It will receive this notification once for each - process still attached to the environment. The parameter of the - <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> callback will contain the process identifier of the - process still attached. The process doing the recovery can then - signal the attached process or perform some other operation prior to - recovery (i.e. kill the attached process). - </p> - <p> - The <a href="../api_reference/C/envset_timeout.html" class="olink">DB_ENV->set_timeout()</a> method's <a href="../api_reference/C/envset_timeout.html#set_timeout_DB_SET_REG_TIMEOUT" class="olink">DB_SET_REG_TIMEOUT</a> flag can be set to - establish a wait period before starting recovery. This creates a - window of time for other processes to receive the DB_EVENT_REG_PANIC - event and exit the environment. - </p> - <p> - There are three additional requirements for the <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> - architecture to work: - </p> + If <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> broadcasting is enabled for your library + and a thread of control encounters a failure when + <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> is run, then all other threads and processes + operating in that environment will be notified. If a + failure is broadcast, then threads and processes will + receive + <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a> + when they attempt to preform any one of a range of + activities, including: + </p> <div class="itemizedlist"> <ul type="disc"> <li> <p> - First, all applications using the database environment must - specify the <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> flag when opening the environment. - However, there is no additional requirement if the application - chooses a single process to recover the environment, as the - first process to open the database environment will know to - perform recovery. - </p> + When entering a DB API. + </p> </li> <li> <p> - Second, there can only be a single <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handle per database - environment in each process. As the <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> locking is - per-process, not per-thread, multiple <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handles in a single - environment could race with each other, potentially causing - data corruption. - </p> + When locking a mutex. + </p> </li> <li> <p> - Third, the <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> implementation does not explicitly - terminate processes using the database environment which is - being recovered. Instead, it relies on the processes - themselves noticing the database environment has been discarded - from underneath them. For this reason, the <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> flag - should be used with a mutex implementation that does not block - in the operating system, as that risks a thread of control - blocking forever on a mutex which will never be granted. Using - any test-and-set mutex implementation ensures this cannot - happen, and for that reason the <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> flag is generally - used with a test-and-set mutex implementation. - </p> + When performing disk or network I/O. + </p> </li> </ul> </div> <p> - A second solution for groups of unrelated processes is also - based on a "watcher process". This solution is intended for - systems where it is not practical to monitor the processes - sharing a database environment, but it is possible to monitor - the environment to detect if a thread of control has failed - holding open Berkeley DB handles. This would be done by having - a "watcher" process periodically call the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method. - If <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> returns that the environment can no longer be - used, the watcher would then take action, recovering the - environment. - </p> + Threads and processes that are + monitoring events will also receive + <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_FAILCHK_PANIC" class="olink">DB_EVENT_FAILCHK_PANIC</a> or + <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_MUTEX_DIED" class="olink">DB_EVENT_MUTEX_DIED</a>. You use + <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> to examine events. + </p> + </li> + <li> + <p> + The fourth way to architect Transactional Data Store + applications is as a group of unrelated processes (the + processes may or may not be multithreaded). This is + the most difficult architecture to implement because + of the level of difficulty in some systems of finding + and monitoring unrelated processes. There are several + possible techniques to implement this architecture. + </p> + <p> + One solution is to log a thread of control ID when + a new Berkeley DB handle is opened. For example, an + initial "watcher" process could run recovery on the + database environment and then create a sentinel file. + Any "worker" process wanting to use the environment + would check for the sentinel file. If the sentinel + file does not exist, the worker would fail or wait for + the sentinel file to be created. Once the sentinel + file exists, the worker would register its process ID + with the watcher (via shared memory, IPC or some other + registry mechanism), and then the worker would open + its <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handles and proceed. When the worker + finishes using the environment, it would unregister + its process ID with the watcher. The watcher + periodically checks to ensure that no worker has + failed while using the environment. If a worker fails + while using the environment, the watcher removes the + sentinel file, kills all of the workers currently + using the environment, runs recovery on the + environment, and finally creates a new sentinel file. + </p> + <p> + The weakness of this approach is that, on some + systems, it is difficult to determine if an unrelated + process is still running. For example, POSIX systems + generally disallow sending signals to unrelated + processes. The trick to monitoring unrelated processes + is to find a system resource held by the process that + will be modified if the process dies. On POSIX + systems, flock- or fcntl-style locking will work, as + will LockFile on Windows systems. Other systems may + have to use other process-related information such as + file reference counts or modification times. In the + worst case, threads of control can be required to + periodically re-register with the watcher process: if + the watcher has not heard from a thread of control in + a specified period of time, the watcher will take + action, recovering the environment. + </p> + <p> + The Berkeley DB library includes one built-in + implementation of this approach, the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> + method's <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> flag: + </p> <p> - The weakness of this approach is that all threads of control - using the environment must specify an "ID" function and an - "is-alive" function using the <a href="../api_reference/C/envset_thread_id.html" class="olink">DB_ENV->set_thread_id()</a> method. (In - other words, the Berkeley DB library must be able to assign a - unique ID to each thread of control, and additionally determine - if the thread of control is still running. It can be difficult - to portably provide that information in applications using a - variety of different programming languages and running on a - variety of different platforms.) - </p> + If the <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> flag is set, each process + opening the database environment first checks to see + if recovery needs to be performed. If recovery needs + to be performed for any reason (including the initial + creation of the database environment), and + <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> is also specified, recovery will be + performed and then the open will proceed normally. If + recovery needs to be performed and <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> is not + specified, <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a> + will be returned. If recovery does not need to be performed, <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> + will be ignored. + </p> <p> - A third solution for groups of unrelated processes is a hybrid of the two - above. Along with implementing the built-in sentinel approach with the - the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> methods <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> flag, the <a href="../api_reference/C/envopen.html#envopen_DB_FAILCHK" class="olink">DB_FAILCHK</a> flag can be specified. - When using both flags, each process opening the database environment first - checks to see if recocvery needs to be performed. If recovery needs to be - performed for any reason, it will first determine if a thread of control - exited while holding database read locks, and release those. Then it will - abort any unresolved transactions. If these steps are successful, the process - opening the environment will continue without the need for any - additional recocvery. If these steps are unsuccessful, then additional - recovery will be performed if <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> is specified and if <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> is not - specified, <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>will be returned. - </p> + Prior to the actual recovery beginning, the + <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REG_PANIC" class="olink">DB_EVENT_REG_PANIC</a> event is set for the environment. + Processes in the application using the + <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> method will be notified when they do + their next operations in the environment. Processes + receiving this event should exit the environment. + Also, the <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_REG_ALIVE" class="olink">DB_EVENT_REG_ALIVE</a> event will be triggered + if there are other processes currently attached to the + environment. Only the process doing the recovery will + receive this event notification. It will receive this + notification once for each process still attached to + the environment. The parameter of the + <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> callback will contain the process + identifier of the process still attached. The process + doing the recovery can then signal the attached + process or perform some other operation prior to + recovery (i.e. kill the attached process). + </p> <p> - Since this solution is hybrid of the first two, all of the requirements of both - of them must be implemented (will need "ID" function, "is-alive" function, - single <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handle per database, etc.) - </p> + The <a href="../api_reference/C/envset_timeout.html" class="olink">DB_ENV->set_timeout()</a> method's <a href="../api_reference/C/envset_timeout.html#set_timeout_DB_SET_REG_TIMEOUT" class="olink">DB_SET_REG_TIMEOUT</a> + flag can be set to establish a wait period before + starting recovery. This creates a window of time for + other processes to receive the DB_EVENT_REG_PANIC + event and exit the environment. + </p> <p> - The described approaches are different, and should not be - combined. Applications might use either the <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> - approach, the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> or the hybrid approach, but not together in - the same application. For example, a POSIX application written - as a library underneath a wide variety of interfaces and - differing APIs might choose the <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> approach for a - few reasons: first, it does not require making periodic calls - to the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method; second, when implementing in a - variety of languages, is may be more difficult to specify - unique IDs for each thread of control; third, it may be more - difficult determine if a thread of control is still running, as - any particular thread of control is likely to lack sufficient - permissions to signal other processes. Alternatively, an - application with a dedicated watcher process, running with - appropriate permissions, might choose the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> approach - as supporting higher overall throughput and reliability, as - that approach allows the application to abort unresolved - transactions and continue forward without having to recover the - database environment. The hybrid approach is useful in situations - where running a dedicated watcher process is not practical but getting the - equivalent of <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> on the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> is important. - </p> + There are three additional requirements for the + <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> architecture to work: + </p> + <div class="itemizedlist"> + <ul type="disc"> + <li> + <p> + First, all applications using the database + environment must specify the <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> + flag when opening the environment. However, + there is no additional requirement if the + application chooses a single process to + recover the environment, as the first process + to open the database environment will know to + perform recovery. + </p> + </li> + <li> + <p> + Second, there can only be a single <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> + handle per database environment in each + process. As the <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> locking is + per-process, not per-thread, multiple <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> + handles in a single environment could race + with each other, potentially causing data + corruption. + </p> + </li> + <li> + <p> + Third, the <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> implementation + does not explicitly terminate processes using + the database environment which is being + recovered. Instead, it relies on the processes + themselves noticing the database environment + has been discarded from underneath them. For + this reason, the <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> flag should be + used with a mutex implementation that does not + block in the operating system, as that risks a + thread of control blocking forever on a mutex + which will never be granted. Using any + test-and-set mutex implementation ensures this + cannot happen, and for that reason the + <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> flag is generally used with a + test-and-set mutex implementation. + </p> + </li> + </ul> + </div> + <p> + A second solution for groups of unrelated processes + is also based on a "watcher process". This solution is + intended for systems where it is not practical to + monitor the processes sharing a database environment, + but it is possible to monitor the environment to + detect if a thread of control has failed holding open + Berkeley DB handles. This would be done by having a + "watcher" process periodically call the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> + method. If <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> returns that the environment + can no longer be used, the watcher would then take + action, recovering the environment. + </p> + <p> + The weakness of this approach is that all threads + of control using the environment must specify an "ID" + function and an "is-alive" function using the + <a href="../api_reference/C/envset_thread_id.html" class="olink">DB_ENV->set_thread_id()</a> method. (In other words, the + Berkeley DB library must be able to assign a unique ID + to each thread of control, and additionally determine + if the thread of control is still running. It can be + difficult to portably provide that information in + applications using a variety of different programming + languages and running on a variety of different + platforms.) + </p> + <p> + A third solution for groups of unrelated processes + is a hybrid of the two above. Along with implementing + the built-in sentinel approach with the the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> + methods <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> flag, the <a href="../api_reference/C/envopen.html#envopen_DB_FAILCHK" class="olink">DB_FAILCHK</a> flag can + be specified. When using both flags, each process + opening the database environment first checks to see + if recovery needs to be performed. If recovery needs + to be performed for any reason, it will first + determine if a thread of control exited while holding + database read locks, and release those. Then it will + abort any unresolved transactions. If these steps are + successful, the process opening the environment will + continue without the need for any additional recovery. + If these steps are unsuccessful, then additional + recovery will be performed if <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> is + specified and if <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> is not specified, <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a> + will be returned. + </p> + <p> + Since this solution is hybrid of the first two, all + of the requirements of both of them must be + implemented (will need "ID" function, "is-alive" + function, single <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> handle per database, etc.) + </p> + <p> + The described approaches are different, and should + not be combined. Applications might use either the + <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> approach, the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> or the hybrid + approach, but not together in the same application. + For example, a POSIX application written as a library + underneath a wide variety of interfaces and differing + APIs might choose the <a href="../api_reference/C/envopen.html#envopen_DB_REGISTER" class="olink">DB_REGISTER</a> approach for a few + reasons: first, it does not require making periodic + calls to the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method; second, when + implementing in a variety of languages, is may be more + difficult to specify unique IDs for each thread of + control; third, it may be more difficult determine if + a thread of control is still running, as any + particular thread of control is likely to lack + sufficient permissions to signal other processes. + Alternatively, an application with a dedicated watcher + process, running with appropriate permissions, might + choose the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> approach as supporting higher + overall throughput and reliability, as that approach + allows the application to abort unresolved + transactions and continue forward without having to + recover the database environment. The hybrid approach + is useful in situations where running a dedicated + watcher process is not practical but getting the + equivalent of <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> on the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> is + important. + </p> </li> </ol> </div> <p> - Obviously, when implementing a process to monitor other threads of - control, it is important the watcher process' code be as simple and - well-tested as possible, because the application may hang if it fails. -</p> + Obviously, when implementing a process to monitor other + threads of control, it is important the watcher process' code + be as simple and well-tested as possible, because the + application may hang if it fails. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/transapp_archival.html b/docs/programmer_reference/transapp_archival.html index 2923d54c..e007b697 100644 --- a/docs/programmer_reference/transapp_archival.html +++ b/docs/programmer_reference/transapp_archival.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Database and log file archival</th> + <th colspan="3" align="center">Database and log file + archival</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_checkpoint.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_logfile.html">Next</a></td> </tr> </table> @@ -34,218 +33,237 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="transapp_archival"></a>Database and log file archival</h2> + <h2 class="title" style="clear: both"><a id="transapp_archival"></a>Database and log file + archival</h2> </div> </div> </div> - <p> - The third component of the administrative infrastructure, archival - for catastrophic recovery, concerns the recoverability of the - database in the face of catastrophic failure. Recovery after - catastrophic failure is intended to minimize data loss when - physical hardware has been destroyed — for example, loss of a - disk that contains databases or log files. Although the - application may still experience data loss in this case, it is - possible to minimize it. + <p> + The third component of the administrative infrastructure, + archival for catastrophic recovery, concerns the + recoverability of the database in the face of catastrophic + failure. Recovery after catastrophic failure is intended to + minimize data loss when physical hardware has been destroyed + — for example, loss of a disk that contains databases or + log files. Although the application may still experience data + loss in this case, it is possible to minimize it. </p> - <p> - First, you may want to periodically create snapshots (that is, - backups) of your databases to make it possible to recover from - catastrophic failure. These snapshots are either a standard - backup, which creates a consistent picture of the databases as of a - single instant in time; or an on-line backup (also known as a - <span class="emphasis"><em>hot</em></span> backup), which creates a consistent - picture of the databases as of an unspecified instant during the - period of time when the snapshot was made. The advantage of a hot - backup is that applications may continue to read and write the - databases while the snapshot is being taken. The disadvantage of a - hot backup is that more information must be archived, and recovery - based on a hot backup is to an unspecified time between the start - of the backup and when the backup is completed. + <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> + <h3 class="title">Note</h3> + <p> + Berkeley DB backups (archives) can be recovered using + machines of differing byte order. That is, a backup taken + on a big-endian machine can be used to restore a database + residing on a little-endian machine. + </p> + </div> + <p> + First, you may want to periodically create snapshots (that + is, backups) of your databases to make it possible to recover + from catastrophic failure. These snapshots are either a + standard backup, which creates a consistent picture of the + databases as of a single instant in time; or an on-line backup + (also known as a <span class="emphasis"><em>hot</em></span> backup), which + creates a consistent picture of the databases as of an + unspecified instant during the period of time when the + snapshot was made. The advantage of a hot backup is that + applications may continue to read and write the databases + while the snapshot is being taken. The disadvantage of a hot + backup is that more information must be archived, and recovery + based on a hot backup is to an unspecified time between the + start of the backup and when the backup is completed. </p> <p> - Second, after taking a snapshot, you should periodically archive - the log files being created in the environment. It is often - helpful to think of database archival in terms of full and - incremental filesystem backups. A snapshot is a full backup, - whereas the periodic archival of the current log files is an - incremental backup. For example, it might be reasonable to take a - full snapshot of a database environment weekly or monthly, and - archive additional log files daily. Using both the snapshot and - the log files, a catastrophic crash at any time can be recovered to - the time of the most recent log archival; a time long after the - original snapshot. + Second, after taking a snapshot, you should periodically + archive the log files being created in the environment. It is + often helpful to think of database archival in terms of full + and incremental filesystem backups. A snapshot is a full + backup, whereas the periodic archival of the current log files + is an incremental backup. For example, it might be reasonable + to take a full snapshot of a database environment weekly or + monthly, and archive additional log files daily. Using both + the snapshot and the log files, a catastrophic crash at any + time can be recovered to the time of the most recent log + archival; a time long after the original snapshot. </p> <p> - When incremental backups are implemented using this procedure, it - is important to know that a database copy taken prior to a bulk - loading event (that is, a transaction started with the - <a href="../api_reference/C/txnbegin.html#txnbegin_DB_TXN_BULK" class="olink">DB_TXN_BULK</a> flag) can no longer be used as the target of an - incremental backup. This is true because bulk loading omits - logging of some record insertions, so these insertions cannot be - rolled forward by recovery. It is recommended that a full backup - be scheduled following a bulk loading event. + When incremental backups are implemented using this + procedure, it is important to know that a database copy taken + prior to a bulk loading event (that is, a transaction started + with the <a href="../api_reference/C/txnbegin.html#txnbegin_DB_TXN_BULK" class="olink">DB_TXN_BULK</a> flag) can no longer be used as the + target of an incremental backup. This is true because bulk + loading omits logging of some record insertions, so these + insertions cannot be rolled forward by recovery. It is + recommended that a full backup be scheduled following a bulk + loading event. </p> - <p> - To create a standard backup of your database that can be used to - recover from catastrophic failure, take the following steps: + <p> + To create a standard backup of your database that can be + used to recover from catastrophic failure, take the following + steps: </p> <div class="orderedlist"> <ol type="1"> <li> - Commit or abort all ongoing transactions. + Commit or abort all ongoing transactions. </li> - <li> - Stop writing your databases until the backup has completed. - Read-only operations are permitted, but no write operations and - no filesystem operations may be performed (for example, the - <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> and <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> methods may not be called). + <li> + Stop writing your databases until the backup has + completed. Read-only operations are permitted, but no + write operations and no filesystem operations may be + performed (for example, the <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> and <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> + methods may not be called). </li> - <li> - Force an environment checkpoint (see the <a href="../api_reference/C/db_checkpoint.html" class="olink">db_checkpoint</a> utility for - more information). + <li> + Force an environment checkpoint (see the + <a href="../api_reference/C/db_checkpoint.html" class="olink">db_checkpoint</a> utility for more information). </li> <li> - <p> - Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility with option <span class="bold"><strong>-s</strong></span> to identify all the database data - files, and copy them to a backup device such as CD-ROM, alternate - disk, or tape. + <p> + Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility with option <span class="bold"><strong>-s</strong></span> to identify all the + database data files, and copy them to a backup device + such as CD-ROM, alternate disk, or tape. </p> <p> - If the database files are stored in a separate directory from the - other Berkeley DB files, it may be simpler to archive the directory - itself instead of the individual files (see <a href="../api_reference/C/envset_data_dir.html" class="olink">DB_ENV->set_data_dir()</a> for - additional information). + If the database files are stored in a separate + directory from the other Berkeley DB files, it may be + simpler to archive the directory itself instead of the + individual files (see <a href="../api_reference/C/envadd_data_dir.html" class="olink">DB_ENV->add_data_dir()</a> for additional + information on storing database files in separate + directories). </p> <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> <h3 class="title">Note</h3> <p> - If any of the database files did not have an open <a href="../api_reference/C/db.html" class="olink">DB</a> - handle during the lifetime of the current log files, - the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility will not list them in its output. This - is another reason it may be simpler to use a separate - database file directory and archive the entire - directory instead of archiving only the files listed by - the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility. + If any of the database files did not have an + open <a href="../api_reference/C/db.html" class="olink">DB</a> handle during the lifetime of the + current log files, the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility will not list + them in its output. This is another reason it may + be simpler to use a separate database file + directory and archive the entire directory instead + of archiving only the files listed by the + <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility. </p> </div> </li> <li> - Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility with option <span class="bold"><strong>-l</strong></span> to identify all the log files, - and copy the last one (that is, the one with the highest - number) to a backup device such as CD-ROM, alternate disk, or - tape. + Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility with option <span class="bold"><strong>-l</strong></span> to identify all the log + files, and copy the last one (that is, the one with the + highest number) to a backup device such as CD-ROM, + alternate disk, or tape. </li> </ol> </div> <p> - To create a <span class="emphasis"><em>hot</em></span> backup of your database that - can be used to recover from catastrophic failure, take the - following steps: + To create a <span class="emphasis"><em>hot</em></span> backup of your + database that can be used to recover from catastrophic + failure, take the following steps: </p> <div class="orderedlist"> <ol type="1"> <li> - Set the <a href="../api_reference/C/envset_flags.html#set_flags_DB_HOTBACKUP_IN_PROGRESS" class="olink">DB_HOTBACKUP_IN_PROGRESS</a> flag in the environment. - This affects the behavior of transactions started with the - <a href="../api_reference/C/txnbegin.html#txnbegin_DB_TXN_BULK" class="olink">DB_TXN_BULK</a> flag. + Set the <a href="../api_reference/C/envset_flags.html#set_flags_DB_HOTBACKUP_IN_PROGRESS" class="olink">DB_HOTBACKUP_IN_PROGRESS</a> flag in the + environment. This affects the behavior of transactions + started with the <a href="../api_reference/C/txnbegin.html#txnbegin_DB_TXN_BULK" class="olink">DB_TXN_BULK</a> flag. </li> <li> <p> - Archive your databases, as described in the previous step - #4. You do not have to halt ongoing transactions or force - a checkpoint. As this is a hot backup, and the databases - may be modified during the copy, it is critical that - database pages be read atomically as described by - <a class="xref" href="transapp_reclimit.html" title="Berkeley DB recoverability">Berkeley DB recoverability</a>. - </p> + Archive your databases, as described in the + previous step #4. You do not have to halt ongoing + transactions or force a checkpoint. As this is a hot + backup, and the databases may be modified during the + copy, it is critical that database pages be read + atomically as described by <a class="xref" href="transapp_reclimit.html" title="Berkeley DB recoverability">Berkeley DB recoverability</a>. </p> <p> - Note that only UNIX based systems are known to support the - atomicity of reads. These systems include: Solaris, Mac OSX, - HPUX and various BSD based systems. Linux and Windows - based systems do not support atomic filesystem reads - directly. The XFS file system supports atomic reads - despite the lack of it in Linux. On systems that do not - support atomic file system reads, the <a href="../api_reference/C/db_hotbackup.html" class="olink">db_hotbackup</a> utility - should be used or a tool can be constructed - using the <a href="../api_reference/C/envbackup.html" class="olink">DB_ENV->backup()</a> method. Alternatively, you can - construct a tool using the the <a href="../api_reference/C/db_copy.html" class="olink">db_copy()</a> method. You can - also perform a hot backup of just a single database in your - environment using the <a href="../api_reference/C/envdbbackup.html" class="olink">DB_ENV->dbbackup()</a> method. + Note that only UNIX based systems are known to + support the atomicity of reads. These systems include: + Solaris, Mac OSX, HPUX and various BSD based systems. + Linux and Windows based systems do not support atomic + filesystem reads directly. The XFS file system + supports atomic reads despite the lack of it in Linux. + On systems that do not support atomic file system + reads, the <a href="../api_reference/C/db_hotbackup.html" class="olink">db_hotbackup</a> utility should be used or a tool can + be constructed using the <a href="../api_reference/C/envbackup.html" class="olink">DB_ENV->backup()</a> method. + Alternatively, you can construct a tool using the the + <a href="../api_reference/C/db_copy.html" class="olink">db_copy()</a> method. You can also perform a hot backup of + just a single database in your environment using the + <a href="../api_reference/C/envdbbackup.html" class="olink">DB_ENV->dbbackup()</a> method. </p> </li> <li> - <p> - Archive <span class="bold"><strong>all</strong></span> of the log files. - The order of these two operations is required, and the database - files must be archived <span class="bold"><strong>before</strong></span> - the log files. This means that if the database files and log - files are in the same directory, you cannot simply archive the - directory; you must make sure that the correct order of - archival is maintained. + <p> + Archive <span class="bold"><strong>all</strong></span> of the + log files. The order of these two operations is + required, and the database files must be archived + <span class="bold"><strong>before</strong></span> the log + files. This means that if the database files and log + files are in the same directory, you cannot simply + archive the directory; you must make sure that the + correct order of archival is maintained. </p> <p> - To archive your log files, run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility using the - <span class="bold"><strong>-l</strong></span> option to identify all the - database log files, and copy them to your backup media. If the - database log files are stored in a separate directory from the - other database files, it may be simpler to archive the - directory itself instead of the individual files (see the + To archive your log files, run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility + using the <span class="bold"><strong>-l</strong></span> option + to identify all the database log files, and copy them + to your backup media. If the database log files are + stored in a separate directory from the other database + files, it may be simpler to archive the directory + itself instead of the individual files (see the <a href="../api_reference/C/envset_lg_dir.html" class="olink">DB_ENV->set_lg_dir()</a> method for more information). </p> </li> - <li> - Reset the <a href="../api_reference/C/envset_flags.html#set_flags_DB_HOTBACKUP_IN_PROGRESS" class="olink">DB_HOTBACKUP_IN_PROGRESS</a> flag. + <li> Reset the <a href="../api_reference/C/envset_flags.html#set_flags_DB_HOTBACKUP_IN_PROGRESS" class="olink">DB_HOTBACKUP_IN_PROGRESS</a> flag. </li> </ol> </div> <p> - To minimize the archival space needed for log files when doing a - hot backup, run db_archive to identify those log files which are - not in use. Log files which are not in use do not need to be - included when creating a hot backup, and you can discard them or - move them aside for use with previous backups (whichever is - appropriate), before beginning the hot backup. + To minimize the archival space needed for log files when + doing a hot backup, run db_archive to identify those log files + which are not in use. Log files which are not in use do not + need to be included when creating a hot backup, and you can + discard them or move them aside for use with previous backups + (whichever is appropriate), before beginning the hot backup. </p> - <p> - After completing one of these two sets of steps, the database - environment can be recovered from catastrophic failure (see - <a class="xref" href="transapp_recovery.html" title="Recovery procedures">Recovery procedures</a> - for more information). + <p> + After completing one of these two sets of steps, the + database environment can be recovered from catastrophic + failure (see <a class="xref" href="transapp_recovery.html" title="Recovery procedures">Recovery procedures</a> for more information). </p> <p> To update either a hot or cold backup so that recovery from - catastrophic failure is possible to a new point in time, repeat - step #2 under the hot backup instructions and archive - <span class="bold"><strong>all</strong></span> of the log files in the - database environment. Each time both the database and log files - are copied to backup media, you may discard all previous database - snapshots and saved log files. Archiving additional log files does - not allow you to discard either previous database snapshots or log - files. Generally, updating a backup must be integrated with the - application's log file removal procedures. + catastrophic failure is possible to a new point in time, + repeat step #2 under the hot backup instructions and archive + <span class="bold"><strong>all</strong></span> of the log files in + the database environment. Each time both the database and log + files are copied to backup media, you may discard all previous + database snapshots and saved log files. Archiving additional + log files does not allow you to discard either previous + database snapshots or log files. Generally, updating a backup + must be integrated with the application's log file removal + procedures. </p> - <p> - The time to restore from catastrophic failure is a function of the - number of log records that have been written since the snapshot was - originally created. Perhaps more importantly, the more separate - pieces of backup media you use, the more likely it is that you will - have a problem reading from one of them. For these reasons, it is - often best to make snapshots on a regular basis. + <p> + The time to restore from catastrophic failure is a function + of the number of log records that have been written since the + snapshot was originally created. Perhaps more importantly, the + more separate pieces of backup media you use, the more likely + it is that you will have a problem reading from one of them. + For these reasons, it is often best to make snapshots on a + regular basis. </p> <p> - <span class="bold"><strong>Obviously, the reliability of your archive - media will affect the safety of your data. For archival - safety, ensure that you have multiple copies of your database - backups, verify that your archival media is error-free and - readable, and that copies of your backups are stored - offsite!</strong></span> + <span class="bold"><strong>Obviously, the reliability of your + archive media will affect the safety of your data. For + archival safety, ensure that you have multiple copies of + your database backups, verify that your archival media is + error-free and readable, and that copies of your backups + are stored offsite!</strong></span> </p> - <p> - The functionality provided by the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility is also available - directly from the Berkeley DB library. The following code fragment - prints out a list of log and database files that need to be - archived: + <p> + The functionality provided by the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility is also + available directly from the Berkeley DB library. The following + code fragment prints out a list of log and database files that + need to be archived: </p> <pre class="programlisting">void log_archlist(DB_ENV *dbenv) diff --git a/docs/programmer_reference/transapp_atomicity.html b/docs/programmer_reference/transapp_atomicity.html index 8813e7a2..a5d8a748 100644 --- a/docs/programmer_reference/transapp_atomicity.html +++ b/docs/programmer_reference/transapp_atomicity.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_put.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_inc.html">Next</a></td> </tr> </table> @@ -38,47 +36,63 @@ </div> </div> </div> - <p>The second reason listed for using transactions was <span class="emphasis"><em>atomicity</em></span>. -Atomicity means that multiple operations can be grouped into a single -logical entity, that is, other threads of control accessing the database -will either see all of the changes or none of the changes. Atomicity -is important for applications wanting to update two related databases -(for example, a primary database and secondary index) in a single -logical action. Or, for an application wanting to update multiple -records in one database in a single logical action.</p> - <p>Any number of operations on any number of databases can be included in -a single transaction to ensure the atomicity of the operations. There -is, however, a trade-off between the number of operations included in -a single transaction and both throughput and the possibility of -deadlock. The reason for this is because transactions acquire locks -throughout their lifetime and do not release the locks until commit or -abort time. So, the more operations included in a transaction, the more -likely it is that a transaction will block other operations and that -deadlock will occur. However, each transaction commit requires a -synchronous disk I/O, so grouping multiple operations into a transaction -can increase overall throughput. (There is one exception to this: the -<a href="../api_reference/C/envset_flags.html#set_flags_DB_TXN_WRITE_NOSYNC" class="olink">DB_TXN_WRITE_NOSYNC</a> and <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> flags cause -transactions to exhibit the ACI (atomicity, consistency and isolation) -properties, but not D (durability); avoiding the write and/or -synchronous disk I/O on transaction commit greatly increases transaction -throughput for some applications.)</p> - <p>When applications do create complex transactions, they often avoid -having more than one complex transaction at a time because simple -operations like a single <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> are unlikely to deadlock with -each other or the complex transaction; while multiple complex -transactions are likely to deadlock with each other because they will -both acquire many locks over their lifetime. Alternatively, complex -transactions can be broken up into smaller sets of operations, and each -of those sets may be encapsulated in a nested transaction. Because -nested transactions may be individually aborted and retried without -causing the entire transaction to be aborted, this allows complex -transactions to proceed even in the face of heavy contention, repeatedly -trying the suboperations until they succeed.</p> - <p>It is also helpful to order operations within a transaction; that is, -access the databases and items within the databases in the same order, -to the extent possible, in all transactions. Accessing databases and -items in different orders greatly increases the likelihood of operations -being blocked and failing due to deadlocks.</p> + <p> + The second reason listed for using transactions was + <span class="emphasis"><em>atomicity</em></span>. Atomicity means that + multiple operations can be grouped into a single logical + entity, that is, other threads of control accessing the + database will either see all of the changes or none of the + changes. Atomicity is important for applications wanting to + update two related databases (for example, a primary database + and secondary index) in a single logical action. Or, for an + application wanting to update multiple records in one database + in a single logical action. + </p> + <p> + Any number of operations on any number of databases can be + included in a single transaction to ensure the atomicity of + the operations. There is, however, a trade-off between the + number of operations included in a single transaction and both + throughput and the possibility of deadlock. The reason for + this is because transactions acquire locks throughout their + lifetime and do not release the locks until commit or abort + time. So, the more operations included in a transaction, the + more likely it is that a transaction will block other + operations and that deadlock will occur. However, each + transaction commit requires a synchronous disk I/O, so + grouping multiple operations into a transaction can increase + overall throughput. (There is one exception to this: the + <a href="../api_reference/C/envset_flags.html#set_flags_DB_TXN_WRITE_NOSYNC" class="olink">DB_TXN_WRITE_NOSYNC</a> and <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> flags cause + transactions to exhibit the ACI (atomicity, consistency and + isolation) properties, but not D (durability); avoiding the + write and/or synchronous disk I/O on transaction commit + greatly increases transaction throughput for some + applications.) + </p> + <p> + When applications do create complex transactions, they often + avoid having more than one complex transaction at a time + because simple operations like a single <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> are unlikely + to deadlock with each other or the complex transaction; while + multiple complex transactions are likely to deadlock with each + other because they will both acquire many locks over their + lifetime. Alternatively, complex transactions can be broken up + into smaller sets of operations, and each of those sets may be + encapsulated in a nested transaction. Because nested + transactions may be individually aborted and retried without + causing the entire transaction to be aborted, this allows + complex transactions to proceed even in the face of heavy + contention, repeatedly trying the suboperations until they + succeed. + </p> + <p> + It is also helpful to order operations within a transaction; + that is, access the databases and items within the databases + in the same order, to the extent possible, in all + transactions. Accessing databases and items in different + orders greatly increases the likelihood of operations being + blocked and failing due to deadlocks. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/transapp_checkpoint.html b/docs/programmer_reference/transapp_checkpoint.html index c5044179..bbfbe4d7 100644 --- a/docs/programmer_reference/transapp_checkpoint.html +++ b/docs/programmer_reference/transapp_checkpoint.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_deadlock.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_archival.html">Next</a></td> </tr> </table> @@ -38,31 +36,41 @@ </div> </div> </div> - <p>The second component of the infrastructure is performing checkpoints of -the log files. Performing checkpoints is necessary for two reasons.</p> - <p>First, you may be able to remove Berkeley DB log files from your database -environment after a checkpoint. Change records are written into the log -files when databases are modified, but the actual changes to the -database are not necessarily written to disk. When a checkpoint is -performed, changes to the database are written into the backing database -file. Once the database pages are written, log files can be archived -and removed from the database environment because they will never be -needed for anything other than catastrophic failure. (Log files which -are involved in active transactions may not be removed, and there must -always be at least one log file in the database environment.)</p> - <p>The second reason to perform checkpoints is because checkpoint frequency -is inversely proportional to the amount of time it takes to run database -recovery after a system or application failure. This is because -recovery after failure has to redo or undo changes only since the last -checkpoint, as changes before the checkpoint have all been flushed to -the databases.</p> <p> - Berkeley DB provides the <a href="../api_reference/C/db_checkpoint.html" class="olink">db_checkpoint</a> utility, which can be used to perform - checkpoints. Alternatively, applications can write their own - checkpoint thread using the underlying <a href="../api_reference/C/txncheckpoint.html" class="olink">DB_ENV->txn_checkpoint()</a> function. The - following code fragment checkpoints the database environment every 60 - seconds: -</p> + The second component of the infrastructure is performing + checkpoints of the log files. Performing checkpoints is + necessary for two reasons. + </p> + <p> + First, you may be able to remove Berkeley DB log files from + your database environment after a checkpoint. Change records + are written into the log files when databases are modified, + but the actual changes to the database are not necessarily + written to disk. When a checkpoint is performed, changes to + the database are written into the backing database file. Once + the database pages are written, log files can be archived and + removed from the database environment because they will never + be needed for anything other than catastrophic failure. (Log + files which are involved in active transactions may not be + removed, and there must always be at least one log file in the + database environment.) + </p> + <p> + The second reason to perform checkpoints is because + checkpoint frequency is inversely proportional to the amount + of time it takes to run database recovery after a system or + application failure. This is because recovery after failure + has to redo or undo changes only since the last checkpoint, as + changes before the checkpoint have all been flushed to the + databases. + </p> + <p> + Berkeley DB provides the <a href="../api_reference/C/db_checkpoint.html" class="olink">db_checkpoint</a> utility, which can be used + to perform checkpoints. Alternatively, applications can write + their own checkpoint thread using the underlying + <a href="../api_reference/C/txncheckpoint.html" class="olink">DB_ENV->txn_checkpoint()</a> function. The following code fragment + checkpoints the database environment every 60 seconds: + </p> <pre class="programlisting">int main(int argc, char *argv) { @@ -139,9 +147,11 @@ checkpoint_thread(void *arg) /* NOTREACHED */ }</strong></span></pre> - <p>Because checkpoints can be quite expensive, choosing how often to -perform a checkpoint is a common tuning parameter for Berkeley DB -applications.</p> + <p> + Because checkpoints can be quite expensive, choosing how + often to perform a checkpoint is a common tuning parameter for + Berkeley DB applications. + </p> </div> <div class="navfooter"> <hr /> @@ -158,7 +168,8 @@ applications.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Database and log file archival</td> + <td width="40%" align="right" valign="top"> Database and log file + archival</td> </tr> </table> </div> diff --git a/docs/programmer_reference/transapp_cursor.html b/docs/programmer_reference/transapp_cursor.html index b202d9e0..2c36d2cf 100644 --- a/docs/programmer_reference/transapp_cursor.html +++ b/docs/programmer_reference/transapp_cursor.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_read.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_nested.html">Next</a></td> </tr> </table> @@ -38,18 +36,24 @@ </div> </div> </div> - <p>Berkeley DB cursors may be used inside a transaction, exactly as any other -<a href="../api_reference/C/db.html" class="olink">DB</a> method. The enclosing transaction ID must be specified when -the cursor is created, but it does not then need to be further specified -on operations performed using the cursor. One important point to -remember is that a cursor <span class="bold"><strong>must be closed</strong></span> before the enclosing -transaction is committed or aborted.</p> - <p>The following code fragment uses a cursor to store a new key in the cats -database with four associated data items. The key is a name. The data -items are a company name and a list of the breeds of cat owned. Each -of the data entries is stored as a duplicate data item. In this -example, transactions are necessary to ensure that either all or none -of the data items appear in case of system or application failure.</p> + <p> + Berkeley DB cursors may be used inside a transaction, + exactly as any other <a href="../api_reference/C/db.html" class="olink">DB</a> method. The enclosing transaction ID + must be specified when the cursor is created, but it does not + then need to be further specified on operations performed + using the cursor. One important point to remember is that a + cursor <span class="bold"><strong>must be closed</strong></span> before + the enclosing transaction is committed or aborted. + </p> + <p> + The following code fragment uses a cursor to store a new key + in the cats database with four associated data items. The key + is a name. The data items are a company name and a list of the + breeds of cat owned. Each of the data entries is stored as a + duplicate data item. In this example, transactions are + necessary to ensure that either all or none of the data items + appear in case of system or application failure. + </p> <pre class="programlisting">int main(int argc, char *argv) { diff --git a/docs/programmer_reference/transapp_data_open.html b/docs/programmer_reference/transapp_data_open.html index 7315fe12..ca9ca0b4 100644 --- a/docs/programmer_reference/transapp_data_open.html +++ b/docs/programmer_reference/transapp_data_open.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_env_open.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_put.html">Next</a></td> </tr> </table> @@ -38,10 +36,13 @@ </div> </div> </div> - <p>Next, we open three databases ("color" and "fruit" and "cats"), in the -database environment. Again, our <a href="../api_reference/C/db.html" class="olink">DB</a> database handles are -declared to be free-threaded using the <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flag, and so -may be used by any number of threads we subsequently create.</p> + <p> + Next, we open three databases ("color" and "fruit" and + "cats"), in the database environment. Again, our <a href="../api_reference/C/db.html" class="olink">DB</a> database + handles are declared to be free-threaded using the <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> + flag, and so may be used by any number of threads we + subsequently create. + </p> <a id="prog_transapp2"></a> <pre class="programlisting">int main(int argc, char *argv[]) @@ -119,8 +120,10 @@ db_open(DB_ENV *dbenv, DB **dbp, char *name, int dups) *dbp = db; return (0); }</pre> - <p>After opening the database, we can use the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility to -display information about a database we have created:</p> + <p> + After opening the database, we can use the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility to + display information about a database we have created: + </p> <pre class="programlisting">prompt> db_stat -h TXNAPP -d color 53162 Btree magic number. 8 Btree version number. @@ -139,30 +142,40 @@ Flags: 0 Number of tree overflow pages. 0 Number of bytes free in tree overflow pages (0% ff). 0 Number of pages on the free list.</pre> - <p>The database open must be enclosed within a transaction in order to be -recoverable. The transaction will ensure that created files are -re-created in recovered environments (or do not appear at all). -Additional database operations or operations on other databases can be -included in the same transaction, of course. In the simple case, where -the open is the only operation in the transaction, an application can -set the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_AUTO_COMMIT" class="olink">DB_AUTO_COMMIT</a> flag instead of creating and managing -its own transaction handle. The <a href="../api_reference/C/envset_flags.html#envset_flags_DB_AUTO_COMMIT" class="olink">DB_AUTO_COMMIT</a> flag will -internally wrap the operation in a transaction, simplifying application -code.</p> - <p>The previous example is the simplest case of transaction protection for -database open. Obviously, additional database operations can be done -in the scope of the same transaction. For example, an application -maintaining a list of the databases in a database environment in a -well-known file might include an update of the list in the same -transaction in which the database is created. Or, an application might -create both a primary and secondary database in a single transaction.</p> - <p><a href="../api_reference/C/db.html" class="olink">DB</a> handles that will later be used for transactionally protected -database operations must be opened within a transaction. Specifying a -transaction handle to database operations using <a href="../api_reference/C/db.html" class="olink">DB</a> handles not -opened within a transaction will return an error. Similarly, not -specifying a transaction handle to database operations that will modify -the database, using handles that were opened within a transaction, will -also return an error.</p> + <p> + The database open must be enclosed within a transaction in + order to be recoverable. The transaction will ensure that + created files are re-created in recovered environments (or do + not appear at all). Additional database operations or + operations on other databases can be included in the same + transaction, of course. In the simple case, where the open is + the only operation in the transaction, an application can set + the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_AUTO_COMMIT" class="olink">DB_AUTO_COMMIT</a> flag instead of creating and managing its + own transaction handle. The <a href="../api_reference/C/envset_flags.html#envset_flags_DB_AUTO_COMMIT" class="olink">DB_AUTO_COMMIT</a> flag will + internally wrap the operation in a transaction, simplifying + application code. + </p> + <p> + The previous example is the simplest case of transaction + protection for database open. Obviously, additional database + operations can be done in the scope of the same transaction. + For example, an application maintaining a list of the + databases in a database environment in a well-known file might + include an update of the list in the same transaction in which + the database is created. Or, an application might create both + a primary and secondary database in a single + transaction. + </p> + <p> + <a href="../api_reference/C/db.html" class="olink">DB</a> handles that will later be used for transactionally + protected database operations must be opened within a + transaction. Specifying a transaction handle to database + operations using <a href="../api_reference/C/db.html" class="olink">DB</a> handles not opened within a transaction + will return an error. Similarly, not specifying a transaction + handle to database operations that will modify the database, + using handles that were opened within a transaction, will also + return an error. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/transapp_deadlock.html b/docs/programmer_reference/transapp_deadlock.html index d457745c..e0a0ceb8 100644 --- a/docs/programmer_reference/transapp_deadlock.html +++ b/docs/programmer_reference/transapp_deadlock.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_admin.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_checkpoint.html">Next</a></td> </tr> </table> @@ -38,58 +36,63 @@ </div> </div> </div> - <p> - The first component of the infrastructure, - <span class="emphasis"><em>deadlock detection</em></span>, is not so much a - requirement specific to transaction-protected applications, but instead - is necessary for almost all applications in which more than a single - thread of control will be accessing the database at one time. Even - when Berkeley DB automatically handles database locking, it is normally - possible for deadlock to occur. Because the underlying database access - methods may update multiple pages during a single Berkeley DB API call, - deadlock is possible even when threads of control are making only - single update calls into the database. The exception to this rule is - when all the threads of control accessing the database are read-only or - when the Berkeley DB Concurrent Data Store product is used; the - Berkeley DB Concurrent Data Store product guarantees deadlock-free - operation at the expense of reduced concurrency. -</p> - <p> - When the deadlock occurs, two (or more) threads of control each request - additional locks that can never be granted because one of the threads - of control waiting holds the requested resource. For example, consider - two processes: A and B. Let's say that A obtains a write lock on item - X, and B obtains a write lock on item Y. Then, A requests a lock on Y, - and B requests a lock on X. A will wait until resource Y becomes - available and B will wait until resource X becomes available. - Unfortunately, because both A and B are waiting, neither will release - the locks they hold and neither will ever obtain the resource on which - it is waiting. For another example, consider two transactions, A and - B, each of which may want to modify item X. Assume that transaction A - obtains a read lock on X and confirms that a modification is needed. - Then it is descheduled and the thread containing transaction B runs. - At that time, transaction B obtains a read lock on X and confirms that - it also wants to make a modification. Both transactions A and B will - block when they attempt to upgrade their read locks to write locks - because the other already holds a read lock. This is a deadlock. - Transaction A cannot make forward progress until Transaction B releases - its read lock on X, but Transaction B cannot make forward progress - until Transaction A releases its read lock on X. -</p> - <p> - In order to detect that deadlock has happened, a separate process or - thread must review the locks currently held in the database. If - deadlock has occurred, a victim must be selected, and that victim will - then return the error - <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a> - from whatever Berkeley DB call it was making. Berkeley DB provides the - <a href="../api_reference/C/db_deadlock.html" class="olink">db_deadlock</a> utility that can be used to perform this deadlock detection. - Alternatively, applications can create their own deadlock utility or - thread using the underlying <a href="../api_reference/C/lockdetect.html" class="olink">DB_ENV->lock_detect()</a> function, or specify that - Berkeley DB run the deadlock detector internally whenever there is a - conflict over a lock (see <a href="../api_reference/C/envset_lk_detect.html" class="olink">DB_ENV->set_lk_detect()</a> for more information). - The following code fragment does the latter: -</p> + <p> + The first component of the infrastructure, + <span class="emphasis"><em>deadlock detection</em></span>, is not so much a + requirement specific to transaction-protected applications, + but instead is necessary for almost all applications in which + more than a single thread of control will be accessing the + database at one time. Even when Berkeley DB automatically + handles database locking, it is normally possible for deadlock + to occur. Because the underlying database access methods may + update multiple pages during a single Berkeley DB API call, + deadlock is possible even when threads of control are making + only single update calls into the database. The exception to + this rule is when all the threads of control accessing the + database are read-only or when the Berkeley DB Concurrent Data + Store product is used; the Berkeley DB Concurrent Data Store + product guarantees deadlock-free operation at the expense of + reduced concurrency. + </p> + <p> + When the deadlock occurs, two (or more) threads of control + each request additional locks that can never be granted + because one of the threads of control waiting holds the + requested resource. For example, consider two processes: A and + B. Let's say that A obtains a write lock on item X, and B + obtains a write lock on item Y. Then, A requests a lock on Y, + and B requests a lock on X. A will wait until resource Y + becomes available and B will wait until resource X becomes + available. Unfortunately, because both A and B are waiting, + neither will release the locks they hold and neither will ever + obtain the resource on which it is waiting. For another + example, consider two transactions, A and B, each of which may + want to modify item X. Assume that transaction A obtains a + read lock on X and confirms that a modification is needed. + Then it is descheduled and the thread containing transaction B + runs. At that time, transaction B obtains a read lock on X and + confirms that it also wants to make a modification. Both + transactions A and B will block when they attempt to upgrade + their read locks to write locks because the other already + holds a read lock. This is a deadlock. Transaction A cannot + make forward progress until Transaction B releases its read + lock on X, but Transaction B cannot make forward progress + until Transaction A releases its read lock on X. + </p> + <p> + In order to detect that deadlock has happened, a separate + process or thread must review the locks currently held in the + database. If deadlock has occurred, a victim must be selected, + and that victim will then return the error <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a> from + whatever Berkeley DB call it was making. Berkeley DB provides the <a href="../api_reference/C/db_deadlock.html" class="olink">db_deadlock</a> utility that + can be used to perform this deadlock detection. Alternatively, + applications can create their own deadlock utility or thread + using the underlying <a href="../api_reference/C/lockdetect.html" class="olink">DB_ENV->lock_detect()</a> function, or specify that + Berkeley DB run the deadlock detector internally whenever + there is a conflict over a lock (see <a href="../api_reference/C/envset_lk_detect.html" class="olink">DB_ENV->set_lk_detect()</a> for + more information). The following code fragment does the + latter: + </p> <pre class="programlisting">void env_open(DB_ENV **dbenvp) { @@ -130,11 +133,12 @@ env_open(DB_ENV **dbenvp) *dbenvp = dbenv; }</pre> - <p> - Deciding how often to run the deadlock detector and which of the - deadlocked transactions will be forced to abort when the deadlock is - detected is a common tuning parameter for Berkeley DB applications. -</p> + <p> + Deciding how often to run the deadlock detector and which + of the deadlocked transactions will be forced to abort when + the deadlock is detected is a common tuning parameter for + Berkeley DB applications. + </p> </div> <div class="navfooter"> <hr /> @@ -147,7 +151,8 @@ env_open(DB_ENV **dbenvp) <td width="40%" align="right"> <a accesskey="n" href="transapp_checkpoint.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Environment infrastructure </td> + <td width="40%" align="left" valign="top">Environment + infrastructure </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/transapp_env_open.html b/docs/programmer_reference/transapp_env_open.html index 414a013d..6630d687 100644 --- a/docs/programmer_reference/transapp_env_open.html +++ b/docs/programmer_reference/transapp_env_open.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_app.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_data_open.html">Next</a></td> </tr> </table> @@ -38,39 +36,42 @@ </div> </div> </div> + <p> + Creating transaction-protected applications using the + Berkeley DB library is quite easy. Applications first use + <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> to initialize the database environment. + Transaction-protected applications normally require all four + Berkeley DB subsystems, so the <a href="../api_reference/C/envopen.html#envopen_DB_INIT_MPOOL" class="olink">DB_INIT_MPOOL</a>, + <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOCK" class="olink">DB_INIT_LOCK</a>, <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOG" class="olink">DB_INIT_LOG</a>, and <a href="../api_reference/C/envopen.html#envopen_DB_INIT_TXN" class="olink">DB_INIT_TXN</a> flags should + be specified. + </p> + <p> + Once the application has called <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a>, it opens its + databases within the environment. Once the databases are + opened, the application makes changes to the databases inside + of transactions. Each set of changes that entails a unit of + work should be surrounded by the appropriate <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a>, + <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a> and <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> calls. The Berkeley DB access + methods will make the appropriate calls into the Lock, Log and + Memory Pool subsystems in order to guarantee transaction + semantics. When the application is ready to exit, all + outstanding transactions should have been committed or + aborted. + </p> <p> - Creating transaction-protected applications using the Berkeley DB - library is quite easy. Applications first use <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> to initialize - the database environment. Transaction-protected applications normally - require all four Berkeley DB subsystems, so the <a href="../api_reference/C/envopen.html#envopen_DB_INIT_MPOOL" class="olink">DB_INIT_MPOOL</a>, - <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOCK" class="olink">DB_INIT_LOCK</a>, <a href="../api_reference/C/envopen.html#envopen_DB_INIT_LOG" class="olink">DB_INIT_LOG</a>, and <a href="../api_reference/C/envopen.html#envopen_DB_INIT_TXN" class="olink">DB_INIT_TXN</a> flags should be - specified. -</p> + Databases accessed by a transaction must not be closed + during the transaction. Once all outstanding transactions are + finished, all open Berkeley DB files should be closed. When + the Berkeley DB database files have been closed, the + environment should be closed by calling <a href="../api_reference/C/envclose.html" class="olink">DB_ENV->close()</a>. + </p> <p> - Once the application has called <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a>, it opens its databases - within the environment. Once the databases are opened, the application - makes changes to the databases inside of transactions. Each set of - changes that entails a unit of work should be surrounded by the - appropriate <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a>, <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a> and <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> calls. The Berkeley - DB access methods will make the appropriate calls into the Lock, Log - and Memory Pool subsystems in order to guarantee transaction semantics. - When the application is ready to exit, all outstanding transactions - should have been committed or aborted. -</p> - <p> - Databases accessed by a transaction must not be closed during the - transaction. Once all outstanding transactions are finished, all open - Berkeley DB files should be closed. When the Berkeley DB database - files have been closed, the environment should be closed by calling - <a href="../api_reference/C/envclose.html" class="olink">DB_ENV->close()</a>. -</p> - <p> - The following code fragment creates the database environment directory - then opens the environment, running recovery. Our <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> database - environment handle is declared to be free-threaded using the - <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flag, and so may be used by any number of threads that we - may subsequently create. -</p> + The following code fragment creates the database + environment directory then opens the environment, running + recovery. Our <a href="../api_reference/C/env.html" class="olink">DB_ENV</a> database environment handle is declared to + be free-threaded using the <a href="../api_reference/C/dbopen.html#open_DB_THREAD" class="olink">DB_THREAD</a> flag, and so may be + used by any number of threads that we may subsequently create. + </p> <a id="prog_transapp1"></a> <pre class="programlisting">#include <sys/types.h> #include <sys/stat.h> @@ -172,9 +173,10 @@ env_open(DB_ENV **dbenvp) *dbenvp = dbenv; }</pre> <p> - After running this initial program, we can use the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility to display - the contents of the environment directory: -</p> + After running this initial program, we can use the + <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility to display the contents of the environment + directory: + </p> <pre class="programlisting">prompt> db_stat -e -h TXNAPP 3.2.1 Environment version. 120897 Magic number. @@ -205,8 +207,7 @@ Txn Region: 5. 8KB Size (8192 bytes). -1 Segment ID. 1 Locks granted without waiting. -0 Locks granted after waiting. -</pre> +0 Locks granted after waiting.</pre> </div> <div class="navfooter"> <hr /> @@ -219,7 +220,8 @@ Txn Region: 5. <td width="40%" align="right"> <a accesskey="n" href="transapp_data_open.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Architecting Transactional Data Store applications </td> + <td width="40%" align="left" valign="top">Architecting Transactional Data + Store applications </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/transapp_fail.html b/docs/programmer_reference/transapp_fail.html index 1ac9442f..f58cfb20 100644 --- a/docs/programmer_reference/transapp_fail.html +++ b/docs/programmer_reference/transapp_fail.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_term.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_app.html">Next</a></td> </tr> </table> @@ -38,77 +36,101 @@ </div> </div> </div> - <p> - When building Transactional Data Store applications, there are design - issues to consider whenever a thread of control with open Berkeley DB - handles fails for any reason (where a thread of control may be either a - true thread or a process). -</p> - <p> - The first case is handling system failure: if the system fails, the - database environment and the databases may be left in a corrupted - state. In this case, recovery must be performed on the database - environment before any further action is taken, in order to: -</p> + <p> + When building Transactional Data Store applications, there + are design issues to consider whenever a thread of control + with open Berkeley DB handles fails for any reason (where a + thread of control may be either a true thread or a process). + </p> + <p> + The first case is handling system failure: if the system + fails, the database environment and the databases may be left + in a corrupted state. In this case, recovery must be performed + on the database environment before any further action is + taken, in order to: + </p> <div class="itemizedlist"> <ul type="disc"> - <li>recover the database environment resources,</li> - <li>release any locks or mutexes that may have been held to avoid starvation -as the remaining threads of control convoy behind the held locks, and</li> - <li>resolve any partially completed operations that may have left a database -in an inconsistent or corrupted state.</li> + <li> + recover the database environment + resources, + </li> + <li> + release any locks or mutexes that may have been held + to avoid starvation as the remaining threads of control + convoy behind the held locks, and + </li> + <li> + resolve any partially completed operations that may + have left a database in an inconsistent or corrupted + state. + </li> </ul> </div> <p> - For details on performing recovery, see the - <a class="xref" href="transapp_recovery.html" title="Recovery procedures">Recovery procedures</a>. -</p> - <p> - The second case is handling the failure of a thread of control. There - are resources maintained in database environments that may be left - locked or corrupted if a thread of control exits unexpectedly. These - resources include data structure mutexes, logical database locks and - unresolved transactions (that is, transactions which were never aborted - or committed). While Transactional Data Store applications can treat - the failure of a thread of control in the same way as they do a system - failure, they have an alternative choice, the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method. -</p> + For details on performing recovery, see the <a class="xref" href="transapp_recovery.html" title="Recovery procedures">Recovery procedures</a>. + </p> <p> - The <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> will return - <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a> - if the database - environment is unusable as a result of the thread of control failure. - (If a data structure mutex or a database write lock is left held by - thread of control failure, the application should not continue to use - the database environment, as subsequent use of the environment is - likely to result in threads of control convoying behind the held - locks.) The <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> call will release any database read locks - that have been left held by the exit of a thread of control, and abort - any unresolved transactions. In this case, the application can - continue to use the database environment. -</p> + The second case is handling the failure of a thread of + control. There are resources maintained in database + environments that may be left locked or corrupted if a thread + of control exits unexpectedly. These resources include data + structure mutexes, logical database locks and unresolved + transactions (that is, transactions which were never aborted + or committed). While Transactional Data Store applications can + treat the failure of a thread of control in the same way as + they do a system failure, they have an alternative choice, the + <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method. + </p> + <p> + The <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method will return + <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a> + if the database environment is unusable as a result of the thread + of control failure. (If a data structure mutex or a database write + lock is left held by thread of control failure, the application + should not continue to use the database environment, as subsequent + use of the environment is likely to result in threads of control + convoying behind the held locks.) The <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> call will + release any database read locks that have been left held by the + exit of a thread of control, and abort any unresolved transactions. + In this case, the application can continue to use the database + environment. + </p> <p> - A Transactional Data Store application recovering from a thread of - control failure should call <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a>, and, if it returns success, - the application can continue. If <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> returns - <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, - the application should proceed as described for - the case of system failure. -</p> + Note that you can optionally cause <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> to broadcast a database + environment failure to other threads of control by using the + <code class="literal">--enable-failchk_broadcast</code> flag when you compile + your Berkeley DB library. If this option is turned on, then all + threads of control using the database environment will return + <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a> + when they attempt to obtain a mutex lock. In this situation, a + <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_FAILCHK_PANIC" class="olink">DB_EVENT_FAILCHK_PANIC</a> or + <a href="../api_reference/C/envevent_notify.html#event_notify_DB_EVENT_MUTEX_DIED" class="olink">DB_EVENT_MUTEX_DIED</a> event will also be raised. + (You use <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> to examine events). + </p> + <p> + A Transactional Data Store application recovering from a + thread of control failure should call <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a>, and, if it + returns success, the application can continue. If <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> + returns <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, + the application should proceed as described for the case of system + failure. In addition, threads notified of failure by <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> + should also proceed as described for the case of system failure. + </p> <p> - It greatly simplifies matters that recovery may be performed regardless - of whether recovery needs to be performed; that is, it is not an error - to recover a database environment for which recovery is not strictly - necessary. For this reason, applications should not try to determine - if the database environment was active when the application or system - failed. Instead, applications should run recovery any time the - <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method returns - <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, - or, if the application is - not calling the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method, any time any thread of control - accessing the database environment fails, as well as any time the - system reboots. -</p> + It greatly simplifies matters that recovery may be + performed regardless of whether recovery needs to be + performed; that is, it is not an error to recover a database + environment for which recovery is not strictly necessary. For + this reason, applications should not try to determine if the + database environment was active when the application or system + failed. Instead, applications should run recovery any time the + <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method returns <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY"> + DB_RUNRECOVERY</a>, or, if the application is not + calling the <a href="../api_reference/C/envfailchk.html" class="olink">DB_ENV->failchk()</a> method, any time any thread of + control accessing the database environment fails, as well as + any time the system reboots. + </p> </div> <div class="navfooter"> <hr /> @@ -125,7 +147,8 @@ in an inconsistent or corrupted state.</li> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Architecting Transactional Data Store applications</td> + <td width="40%" align="right" valign="top"> Architecting Transactional Data + Store applications</td> </tr> </table> </div> diff --git a/docs/programmer_reference/transapp_faq.html b/docs/programmer_reference/transapp_faq.html index f2e0b872..86d8a540 100644 --- a/docs/programmer_reference/transapp_faq.html +++ b/docs/programmer_reference/transapp_faq.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_throughput.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="rep.html">Next</a></td> </tr> </table> @@ -42,145 +40,169 @@ <ol type="1"> <li> <p> - <span class="bold"><strong>What should a transactional program do when an error occurs?</strong></span> - </p> + <span class="bold"><strong>What should a transactional + program do when an error occurs?</strong></span> + </p> <p> - Any time an error occurs, such that a transactionally protected - set of operations cannot complete successfully, the transaction - must be aborted. While deadlock is by far the most common of - these errors, there are other possibilities; for example, - running out of disk space for the filesystem. In Berkeley DB - transactional applications, there are three classes of error - returns: "expected" errors, "unexpected but recoverable" - errors, and a single "unrecoverable" error. Expected errors - are errors like - <a class="link" href="program_errorret.html#program_errorret.DB_NOTFOUND">DB_NOTFOUND</a>, - which indicates that a searched-for key item is not present in - the database. Applications may want to explicitly test for and - handle this error, or, in the case where the absence of a key - implies the enclosing transaction should fail, simply call - <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a>. Unexpected but recoverable errors are errors like - <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a>, - which indicates that an operation has been selected to resolve - a deadlock, or a system error such as EIO, which likely - indicates that the filesystem has no available disk space. - Applications must immediately call <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> when these - returns occur, as it is not possible to proceed otherwise. The - only unrecoverable error is - <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, - which indicates that the system must stop and recovery must be - run. - </p> + Any time an error occurs, such that a + transactionally protected set of operations cannot + complete successfully, the transaction must be + aborted. While deadlock is by far the most common of + these errors, there are other possibilities; for + example, running out of disk space for the filesystem. + In Berkeley DB transactional applications, there are + three classes of error returns: "expected" errors, + "unexpected but recoverable" errors, and a single + "unrecoverable" error. Expected errors are errors like + <a class="link" href="program_errorret.html#program_errorret.DB_NOTFOUND">DB_NOTFOUND</a>, + which indicates that a searched-for key item is not + present in the database. Applications may want to + explicitly test for and handle this error, or, in + the case where the absence of a key implies the enclosing + transaction should fail, simply call <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a>. Unexpected but + recoverable errors are errors like <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a>, + which indicates that an + operation has been selected to resolve a deadlock, or + a system error such as EIO, which likely indicates + that the filesystem has no available disk space. + Applications must immediately call <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> when + these returns occur, as it is not possible to proceed + otherwise. The only unrecoverable error is <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, + which indicates that the system must stop and recovery must + be run. + </p> </li> <li> <p> - <span class="bold"><strong>How can hot backups work? Can't you get an - inconsistent picture of the database when you copy - it?</strong></span> - </p> + <span class="bold"><strong>How can hot backups work? Can't + you get an inconsistent picture of the database + when you copy it?</strong></span> + </p> <p> - First, Berkeley DB is based on the technique of "write-ahead - logging", which means that before any change is made to a - database, a log record is written that describes the change. - Further, Berkeley DB guarantees that the log record that - describes the change will always be written to stable storage - (that is, disk) before the database page where the change was - made is written to stable storage. Because of this guarantee, - we know that any change made to a database will appear either - in just a log file, or both the database and a log file, but - never in just the database. - </p> - <p> - Second, you can always create a consistent and correct database - based on the log files and the databases from a database - environment. So, during a hot backup, we first make a copy of - the databases and then a copy of the log files. The tricky - part is that there may be pages in the database that are - related for which we won't get a consistent picture during this - copy. For example, let's say that we copy pages 1-4 of the - database, and then are swapped out. For whatever reason - (perhaps because we needed to flush pages from the cache, or - because of a checkpoint), the database pages 1 and 5 are - written. Then, the hot backup process is re-scheduled, and it - copies page 5. Obviously, we have an inconsistent database - snapshot, because we have a copy of page 1 from before it was - written by the other thread of control, and a copy of page 5 - after it was written by the other thread. What makes this work - is the order of operations in a hot backup. Because of the - write-ahead logging guarantees, we know that any page written - to the database will first be referenced in the log. If we - copy the database first, then we can also know that any - inconsistency in the database will be described in the log - files, and so we know that we can fix everything up during - recovery. - </p> + First, Berkeley DB is based on the technique of + "write-ahead logging", which means that before any + change is made to a database, a log record is written + that describes the change. Further, Berkeley DB + guarantees that the log record that describes the + change will always be written to stable storage (that + is, disk) before the database page where the change + was made is written to stable storage. Because of this + guarantee, we know that any change made to a database + will appear either in just a log file, or both the + database and a log file, but never in just the + database. + </p> + <p> + Second, you can always create a consistent and + correct database based on the log files and the + databases from a database environment. So, during a + hot backup, we first make a copy of the databases and + then a copy of the log files. The tricky part is that + there may be pages in the database that are related + for which we won't get a consistent picture during + this copy. For example, let's say that we copy pages + 1-4 of the database, and then are swapped out. For + whatever reason (perhaps because we needed to flush + pages from the cache, or because of a checkpoint), the + database pages 1 and 5 are written. Then, the hot + backup process is re-scheduled, and it copies page 5. + Obviously, we have an inconsistent database snapshot, + because we have a copy of page 1 from before it was + written by the other thread of control, and a copy of + page 5 after it was written by the other thread. What + makes this work is the order of operations in a hot + backup. Because of the write-ahead logging guarantees, + we know that any page written to the database will + first be referenced in the log. If we copy the + database first, then we can also know that any + inconsistency in the database will be described in the + log files, and so we know that we can fix everything + up during recovery. + </p> </li> <li> <p> - <span class="bold"><strong>My application has <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a> - errors. Is the normal, and what should I do?</strong></span> - </p> + <span class="bold"><strong>My application has <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a> + errors. Is the + normal, and what should I do?</strong></span> + </p> <p> - It is quite rare for a transactional application to be deadlock - free. All applications should be prepared to handle deadlock - returns, because even if the application is deadlock free when - deployed, future changes to the application or the Berkeley DB - implementation might introduce deadlocks. - </p> + It is quite rare for a transactional application to + be deadlock free. All applications should be prepared + to handle deadlock returns, because even if the + application is deadlock free when deployed, future + changes to the application or the Berkeley DB + implementation might introduce deadlocks. + </p> <p> - Practices which reduce the chance of deadlock include: - -</p> + Practices which reduce the chance of deadlock + include: </p> <div class="itemizedlist"> <ul type="disc"> - <li>Not using cursors which move backwards through the database (<a href="../api_reference/C/dbcget.html#dbcget_DB_PREV" class="olink">DB_PREV</a>), -as backward scanning cursors can deadlock with page splits;</li> - <li>Configuring <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_REVSPLITOFF" class="olink">DB_REVSPLITOFF</a> to turn off reverse splits in -applications which repeatedly delete and re-insert the same keys, to -minimize the number of page splits as keys are re-inserted;</li> - <li>Not configuring <a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a> as that flag requires write -transactions upgrade their locks when aborted, which can lead to deadlock. -Generally, <a href="../api_reference/C/dbcget.html#dbcget_DB_READ_COMMITTED" class="olink">DB_READ_COMMITTED</a> or non-transactional read operations -are less prone to deadlock than <a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a>.</li> + <li>Not using cursors which move backwards + through the database (<a href="../api_reference/C/dbcget.html#dbcget_DB_PREV" class="olink">DB_PREV</a>), as backward + scanning cursors can deadlock with page + splits;</li> + <li>Configuring + <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_REVSPLITOFF" class="olink">DB_REVSPLITOFF</a> to turn off reverse splits in + applications which repeatedly delete and + re-insert the same keys, to minimize the + number of page splits as keys are + re-inserted;</li> + <li>Not + configuring <a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a> as that flag + requires write transactions upgrade their + locks when aborted, which can lead to + deadlock. Generally, <a href="../api_reference/C/dbcget.html#dbcget_DB_READ_COMMITTED" class="olink">DB_READ_COMMITTED</a> or + non-transactional read operations are less + prone to deadlock than + <a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a>.</li> </ul> </div> <p> - </p> + </p> </li> <li> <p> - <span class="bold"><strong>How can I move a database from one - transactional environment into another?</strong></span> - </p> - <p> - Because database pages contain references to log records, - databases cannot be simply moved into different database - environments. To move a database into a different environment, - dump and reload the database before moving it. If the database - is too large to dump and reload, the database may be prepared - in place using the <a href="../api_reference/C/envlsn_reset.html" class="olink">DB_ENV->lsn_reset()</a> method or the <span class="bold"><strong>-r</strong></span> argument to the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility. - </p> + <span class="bold"><strong>How can I move a database from + one transactional environment into + another?</strong></span> + </p> + <p> + Because database pages contain references to log + records, databases cannot be simply moved into + different database environments. To move a database + into a different environment, dump and reload the + database before moving it. If the database is too + large to dump and reload, the database may be prepared + in place using the <a href="../api_reference/C/envlsn_reset.html" class="olink">DB_ENV->lsn_reset()</a> method or the + <span class="bold"><strong>-r</strong></span> argument to + the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility. + </p> </li> <li> <p> - <span class="bold"><strong>I'm seeing the error "log_flush: LSN past - current end-of-log", what does that mean?</strong></span> - </p> + <span class="bold"><strong>I'm seeing the error "log_flush: + LSN past current end-of-log", what does that + mean?</strong></span> + </p> <p> - The most common cause of this error is that a system - administrator has removed all of the log files from a database - environment. You should shut down your database environment as - gracefully as possible, first flushing the database environment - cache to disk, if that's possible. Then, dump and reload your - databases. If the database is too large to dump and reload, - the database may be reset in place using the <a href="../api_reference/C/envlsn_reset.html" class="olink">DB_ENV->lsn_reset()</a> - method or the <span class="bold"><strong>-r</strong></span> argument to - the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility. However, if you reset the database in place, - you should verify your databases before using them again. (It - is possible for the databases to be corrupted by running after - all of the log files have been removed, and the longer the - application runs, the worse it can get.) - </p> + The most common cause of this error is that a + system administrator has removed all of the log files + from a database environment. You should shut down your + database environment as gracefully as possible, first + flushing the database environment cache to disk, if + that's possible. Then, dump and reload your databases. + If the database is too large to dump and reload, the + database may be reset in place using the + <a href="../api_reference/C/envlsn_reset.html" class="olink">DB_ENV->lsn_reset()</a> method or the <span class="bold"><strong>-r</strong></span> + argument to the <a href="../api_reference/C/db_load.html" class="olink">db_load</a> utility. However, + if you reset the database in place, you should verify + your databases before using them again. (It is + possible for the databases to be corrupted by running + after all of the log files have been removed, and the + longer the application runs, the worse it can get.) + </p> </li> </ol> </div> @@ -196,13 +218,12 @@ are less prone to deadlock than <a href="../api_reference/C/dbopen.html#dbopen_D <td width="40%" align="right"> <a accesskey="n" href="rep.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Transaction throughput </td> + <td width="40%" align="left" valign="top">Transaction + throughput </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 12. - Berkeley DB Replication - </td> + <td width="40%" align="right" valign="top"> Chapter 12. Berkeley DB Replication </td> </tr> </table> </div> diff --git a/docs/programmer_reference/transapp_filesys.html b/docs/programmer_reference/transapp_filesys.html index 549dc2a7..70dbb49d 100644 --- a/docs/programmer_reference/transapp_filesys.html +++ b/docs/programmer_reference/transapp_filesys.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_journal.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_reclimit.html">Next</a></td> </tr> </table> @@ -38,60 +36,79 @@ </div> </div> </div> + <p> + The Berkeley DB API supports creating, removing and + renaming files. Creating files is supported by the <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> + method. Removing files is supported by the <a href="../api_reference/C/envdbremove.html" class="olink">DB_ENV->dbremove()</a> and + <a href="../api_reference/C/dbremove.html" class="olink">DB->remove()</a> methods. Renaming files is supported by the + <a href="../api_reference/C/envdbrename.html" class="olink">DB_ENV->dbrename()</a> and <a href="../api_reference/C/dbrename.html" class="olink">DB->rename()</a> methods. (There are two methods + for removing and renaming files because one of the methods is + transactionally protected and one is not.) + </p> + <p> + Berkeley DB does not permit specifying the <a href="../api_reference/C/dbopen.html#open_DB_TRUNCATE" class="olink">DB_TRUNCATE</a> + flag when opening a file in a transaction-protected + environment. This is an implicit file deletion, but one that + does not always require the same operating system file + permissions as deleting and creating a file do. + </p> + <p> + If you have changed the name of a file or deleted it + outside of the Berkeley DB library (for example, you + explicitly removed a file using your normal operating system + utilities), then it is possible that recovery will not be able + to find a database to which the log refers. In this case, the + <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility will produce a warning message, saying it was + unable to locate a file it expected to find. This message is + only a warning because the file may have been subsequently + deleted as part of normal database operations before the + failure occurred, so is not necessarily a problem. + </p> <p> - The Berkeley DB API supports creating, removing and renaming files. - Creating files is supported by the <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> method. Removing files is - supported by the <a href="../api_reference/C/envdbremove.html" class="olink">DB_ENV->dbremove()</a> and <a href="../api_reference/C/dbremove.html" class="olink">DB->remove()</a> methods. Renaming files - is supported by the <a href="../api_reference/C/envdbrename.html" class="olink">DB_ENV->dbrename()</a> and <a href="../api_reference/C/dbrename.html" class="olink">DB->rename()</a> methods. (There are - two methods for removing and renaming files because one of the methods - is transactionally protected and one is not.) -</p> - <p> - Berkeley DB does not permit specifying the <a href="../api_reference/C/dbopen.html#open_DB_TRUNCATE" class="olink">DB_TRUNCATE</a> flag when - opening a file in a transaction-protected environment. This is an - implicit file deletion, but one that does not always require the same - operating system file permissions as deleting and creating a file do. -</p> - <p> - If you have changed the name of a file or deleted it outside of the - Berkeley DB library (for example, you explicitly removed a file using - your normal operating system utilities), then it is possible that - recovery will not be able to find a database to which the log refers. - In this case, the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility will produce a warning message, saying - it was unable to locate a file it expected to find. This message is - only a warning because the file may have been subsequently deleted as - part of normal database operations before the failure occurred, so is - not necessarily a problem. -</p> - <p> - Generally, any filesystem operations that are performed outside the - Berkeley DB interface should be performed at the same time as making a - snapshot of the database. To perform filesystem operations correctly, - do the following: -</p> + Generally, any filesystem operations that are performed + outside the Berkeley DB interface should be performed at the + same time as making a snapshot of the database. To perform + filesystem operations correctly, do the following: + </p> <div class="orderedlist"> <ol type="1"> <li> - <p>Cleanly shut down database operations.</p> - <p>To shut down database operations cleanly, all applications accessing -the database environment must be shut down and a transaction checkpoint -must be taken. If the applications are not implemented so they can be -shut down gracefully (that is, closing all references to the database -environment), recovery must be performed after all applications have -been killed to ensure that the underlying databases are consistent on -disk.</p> + <p> + Cleanly shut down database operations. + </p> + <p> + To shut down database operations cleanly, all + applications accessing the database environment must + be shut down and a transaction checkpoint must be + taken. If the applications are not implemented so they + can be shut down gracefully (that is, closing all + references to the database environment), recovery must + be performed after all applications have been killed + to ensure that the underlying databases are consistent + on disk. + </p> </li> - <li>Perform the filesystem operations; for example, remove or rename one or -more files.</li> <li> - <p>Make an archival snapshot of the database.</p> - <p>Although this step is not strictly necessary, it is strongly -recommended. If this step is not performed, recovery from catastrophic -failure will require that recovery first be performed up to the time of -the filesystem operations, the filesystem operations be redone, and then -recovery be performed from the filesystem operations forward.</p> + Perform the filesystem operations; for example, + remove or rename one or more files. + </li> + <li> + <p> + Make an archival snapshot of the database. + </p> + <p> + Although this step is not strictly necessary, it is + strongly recommended. If this step is not performed, + recovery from catastrophic failure will require that + recovery first be performed up to the time of the + filesystem operations, the filesystem operations be + redone, and then recovery be performed from the + filesystem operations forward. + </p> </li> - <li>Restart the database applications.</li> + <li> + Restart the database applications. + </li> </ol> </div> </div> diff --git a/docs/programmer_reference/transapp_hotfail.html b/docs/programmer_reference/transapp_hotfail.html index 6d4143dd..512a91a3 100644 --- a/docs/programmer_reference/transapp_hotfail.html +++ b/docs/programmer_reference/transapp_hotfail.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_recovery.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_journal.html">Next</a></td> </tr> </table> @@ -39,142 +37,150 @@ </div> </div> <p> - For some applications, it may be useful to periodically snapshot - the database environment for use as a hot failover should the - primary system fail. The following steps can be taken to keep a - backup environment in close synchrony with an active environment. - The active environment is entirely unaffected by these procedures, - and both read and write operations are allowed during all steps - described here. + For some applications, it may be useful to periodically + snapshot the database environment for use as a hot failover + should the primary system fail. The following steps can be + taken to keep a backup environment in close synchrony with an + active environment. The active environment is entirely + unaffected by these procedures, and both read and write + operations are allowed during all steps described here. </p> - <p> - The procedure described here is not compatible with the concurrent - use of the transactional bulk insert optimization (transactions - started with the <a href="../api_reference/C/txnbegin.html#txnbegin_DB_TXN_BULK" class="olink">DB_TXN_BULK</a> flag). After the bulk optimization - is used, the archive must be created again from scratch starting - with step 1. + <p> + The procedure described here is not compatible with the + concurrent use of the transactional bulk insert optimization + (transactions started with the <a href="../api_reference/C/txnbegin.html#txnbegin_DB_TXN_BULK" class="olink">DB_TXN_BULK</a> flag). After the + bulk optimization is used, the archive must be created again + from scratch starting with step 1. </p> <p> - The <a href="../api_reference/C/db_hotbackup.html" class="olink">db_hotbackup</a> utility is the preferred way to automate generating a hot - failover system. The first step is to run <a href="../api_reference/C/db_hotbackup.html" class="olink">db_hotbackup</a> utility without the - <span class="bold"><strong>-u</strong></span> flag. This will create hot - backup copy of the databases in your environment. After that point - periodically running the <a href="../api_reference/C/db_hotbackup.html" class="olink">db_hotbackup</a> utility with the - <span class="bold"><strong>-u</strong></span> flag will copy the new - log files and run recovery on the backup copy to bring it current - with the primary environment. + The <a href="../api_reference/C/db_hotbackup.html" class="olink">db_hotbackup</a> utility is the preferred way to automate + generating a hot failover system. The first step is to run + <a href="../api_reference/C/db_hotbackup.html" class="olink">db_hotbackup</a> utility without the <span class="bold"><strong>-u</strong></span> + flag. This will create hot backup copy of the databases in + your environment. After that point periodically running the + <a href="../api_reference/C/db_hotbackup.html" class="olink">db_hotbackup</a> utility with the <span class="bold"><strong>-u</strong></span> + flag will copy the new log files and run recovery on the + backup copy to bring it current with the primary environment. </p> - <p> - Note that you can also create your own hot backup solution using - the <a href="../api_reference/C/envbackup.html" class="olink">DB_ENV->backup()</a> or <a href="../api_reference/C/envdbbackup.html" class="olink">DB_ENV->dbbackup()</a> methods. + <p> + Note that you can also create your own hot backup solution + using the <a href="../api_reference/C/envbackup.html" class="olink">DB_ENV->backup()</a> or <a href="../api_reference/C/envdbbackup.html" class="olink">DB_ENV->dbbackup()</a> methods. </p> - <p> - To implement your own hot fail over system, the steps below can be - followed. However, care should be taken on non-UNIX based systems - when copying the database files to be sure that they are either - quiescent, or that either the <a href="../api_reference/C/envbackup.html" class="olink">DB_ENV->backup()</a> or <a href="../api_reference/C/db_copy.html" class="olink">db_copy()</a> routine is - used to ensure atomic reads of the database pages. + <p> + To implement your own hot fail over system, the steps below + can be followed. However, care should be taken on non-UNIX + based systems when copying the database files to be sure that + they are either quiescent, or that either the <a href="../api_reference/C/envbackup.html" class="olink">DB_ENV->backup()</a> or + <a href="../api_reference/C/db_copy.html" class="olink">db_copy()</a> routine is used to ensure atomic reads of the + database pages. </p> <div class="orderedlist"> <ol type="1"> <li> - <p> - Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility with the <span class="bold"><strong>-s</strong></span> option in the active environment - to identify all of the active environment's database files, and - copy them to the backup directory. + <p> + Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility with the <span class="bold"><strong>-s</strong></span> + option in the active environment to + identify all of the active environment's database + files, and copy them to the backup directory. </p> - <p> - If the database files are stored in a separate directory from - the other Berkeley DB files, it will be simpler (and much - faster!) to copy the directory itself instead of the individual - files (see <a href="../api_reference/C/envadd_data_dir.html" class="olink">DB_ENV->add_data_dir()</a> for additional information). + <p> + If the database files are stored in a separate + directory from the other Berkeley DB files, it will be + simpler (and much faster!) to copy the directory + itself instead of the individual files (see + <a href="../api_reference/C/envadd_data_dir.html" class="olink">DB_ENV->add_data_dir()</a> for additional information). </p> <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> <h3 class="title">Note</h3> <p> - If any of the database files did not have an open <a href="../api_reference/C/db.html" class="olink">DB</a> - handle during the lifetime of the current log files, - the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility will not list them in its output. This - is another reason it may be simpler to use a separate - database file directory and copy the entire directory - instead of archiving only the files listed by the + If any of the database files did not have an + open <a href="../api_reference/C/db.html" class="olink">DB</a> handle during the lifetime of the + current log files, the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility will not list + them in its output. This is another reason it may + be simpler to use a separate database file + directory and copy the entire directory instead of + archiving only the files listed by the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility. </p> </div> </li> <li> - Remove all existing log files from the backup directory. + Remove all existing log files from the backup + directory. </li> - <li> - Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility with the <span class="bold"><strong>-l</strong></span> - option in the active environment to identify all of the active - environment's log files, and copy them to the backup directory. + <li> + Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility with the <span class="bold"><strong>-l</strong></span> + option in the active environment to + identify all of the active environment's log files, and + copy them to the backup directory. </li> <li> - Run the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility with the <span class="bold"><strong>-c</strong></span> - option in the backup directory to catastrophically recover the - copied environment. + Run the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility with the <span class="bold"><strong>-c</strong></span> option + in the backup directory to catastrophically recover the copied environment. </li> </ol> </div> <p> - Steps 2, 3 and 4 may be repeated as often as you like. If Step 1 - (the initial copy of the database files) is repeated, then Steps 2, - 3 and 4 <span class="bold"><strong>must</strong></span> be performed at least - once in order to ensure a consistent database environment - snapshot. + Steps 2, 3 and 4 may be repeated as often as you like. If + Step 1 (the initial copy of the database files) is repeated, + then Steps 2, 3 and 4 <span class="bold"><strong>must</strong></span> be + performed at least once in order to ensure a consistent + database environment snapshot. </p> - <p> - These procedures must be integrated with your other archival - procedures, of course. If you are periodically removing log files - from your active environment, you must be sure to copy them to the - backup directory before removing them from the active directory. - Not copying a log file to the backup directory and subsequently - running recovery with it present may leave the backup snapshot of - the environment corrupted. A simple way to ensure this never - happens is to archive the log files in Step 2 as you remove them - from the backup directory, and move inactive log files from your - active environment into your backup directory (rather than copying - them), in Step 3. The following steps describe this procedure in - more detail: + <p> + These procedures must be integrated with your other + archival procedures, of course. If you are periodically + removing log files from your active environment, you must be + sure to copy them to the backup directory before removing them + from the active directory. Not copying a log file to the + backup directory and subsequently running recovery with it + present may leave the backup snapshot of the environment + corrupted. A simple way to ensure this never happens is to + archive the log files in Step 2 as you remove them from the + backup directory, and move inactive log files from your active + environment into your backup directory (rather than copying + them), in Step 3. The following steps describe this procedure + in more detail: </p> <div class="orderedlist"> <ol type="1"> <li> - Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility with the <span class="bold"><strong>-s</strong></span> - option in the active environment to identify all of the active - environment's database files, and copy them to the backup - directory. + Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility with the <span class="bold"><strong>-s</strong></span> option + in the active environment to identify all of the active environment's database files, + and copy them to the backup directory. </li> - <li> - Archive all existing log files from the backup directory, moving them - to a backup device such as CD-ROM, alternate disk, or tape. + <li> + Archive all existing log files from the backup + directory, moving them to a backup device such as CD-ROM, + alternate disk, or tape. </li> <li> - Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility (without any option) in the active environment - to identify all of the log files in the active environment that are - no longer in use, and <span class="bold"><strong>move</strong></span> them to - the backup directory. + Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility (without any option) in the + active environment to identify all of the log files in the + active environment that are no longer in use, and + <span class="bold"><strong>move</strong></span> them to the + backup directory. </li> - <li> - Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility with the <span class="bold"><strong>-l</strong></span> - option in the active environment to identify all of the remaining - log files in the active environment, and <span class="bold"><strong>copy</strong></span> the log files to the backup - directory. + <li> + Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility with the <span class="bold"><strong>-l</strong></span> option + in the active environment to + identify all of the remaining log files in the active + environment, and <span class="bold"><strong>copy</strong></span> the + log files to the backup directory. </li> <li> - Run the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility with the <span class="bold"><strong>-c</strong></span> - option in the backup directory to catastrophically recover the - copied environment. + Run the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility with the <span class="bold"><strong>-c</strong></span> option + in the backup directory to + catastrophically recover the copied environment. </li> </ol> </div> - <p> - As before, steps 2, 3, 4 and 5 may be repeated as often as you - like. If Step 1 (the initial copy of the database files) is - repeated, then Steps 2 through 5 - <span class="bold"><strong>must</strong></span> be performed at least once in - order to ensure a consistent database environment snapshot. + <p> + As before, steps 2, 3, 4 and 5 may be repeated as often as + you like. If Step 1 (the initial copy of the database files) + is repeated, then Steps 2 through 5 <span class="bold"><strong>must</strong></span> be + performed at least once in order to + ensure a consistent database environment snapshot. </p> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/transapp_inc.html b/docs/programmer_reference/transapp_inc.html index 92fbcdef..7ad5eeb3 100644 --- a/docs/programmer_reference/transapp_inc.html +++ b/docs/programmer_reference/transapp_inc.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_atomicity.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_read.html">Next</a></td> </tr> </table> @@ -38,34 +36,45 @@ </div> </div> </div> - <p>The third reason listed for using transactions was <span class="emphasis"><em>isolation</em></span>. -Consider an application suite in which multiple threads of control -(multiple processes or threads in one or more processes) are changing -the values associated with a key in one or more databases. Specifically, -they are taking the current value, incrementing it, and then storing it -back into the database.</p> - <p>Such an application requires isolation. Because we want to change a value -in the database, we must make sure that after we read it, no other thread -of control modifies it. For example, assume that both thread #1 and -thread #2 are doing similar operations in the database, where thread #1 -is incrementing records by 3, and thread #2 is incrementing records by -5. We want to increment the record by a total of 8. If the operations -interleave in the right (well, wrong) order, that is not what will -happen:</p> + <p> + The third reason listed for using transactions was + <span class="emphasis"><em>isolation</em></span>. Consider an application + suite in which multiple threads of control (multiple processes + or threads in one or more processes) are changing the values + associated with a key in one or more databases. Specifically, + they are taking the current value, incrementing it, and then + storing it back into the database. + </p> + <p> + Such an application requires isolation. Because we want to + change a value in the database, we must make sure that after + we read it, no other thread of control modifies it. For + example, assume that both thread #1 and thread #2 are doing + similar operations in the database, where thread #1 is + incrementing records by 3, and thread #2 is incrementing + records by 5. We want to increment the record by a total of 8. + If the operations interleave in the right (well, wrong) order, + that is not what will happen: + </p> <pre class="programlisting">thread #1 <span class="bold"><strong>read</strong></span> record: the value is 2 thread #2 <span class="bold"><strong>read</strong></span> record: the value is 2 thread #2 <span class="bold"><strong>write</strong></span> record + 5 back into the database (new value 7) thread #1 <span class="bold"><strong>write</strong></span> record + 3 back into the database (new value 5)</pre> - <p>As you can see, instead of incrementing the record by a total of 8, -we've incremented it only by 3 because thread #1 overwrote thread #2's -change. By wrapping the operations in transactions, we ensure that this -cannot happen. In a transaction, when the first thread reads the -record, locks are acquired that will not be released until the -transaction finishes, guaranteeing that all writers -will block, waiting for the first thread's transaction to complete (or -to be aborted).</p> - <p>Here is an example function that does transaction-protected increments -on database records to ensure isolation:</p> + <p> + As you can see, instead of incrementing the record by a + total of 8, we've incremented it only by 3 because thread #1 + overwrote thread #2's change. By wrapping the operations in + transactions, we ensure that this cannot happen. In a + transaction, when the first thread reads the record, locks are + acquired that will not be released until the transaction + finishes, guaranteeing that all writers will block, waiting + for the first thread's transaction to complete (or to be + aborted). + </p> + <p> + Here is an example function that does transaction-protected + increments on database records to ensure isolation: + </p> <pre class="programlisting">int main(int argc, char *argv) { @@ -180,12 +189,15 @@ add_color(DB_ENV *dbenv, DB *dbp, char *color, int increment) } } }</strong></span></pre> - <p>The <a href="../api_reference/C/dbcget.html#dbcget_DB_RMW" class="olink">DB_RMW</a> flag in the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> call specifies a write lock -should be acquired on the key/data pair, instead of the more obvious read -lock. We do this because the application expects to write the key/data -pair in a subsequent operation, and the transaction is much more likely to -deadlock if we first obtain a read lock and subsequently a write lock, than -if we obtain the write lock initially.</p> + <p> + The <a href="../api_reference/C/dbcget.html#dbcget_DB_RMW" class="olink">DB_RMW</a> flag in the <a href="../api_reference/C/dbget.html" class="olink">DB->get()</a> call specifies a write lock + should be acquired on the key/data pair, instead of the more + obvious read lock. We do this because the application expects + to write the key/data pair in a subsequent operation, and the + transaction is much more likely to deadlock if we first obtain + a read lock and subsequently a write lock, than if we obtain + the write lock initially. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/transapp_journal.html b/docs/programmer_reference/transapp_journal.html index 760144a2..2576043d 100644 --- a/docs/programmer_reference/transapp_journal.html +++ b/docs/programmer_reference/transapp_journal.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_hotfail.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_filesys.html">Next</a></td> </tr> </table> @@ -39,19 +37,21 @@ </div> </div> <p> - In some cases, the use of meta-data only journaling file systems can - lead to log file corruption. The window of vulnerability is quite - small, but if the operating system experiences an unclean shutdown - while Berkeley DB is creating a new log file, it is possible that upon - file system recovery, the system will be in a state where the log file - has been created, but its own meta-data has not. -</p> + In some cases, the use of meta-data only journaling file + systems can lead to log file corruption. The window of + vulnerability is quite small, but if the operating system + experiences an unclean shutdown while Berkeley DB is creating + a new log file, it is possible that upon file system recovery, + the system will be in a state where the log file has been + created, but its own meta-data has not. + </p> <p> - When a log file is corrupted to this degree, normal recovery can fail - and your application may be unable to open your environment. Instead, - an error something like this is issued when you attempt to run normal - recovery on environment open: -</p> + When a log file is corrupted to this degree, normal + recovery can fail and your application may be unable to open + your environment. Instead, an error something like this is + issued when you attempt to run normal recovery on environment + open: + </p> <pre class="programlisting"> Ignoring log file: /var/dblog/log.0000000074: magic number 6c73732f, not 40988 Invalid log file: log.0000000074: Invalid argument @@ -59,32 +59,35 @@ process-private: unable to find environment txn_checkpoint interface requires an environment configured for the transaction subsystem </pre> - <p> - In this case, it may be possible to successfully recover the - environment by ignoring the log file that was being created — to - do this, rename the log file with the highest number to a temporary - name: -</p> + <p> + In this case, it may be possible to successfully recover + the environment by ignoring the log file that was being + created — to do this, rename the log file with the + highest number to a temporary name: + </p> <pre class="programlisting"> mv DBHOME/log.000000XXX my-temporary-log-file </pre> + <p> + and try running normal environment recovery again. If + recovery is successful, and your application is able to open + the environment, then you can delete the log file that you + renamed. + </p> + <p> + If recovery is not successful, then you must perform a + catastrophic recovery from a previous backup. + </p> + <p> + This situation has been shown to occur when using ext3 in + writeback mode, but other journaling filesystems could exhibit + similar behavior. + </p> <p> - and try running normal environment recovery again. If recovery is - successful, and your application is able to open the environment, then - you can delete the log file that you renamed. -</p> - <p> - If recovery is not successful, then you must perform a catastrophic - recovery from a previous backup. -</p> - <p> - This situation has been shown to occur when using ext3 in writeback - mode, but other journaling filesystems could exhibit similar behavior. -</p> - <p> - To be absolutely certain of your application's ability to recover your - environment in the event of a system crash, either use non-journaling - filesystems, or use a journaling filesystem in a safe (albeit slower) - configuration, such as ext3 in ordered mode. -</p> + To be absolutely certain of your application's ability to + recover your environment in the event of a system crash, + either use non-journaling filesystems, or use a journaling + filesystem in a safe (albeit slower) configuration, such as + ext3 in ordered mode. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/transapp_logfile.html b/docs/programmer_reference/transapp_logfile.html index fd130e3f..f44aebaa 100644 --- a/docs/programmer_reference/transapp_logfile.html +++ b/docs/programmer_reference/transapp_logfile.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_archival.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_recovery.html">Next</a></td> </tr> </table> @@ -38,46 +36,77 @@ </div> </div> </div> - <p>The fourth component of the infrastructure, log file removal, concerns -the ongoing disk consumption of the database log files. Depending on -the rate at which the application writes to the databases and the -available disk space, the number of log files may increase quickly -enough so that disk space will be a resource problem. For this reason, -you will periodically want to remove log files in order to conserve disk -space. This procedure is distinct from database and log file archival -for catastrophic recovery, and you cannot remove the current log files -simply because you have created a database snapshot or copied log files -to archival media.</p> - <p>Log files may be removed at any time, as long as:</p> + <p> + The fourth component of the infrastructure, log file + removal, concerns the ongoing disk consumption of the database + log files. Depending on the rate at which the application + writes to the databases and the available disk space, the + number of log files may increase quickly enough so that disk + space will be a resource problem. For this reason, you will + periodically want to remove log files in order to conserve + disk space. This procedure is distinct from database and log + file archival for catastrophic recovery, and you cannot remove + the current log files simply because you have created a + database snapshot or copied log files to archival + media. + </p> + <p> + Log files may be removed at any time, as long as: + </p> <div class="itemizedlist"> <ul type="disc"> - <li>the log file is not involved in an active transaction.</li> - <li>a checkpoint has been written subsequent to the log file's -creation.</li> - <li>the log file is not the only log file in the environment.</li> + <li> + the log file is not involved in an active + transaction. + </li> + <li> + a checkpoint has been written subsequent to the log + file's creation. + </li> + <li> + the log file is not the only log file in the + environment. + </li> </ul> </div> - <p>Additionally, when Replication Manager is running -the log file is older than the most out of date active site in the -replication group.</p> - <p>If you are preparing for catastrophic failure, you will want to copy -the log files to archival media before you remove them as described in -<a class="xref" href="transapp_archival.html" title="Database and log file archival">Database and log file archival</a>.</p> - <p>If you are not preparing for catastrophic failure, any one of the -following methods can be used to remove log files:</p> + <p> + Additionally, when Replication Manager is running the log + file is older than the most out of date active site in the + replication group. + </p> + <p> + If you are preparing for catastrophic failure, you will want + to copy the log files to archival media before you remove them + as described in <a class="xref" href="transapp_archival.html" title="Database and log file archival">Database and log file + archival</a>. + </p> + <p> + If you are not preparing for catastrophic failure, any one + of the following methods can be used to remove log + files: + </p> <div class="orderedlist"> <ol type="1"> - <li>Run the standalone <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility with the <span class="bold"><strong>-d</strong></span> -option, to remove any log files that are no longer needed at the time -the command is executed.</li> - <li>Call the <a href="../api_reference/C/logarchive.html" class="olink">DB_ENV->log_archive()</a> method from the application, with the -<a href="../api_reference/C/logarchive.html#archive_DB_ARCH_REMOVE" class="olink">DB_ARCH_REMOVE</a> flag, to remove any log files that are no longer -needed at the time the call is made.</li> - <li>Call the <a href="../api_reference/C/envlog_set_config.html" class="olink">DB_ENV->log_set_config()</a> method from the application, with the -<a href="../api_reference/C/envlog_set_config.html#log_set_config_DB_LOG_AUTO_REMOVE" class="olink">DB_LOG_AUTO_REMOVE</a> flag, to remove any log files that are no -longer needed on an ongoing basis. With this configuration, Berkeley DB will -automatically remove log files, and the application will not have an -opportunity to copy the log files to backup media.</li> + <li> + Run the standalone <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility with the <span class="bold"><strong>-d</strong></span> option, to remove any log + files that are no longer needed at the time the command is + executed. + </li> + <li> + Call the <a href="../api_reference/C/logarchive.html" class="olink">DB_ENV->log_archive()</a> method from the application, + with the <a href="../api_reference/C/logarchive.html#archive_DB_ARCH_REMOVE" class="olink">DB_ARCH_REMOVE</a> flag, to remove any log files + that are no longer needed at the time the call is + made. + </li> + <li> + Call the <a href="../api_reference/C/envlog_set_config.html" class="olink">DB_ENV->log_set_config()</a> method from the + application, with the <a href="../api_reference/C/envlog_set_config.html#log_set_config_DB_LOG_AUTO_REMOVE" class="olink">DB_LOG_AUTO_REMOVE</a> flag, to remove + any log files that are no longer needed on an ongoing + basis. With this configuration, Berkeley DB will + automatically remove log files, and the application will + not have an opportunity to copy the log files to backup + media. + </li> </ol> </div> </div> @@ -92,7 +121,8 @@ opportunity to copy the log files to backup media.</li> <td width="40%" align="right"> <a accesskey="n" href="transapp_recovery.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Database and log file archival </td> + <td width="40%" align="left" valign="top">Database and log file + archival </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/transapp_nested.html b/docs/programmer_reference/transapp_nested.html index 0312ae47..61765982 100644 --- a/docs/programmer_reference/transapp_nested.html +++ b/docs/programmer_reference/transapp_nested.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_cursor.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_admin.html">Next</a></td> </tr> </table> @@ -38,48 +36,67 @@ </div> </div> </div> - <p>Berkeley DB provides support for nested transactions. Nested transactions -allow an application to decompose a large or long-running transaction -into smaller units that may be independently aborted.</p> - <p>Normally, when beginning a transaction, the application will pass a NULL -value for the parent argument to <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a>. If, however, the -parent argument is a <a href="../api_reference/C/txn.html" class="olink">TXN</a> handle, the newly created transaction -will be treated as a nested transaction within the parent. Transactions -may nest arbitrarily deeply. For the purposes of this discussion, -transactions created with a parent identifier will be called -<span class="emphasis"><em>child transactions</em></span>.</p> - <p>Once a transaction becomes a parent, as long as any of its child -transactions are unresolved (that is, they have neither committed nor -aborted), the parent may not issue any Berkeley DB calls except to begin more -child transactions, or to commit or abort. For example, it may not -issue any access method or cursor calls. After all of a parent's -children have committed or aborted, the parent may again request -operations on its own behalf.</p> - <p>The semantics of nested transactions are as follows. When a child -transaction is begun, it inherits all the locks of its parent. This -means that the child will never block waiting on a lock held by its -parent. Further, locks held by two children of the same parent will -also conflict. To make this concrete, consider the following set of -transactions and lock acquisitions.</p> - <p>Transaction T1 is the parent transaction. It acquires a write lock on -item A and then begins two child transactions: C1 and C2. C1 also wants -to acquire a write lock on A; this succeeds. If C2 attempts to acquire -a write lock on A, it will block until C1 releases the lock, at which -point it will succeed. Now, let's say that C1 acquires a write lock on -B. If C2 now attempts to obtain a lock on B, it will block. However, -let's now assume that C1 commits. Its locks are anti-inherited, which -means they are given to T1, so T1 will now hold a lock on B. At this -point, C2 would be unblocked and would then acquire a lock on B.</p> - <p>Child transactions are entirely subservient to their parent transaction. -They may abort, undoing their operations regardless of the eventual fate -of the parent. However, even if a child transaction commits, if its -parent transaction is eventually aborted, the child's changes are undone -and the child's transaction is effectively aborted. Any child -transactions that are not yet resolved when the parent commits or aborts -are resolved based on the parent's resolution -- committing if the -parent commits and aborting if the parent aborts. Any child -transactions that are not yet resolved when the parent prepares are also -prepared.</p> + <p> + Berkeley DB provides support for nested transactions. Nested + transactions allow an application to decompose a large or + long-running transaction into smaller units that may be + independently aborted. + </p> + <p> + Normally, when beginning a transaction, the application will + pass a NULL value for the parent argument to <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a>. If, + however, the parent argument is a <a href="../api_reference/C/txn.html" class="olink">TXN</a> handle, the newly + created transaction will be treated as a nested transaction + within the parent. Transactions may nest arbitrarily deeply. + For the purposes of this discussion, transactions created with + a parent identifier will be called <span class="emphasis"><em>child + transactions</em></span>. + </p> + <p> + Once a transaction becomes a parent, as long as any of its + child transactions are unresolved (that is, they have neither + committed nor aborted), the parent may not issue any Berkeley + DB calls except to begin more child transactions, or to commit + or abort. For example, it may not issue any access method or + cursor calls. After all of a parent's children have committed + or aborted, the parent may again request operations on its own + behalf. + </p> + <p> + The semantics of nested transactions are as follows. When a + child transaction is begun, it inherits all the locks of its + parent. This means that the child will never block waiting on + a lock held by its parent. Further, locks held by two children + of the same parent will also conflict. To make this concrete, + consider the following set of transactions and lock + acquisitions. + </p> + <p> + Transaction T1 is the parent transaction. It acquires a + write lock on item A and then begins two child transactions: + C1 and C2. C1 also wants to acquire a write lock on A; this + succeeds. If C2 attempts to acquire a write lock on A, it will + block until C1 releases the lock, at which point it will + succeed. Now, let's say that C1 acquires a write lock on B. If + C2 now attempts to obtain a lock on B, it will block. However, + let's now assume that C1 commits. Its locks are + anti-inherited, which means they are given to T1, so T1 will + now hold a lock on B. At this point, C2 would be unblocked and + would then acquire a lock on B. + </p> + <p> + Child transactions are entirely subservient to their parent + transaction. They may abort, undoing their operations + regardless of the eventual fate of the parent. However, even + if a child transaction commits, if its parent transaction is + eventually aborted, the child's changes are undone and the + child's transaction is effectively aborted. Any child + transactions that are not yet resolved when the parent commits + or aborts are resolved based on the parent's resolution -- + committing if the parent commits and aborting if the parent + aborts. Any child transactions that are not yet resolved when + the parent prepares are also prepared. + </p> </div> <div class="navfooter"> <hr /> @@ -96,7 +113,8 @@ prepared.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Environment infrastructure</td> + <td width="40%" align="right" valign="top"> Environment + infrastructure</td> </tr> </table> </div> diff --git a/docs/programmer_reference/transapp_put.html b/docs/programmer_reference/transapp_put.html index 289e1746..9f042e07 100644 --- a/docs/programmer_reference/transapp_put.html +++ b/docs/programmer_reference/transapp_put.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_data_open.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_atomicity.html">Next</a></td> </tr> </table> @@ -38,32 +36,46 @@ </div> </div> </div> - <p>The first reason listed for using transactions was recoverability. Any -logical change to a database may require multiple changes to underlying -data structures. For example, modifying a record in a Btree may require -leaf and internal pages to split, so a single <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> method -call can potentially require that multiple physical database pages be -written. If only some of those pages are written and then the system -or application fails, the database is left inconsistent and cannot be -used until it has been recovered; that is, until the partially completed -changes have been undone.</p> - <p><span class="emphasis"><em>Write-ahead-logging</em></span> is the term that describes the underlying -implementation that Berkeley DB uses to ensure recoverability. What it means -is that before any change is made to a database, information about the -change is written to a database log. During recovery, the log is read, -and databases are checked to ensure that changes described in the log -for committed transactions appear in the database. Changes that appear -in the database but are related to aborted or unfinished transactions -in the log are undone from the database.</p> - <p>For recoverability after application or system failure, operations that -modify the database must be protected by transactions. More -specifically, operations are not recoverable unless a transaction is -begun and each operation is associated with the transaction via the -Berkeley DB interfaces, and then the transaction successfully committed. This -is true even if logging is turned on in the database environment.</p> - <p>Here is an example function that updates a record in a database in a -transactionally protected manner. The function takes a key and data -items as arguments and then attempts to store them into the database.</p> + <p> + The first reason listed for using transactions was + recoverability. Any logical change to a database may require + multiple changes to underlying data structures. For example, + modifying a record in a Btree may require leaf and internal + pages to split, so a single <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> method call can + potentially require that multiple physical database pages be + written. If only some of those pages are written and then the + system or application fails, the database is left inconsistent + and cannot be used until it has been recovered; that is, until + the partially completed changes have been undone. + </p> + <p> + <span class="emphasis"><em>Write-ahead-logging</em></span> is the term that + describes the underlying implementation that Berkeley DB uses + to ensure recoverability. What it means is that before any + change is made to a database, information about the change is + written to a database log. During recovery, the log is read, + and databases are checked to ensure that changes described in + the log for committed transactions appear in the database. + Changes that appear in the database but are related to aborted + or unfinished transactions in the log are undone from the + database. + </p> + <p> + For recoverability after application or system failure, + operations that modify the database must be protected by + transactions. More specifically, operations are not + recoverable unless a transaction is begun and each operation + is associated with the transaction via the Berkeley DB + interfaces, and then the transaction successfully committed. + This is true even if logging is turned on in the database + environment. + </p> + <p> + Here is an example function that updates a record in a + database in a transactionally protected manner. The function + takes a key and data items as arguments and then attempts to + store them into the database. + </p> <pre class="programlisting">int main(int argc, char *argv) { @@ -145,91 +157,111 @@ add_fruit(DB_ENV *dbenv, DB *db, char *fruit, char *name) } } }</strong></span></pre> - <p>Berkeley DB also uses transactions to recover from deadlock. Database -operations (that is, any call to a function underlying the handles -returned by <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> and <a href="../api_reference/C/dbcursor.html" class="olink">DB->cursor()</a>) are usually -performed on behalf of a unique locker. Transactions can be used to -perform multiple calls on behalf of the same locker within a single -thread of control. For example, consider the case in which an -application uses a cursor scan to locate a record and then the -application accesses another other item in the database, based on the -key returned by the cursor, without first closing the cursor. If these -operations are done using default locker IDs, they may conflict. If the -locks are obtained on behalf of a transaction, using the transaction's -locker ID instead of the database handle's locker ID, the operations -will not conflict.</p> - <p>There is a new error return in this function that you may not have seen -before. In transactional (not Concurrent Data Store) applications -supporting both readers and writers, or just multiple writers, Berkeley DB -functions have an additional possible error return: -<a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a>. This means two threads of control deadlocked, -and the thread receiving the <code class="literal">DB_LOCK_DEADLOCK</code> error return has -been selected to discard its locks in order to resolve the problem. -When an application receives a <code class="literal">DB_LOCK_DEADLOCK</code> return, the -correct action is to close any cursors involved in the operation and -abort any enclosing transaction. In the sample code, any time the -<a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> method returns <code class="literal">DB_LOCK_DEADLOCK</code>, <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> is -called (which releases the transaction's Berkeley DB resources and undoes any -partial changes to the databases), and then the transaction is retried -from the beginning.</p> - <p>There is no requirement that the transaction be attempted again, but -that is a common course of action for applications. Applications may -want to set an upper bound on the number of times an operation will be -retried because some operations on some data sets may simply be unable -to succeed. For example, updating all of the pages on a large Web site -during prime business hours may simply be impossible because of the high -access rate to the database.</p> - <p>The <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> method is called in error cases other than deadlock. -Any time an error occurs, such that a transactionally protected set of -operations cannot complete successfully, the transaction must be -aborted. While deadlock is by far the most common of these errors, -there are other possibilities; for example, running out of disk space -for the filesystem. In Berkeley DB transactional applications, there are -three classes of error returns: "expected" errors, "unexpected but -recoverable" errors, and a single "unrecoverable" error. Expected -errors are errors like -<a class="link" href="program_errorret.html#program_errorret.DB_NOTFOUND">DB_NOTFOUND</a>, -which indicates that a -searched-for key item is not present in the database. Applications may -want to explicitly test for and handle this error, or, in the case where -the absence of a key implies the enclosing transaction should fail, -simply call <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a>. Unexpected but recoverable errors are -errors like -<a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a>, -which indicates that an operation -has been selected to resolve a deadlock, or a system error such as EIO, -which likely indicates that the filesystem has no available disk space. -Applications must immediately call <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> when these returns -occur, as it is not possible to proceed otherwise. The only -unrecoverable error is -<a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, -which indicates that the -system must stop and recovery must be run.</p> - <p>The above code can be simplified in the case of a transaction comprised -entirely of a single database put or delete operation, as operations -occurring in transactional databases are implicitly transaction -protected. For example, in a transactional database, the above code -could be more simply written as:</p> + <p> + Berkeley DB also uses transactions to recover from deadlock. + Database operations (that is, any call to a function + underlying the handles returned by <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> and <a href="../api_reference/C/dbcursor.html" class="olink">DB->cursor()</a>) + are usually performed on behalf of a unique locker. + Transactions can be used to perform multiple calls on behalf + of the same locker within a single thread of control. For + example, consider the case in which an application uses a + cursor scan to locate a record and then the application + accesses another other item in the database, based on the key + returned by the cursor, without first closing the cursor. If + these operations are done using default locker IDs, they may + conflict. If the locks are obtained on behalf of a + transaction, using the transaction's locker ID instead of the + database handle's locker ID, the operations will not + conflict. + </p> + <p> + There is a new error return in this function that you may + not have seen before. In transactional (not Concurrent Data + Store) applications supporting both readers and writers, or + just multiple writers, Berkeley DB functions have an + additional possible error return: <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK"> + DB_LOCK_DEADLOCK</a>. This means two threads of + control deadlocked, and the thread receiving the + <code class="literal">DB_LOCK_DEADLOCK</code> error return has been + selected to discard its locks in order to resolve the problem. + When an application receives a + <code class="literal">DB_LOCK_DEADLOCK</code> return, the correct + action is to close any cursors involved in the operation and + abort any enclosing transaction. In the sample code, any time + the <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> method returns + <code class="literal">DB_LOCK_DEADLOCK</code>, <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> is called + (which releases the transaction's Berkeley DB resources and + undoes any partial changes to the databases), and then the + transaction is retried from the beginning. + </p> + <p> + There is no requirement that the transaction be attempted + again, but that is a common course of action for applications. + Applications may want to set an upper bound on the number of + times an operation will be retried because some operations on + some data sets may simply be unable to succeed. For example, + updating all of the pages on a large Web site during prime + business hours may simply be impossible because of the high + access rate to the database. + </p> + <p> + The <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> method is called in error cases other than + deadlock. Any time an error occurs, such that a + transactionally protected set of operations cannot complete + successfully, the transaction must be aborted. While deadlock + is by far the most common of these errors, there are other + possibilities; for example, running out of disk space for the + filesystem. In Berkeley DB transactional applications, there + are three classes of error returns: "expected" errors, + "unexpected but recoverable" errors, and a single + "unrecoverable" error. Expected errors are errors like <a class="link" href="program_errorret.html#program_errorret.DB_NOTFOUND">DB_NOTFOUND</a>, + which indicates that a searched-for key item is not present in + the database. Applications may want to explicitly test for and + handle this error, or, in the case where the absence of a key + implies the enclosing transaction should fail, simply call + <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a>. Unexpected but recoverable errors are errors like + <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a>, + which indicates that an operation has been selected to resolve a + deadlock, or a system error such as EIO, which likely indicates + that the filesystem has no available disk space. Applications must + immediately call <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> when these returns occur, as it is not + possible to proceed otherwise. The only unrecoverable error is + <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a>, + which indicates that the system + must stop and recovery must be run. + </p> + <p> + The above code can be simplified in the case of a + transaction comprised entirely of a single database put or + delete operation, as operations occurring in transactional + databases are implicitly transaction protected. For example, + in a transactional database, the above code could be more + simply written as: + </p> <pre class="programlisting"> <strong class="userinput"><code>for (fail = 0; fail++ <= MAXIMUM_RETRY && (ret = db->put(db, NULL, &key, &data, 0)) == DB_LOCK_DEADLOCK;) continue; return (ret == 0 ? 0 : 1);</code></strong></pre> - <p>and the underlying transaction would be automatically handled by Berkeley DB.</p> - <p>Programmers should not attempt to enumerate all possible error returns -in their software. Instead, they should explicitly handle expected -returns and default to aborting the transaction for the rest. It is -entirely the choice of the programmer whether to check for -<a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a> -explicitly or not — attempting new Berkeley DB -operations after -<a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a> -is returned does not worsen the -situation. Alternatively, using the <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> method to -handle an unrecoverable error and simply doing some number of -abort-and-retry cycles for any unexpected Berkeley DB or system error in the -mainline code often results in the simplest and cleanest application -code.</p> + <p> + and the underlying transaction would be automatically + handled by Berkeley DB. + </p> + <p> + Programmers should not attempt to enumerate all possible + error returns in their software. Instead, they should + explicitly handle expected returns and default to aborting the + transaction for the rest. It is entirely the choice of the + programmer whether to check for <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY"> + DB_RUNRECOVERY</a> explicitly or not — + attempting new Berkeley DB operations after <a class="link" href="program_errorret.html#program_errorret.DB_RUNRECOVERY">DB_RUNRECOVERY</a> + is returned does not worsen the + situation. Alternatively, using the <a href="../api_reference/C/envevent_notify.html" class="olink">DB_ENV->set_event_notify()</a> method + to handle an unrecoverable error and simply doing some number + of abort-and-retry cycles for any unexpected Berkeley DB or + system error in the mainline code often results in the + simplest and cleanest application code. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/transapp_read.html b/docs/programmer_reference/transapp_read.html index 03b42257..9c148a57 100644 --- a/docs/programmer_reference/transapp_read.html +++ b/docs/programmer_reference/transapp_read.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_inc.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_cursor.html">Next</a></td> </tr> </table> @@ -47,55 +45,71 @@ </dt> </dl> </div> - <p>Transactions can be isolated from each other to different degrees. -<span class="emphasis"><em>Serializable</em></span> provides the most isolation, and means that, for -the life of the transaction, every time a thread of control reads a data -item, it will be unchanged from its previous value (assuming, of course, -the thread of control does not itself modify the item). By default, -Berkeley DB enforces serializability whenever database reads are wrapped in -transactions. This is also known as <span class="emphasis"><em>degree 3 isolation</em></span>.</p> - <p>Most applications do not need to enclose all reads in transactions, and -when possible, transactionally protected reads at serializable isolation -should be avoided as they can cause performance problems. For example, -a serializable cursor sequentially reading each key/data pair in a -database, will acquire a read lock on most of the pages in the database -and so will gradually block all write operations on the databases until -the transaction commits or aborts. Note, however, that if there are -update transactions present in the application, the read operations must -still use locking, and must be prepared to repeat any operation -(possibly closing and reopening a cursor) that fails with a return value -of <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a>. -Applications that need repeatable reads -are ones that require the ability to repeatedly access a data item -knowing that it will not have changed (for example, an operation -modifying a data item based on its existing value).</p> - <p><span class="emphasis"><em>Snapshot isolation</em></span> also guarantees repeatable reads, but -avoids read locks by using multiversion concurrency control (MVCC). -This makes update operations more expensive, because they have to -allocate space for new versions of pages in cache and make copies, but -avoiding read locks can significantly increase throughput for many -applications. Snapshot isolation is discussed in detail below.</p> - <p>A transaction may only require -<span class="emphasis"><em>cursor stability</em></span>, that is only -be guaranteed that cursors see committed data that does not change so -long as it is addressed by the cursor, but may change before the reading -transaction completes. This is also called <span class="emphasis"><em>degree 2 -isolation</em></span>. Berkeley DB provides this level of isolation when a transaction -is started with the <a href="../api_reference/C/dbcget.html#dbcget_DB_READ_COMMITTED" class="olink">DB_READ_COMMITTED</a> flag. This flag may also -be specified when opening a cursor within a fully isolated -transaction.</p> - <p>Berkeley DB optionally supports reading uncommitted data; that is, read -operations may request data which has been modified but not yet -committed by another transaction. This is also called <span class="emphasis"><em>degree -1 isolation</em></span>. This is done by first specifying the -<a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a> flag when opening the underlying database, -and then specifying the <a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a> flag when beginning -a transaction, opening a cursor, or performing a read operation. The -advantage of using <a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a> is that read operations -will not block when another transaction holds a write lock on the -requested data; the disadvantage is that read operations may return data -that will disappear should the transaction holding the write lock -abort.</p> + <p> + Transactions can be isolated from each other to different + degrees. <span class="emphasis"><em>Serializable</em></span> provides the most + isolation, and means that, for the life of the transaction, + every time a thread of control reads a data item, it will be + unchanged from its previous value (assuming, of course, the + thread of control does not itself modify the item). By + default, Berkeley DB enforces serializability whenever + database reads are wrapped in transactions. This is also known + as <span class="emphasis"><em>degree 3 isolation</em></span>. + </p> + <p> + Most applications do not need to enclose all reads in + transactions, and when possible, transactionally protected + reads at serializable isolation should be avoided as they can + cause performance problems. For example, a serializable cursor + sequentially reading each key/data pair in a database, will + acquire a read lock on most of the pages in the database and + so will gradually block all write operations on the databases + until the transaction commits or aborts. Note, however, that + if there are update transactions present in the application, + the read operations must still use locking, and must be + prepared to repeat any operation (possibly closing and + reopening a cursor) that fails with a return value of <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a>. + Applications that need repeatable reads are ones that require + the ability to repeatedly access a data item knowing that it will + not have changed (for example, an operation modifying a data item based + on its existing value). + </p> + <p> + <span class="emphasis"><em>Snapshot isolation</em></span> also guarantees + repeatable reads, but avoids read locks by using multiversion + concurrency control (MVCC). This makes update operations more + expensive, because they have to allocate space for new + versions of pages in cache and make copies, but avoiding read + locks can significantly increase throughput for many + applications. Snapshot isolation is discussed in detail + below. + </p> + <p> + A transaction may only require <span class="emphasis"><em>cursor + stability</em></span>, that is only be guaranteed that + cursors see committed data that does not change so long as it + is addressed by the cursor, but may change before the reading + transaction completes. This is also called <span class="emphasis"><em>degree 2 + isolation</em></span>. Berkeley DB provides this level of + isolation when a transaction is started with the + <a href="../api_reference/C/dbcget.html#dbcget_DB_READ_COMMITTED" class="olink">DB_READ_COMMITTED</a> flag. This flag may also be specified when + opening a cursor within a fully isolated transaction. + </p> + <p> + Berkeley DB optionally supports reading uncommitted data; + that is, read operations may request data which has been + modified but not yet committed by another transaction. This is + also called <span class="emphasis"><em>degree 1 isolation</em></span>. This is + done by first specifying the <a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a> flag when + opening the underlying database, and then specifying the + <a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a> flag when beginning a transaction, + opening a cursor, or performing a read operation. The + advantage of using <a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a> is that read + operations will not block when another transaction holds a + write lock on the requested data; the disadvantage is that + read operations may return data that will disappear should the + transaction holding the write lock abort. + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> @@ -104,53 +118,82 @@ abort.</p> </div> </div> </div> - <p>To make use of snapshot isolation, databases must first be configured -for multiversion access by calling <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> with the -<a href="../api_reference/C/dbopen.html#dbopen_DB_MULTIVERSION" class="olink">DB_MULTIVERSION</a> flag. Then transactions or cursors must be -configured with the <a href="../api_reference/C/txnbegin.html#txnbegin_DB_TXN_SNAPSHOT" class="olink">DB_TXN_SNAPSHOT</a> flag.</p> - <p>When configuring an environment for snapshot isolation, it is important -to realize that having multiple versions of pages in cache means that -the working set will take up more of the cache. As a result, snapshot -isolation is best suited for use with larger cache sizes.</p> - <p>If the cache becomes full of page copies before the old copies can be -discarded, additional I/O will occur as pages are written to temporary -"freezer" files. This can substantially reduce throughput, and should -be avoided if possible by configuring a large cache and keeping snapshot -isolation transactions short. The amount of cache required to avoid -freezing buffers can be estimated by taking a checkpoint followed by a -call to <a href="../api_reference/C/logarchive.html" class="olink">DB_ENV->log_archive()</a>. The amount of cache required is -approximately double the size of logs that remains.</p> - <p>The environment should also be configured for sufficient transactions -using <a href="../api_reference/C/envset_tx_max.html" class="olink">DB_ENV->set_tx_max()</a>. The maximum number of transactions -needs to include all transactions executed concurrently by the -application plus all cursors configured for snapshot isolation. -Further, the transactions are retained until the last page they created -is evicted from cache, so in the extreme case, an additional transaction -may be needed for each page in the cache. Note that cache sizes under -500MB are increased by 25%, so the calculation of number of pages needs -to take this into account.</p> - <p>So when <span class="emphasis"><em>should</em></span> applications use snapshot isolation?</p> + <p> + To make use of snapshot isolation, databases must first + be configured for multiversion access by calling <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> + with the <a href="../api_reference/C/dbopen.html#dbopen_DB_MULTIVERSION" class="olink">DB_MULTIVERSION</a> flag. Then transactions or + cursors must be configured with the <a href="../api_reference/C/txnbegin.html#txnbegin_DB_TXN_SNAPSHOT" class="olink">DB_TXN_SNAPSHOT</a> + flag. + </p> + <p> + When configuring an environment for snapshot isolation, + it is important to realize that having multiple versions + of pages in cache means that the working set will take up + more of the cache. As a result, snapshot isolation is best + suited for use with larger cache sizes. + </p> + <p> + If the cache becomes full of page copies before the old + copies can be discarded, additional I/O will occur as + pages are written to temporary "freezer" files. This can + substantially reduce throughput, and should be avoided if + possible by configuring a large cache and keeping snapshot + isolation transactions short. The amount of cache required + to avoid freezing buffers can be estimated by taking a + checkpoint followed by a call to <a href="../api_reference/C/logarchive.html" class="olink">DB_ENV->log_archive()</a>. The amount + of cache required is approximately double the size of logs + that remains. + </p> + <p> + The environment should also be configured for sufficient + transactions using <a href="../api_reference/C/envset_tx_max.html" class="olink">DB_ENV->set_tx_max()</a>. The maximum number of + transactions needs to include all transactions executed + concurrently by the application plus all cursors + configured for snapshot isolation. Further, the + transactions are retained until the last page they created + is evicted from cache, so in the extreme case, an + additional transaction may be needed for each page in the + cache. Note that cache sizes under 500MB are increased by + 25%, so the calculation of number of pages needs to take + this into account. + </p> + <p> + So when <span class="emphasis"><em>should</em></span> applications use + snapshot isolation? + </p> <div class="itemizedlist"> <ul type="disc"> - <li>There is a large cache relative to size of updates performed by -concurrent transactions; and</li> - <li>Read/write contention is limiting the throughput of the application; -or</li> - <li>The application is all or mostly read-only, and contention for the lock -manager mutex is limiting throughput.</li> + <li> + There is a large cache relative to size of + updates performed by concurrent transactions; + and + </li> + <li> + Read/write contention is limiting the throughput + of the application; or + </li> + <li> + The application is all or mostly read-only, and + contention for the lock manager mutex is limiting + throughput. + </li> </ul> </div> - <p>The simplest way to take advantage of snapshot isolation is for queries: -keep update transactions using full read/write locking and set -<a href="../api_reference/C/txnbegin.html#txnbegin_DB_TXN_SNAPSHOT" class="olink">DB_TXN_SNAPSHOT</a> on read-only transactions or cursors. This -should minimize blocking of snapshot isolation transactions and will -avoid introducing new -<a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a> -errors.</p> - <p>If the application has update transactions which read many items and -only update a small set (for example, scanning until a desired record is -found, then modifying it), throughput may be improved by running some -updates at snapshot isolation as well.</p> + <p> + The simplest way to take advantage of snapshot isolation + is for queries: keep update transactions using full + read/write locking and set <a href="../api_reference/C/txnbegin.html#txnbegin_DB_TXN_SNAPSHOT" class="olink">DB_TXN_SNAPSHOT</a> on read-only + transactions or cursors. This should minimize blocking of + snapshot isolation transactions and will avoid introducing + new <a class="link" href="program_errorret.html#program_errorret.DB_LOCK_DEADLOCK">DB_LOCK_DEADLOCK</a> errors. + </p> + <p> + If the application has update transactions which read + many items and only update a small set (for example, + scanning until a desired record is found, then modifying + it), throughput may be improved by running some updates at + snapshot isolation as well. + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/transapp_reclimit.html b/docs/programmer_reference/transapp_reclimit.html index 47994027..c93d399b 100644 --- a/docs/programmer_reference/transapp_reclimit.html +++ b/docs/programmer_reference/transapp_reclimit.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_filesys.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_tune.html">Next</a></td> </tr> </table> @@ -38,29 +36,30 @@ </div> </div> </div> - <p> - Berkeley DB recovery is based on write-ahead logging. This means - that when a change is made to a database page, a description of the - change is written into a log file. This description in the log - file is guaranteed to be written to stable storage before the - database pages that were changed are written to stable storage. - This is the fundamental feature of the logging system that makes - durability and rollback work. + <p> + Berkeley DB recovery is based on write-ahead logging. This + means that when a change is made to a database page, a + description of the change is written into a log file. This + description in the log file is guaranteed to be written to + stable storage before the database pages that were changed are + written to stable storage. This is the fundamental feature of + the logging system that makes durability and rollback work. </p> - <p> - If the application or system crashes, the log is reviewed during - recovery. Any database changes described in the log that were part - of committed transactions and that were never written to the actual - database itself are written to the database as part of recovery. - Any database changes described in the log that were never committed - and that were written to the actual database itself are backed-out - of the database as part of recovery. This design allows the - database to be written lazily, and only blocks from the log file - have to be forced to disk as part of transaction commit. + <p> + If the application or system crashes, the log is reviewed + during recovery. Any database changes described in the log + that were part of committed transactions and that were never + written to the actual database itself are written to the + database as part of recovery. Any database changes described + in the log that were never committed and that were written to + the actual database itself are backed-out of the database as + part of recovery. This design allows the database to be + written lazily, and only blocks from the log file have to be + forced to disk as part of transaction commit. </p> - <p> - There are two interfaces that are a concern when considering - Berkeley DB recoverability: + <p> + There are two interfaces that are a concern when + considering Berkeley DB recoverability: </p> <div class="orderedlist"> <ol type="1"> @@ -69,153 +68,162 @@ system/filesystem. </li> <li> - The interface between the operating system/filesystem and the - underlying stable storage hardware. + The interface between the operating + system/filesystem and the underlying stable storage + hardware. </li> </ol> </div> - <p> - Berkeley DB uses the operating system interfaces and its underlying - filesystem when writing its files. This means that Berkeley DB can - fail if the underlying filesystem fails in some unrecoverable way. - Otherwise, the interface requirements here are simple: The system - call that Berkeley DB uses to flush data to disk (normally fsync or - fdatasync), must guarantee that all the information necessary for a - file's recoverability has been written to stable storage before it - returns to Berkeley DB, and that no possible application or system - crash can cause that file to be unrecoverable. + <p> + Berkeley DB uses the operating system interfaces and its + underlying filesystem when writing its files. This means that + Berkeley DB can fail if the underlying filesystem fails in + some unrecoverable way. Otherwise, the interface requirements + here are simple: The system call that Berkeley DB uses to + flush data to disk (normally fsync or fdatasync), must + guarantee that all the information necessary for a file's + recoverability has been written to stable storage before it + returns to Berkeley DB, and that no possible application or + system crash can cause that file to be unrecoverable. </p> - <p> - In addition, Berkeley DB implicitly uses the interface between the - operating system and the underlying hardware. The interface - requirements here are not as simple. + <p> + In addition, Berkeley DB implicitly uses the interface + between the operating system and the underlying hardware. The + interface requirements here are not as simple. </p> - <p> - First, it is necessary to consider the underlying page size of the - Berkeley DB databases. The Berkeley DB library performs all - database writes using the page size specified by the application, - and Berkeley DB assumes pages are written atomically. This means - that if the operating system performs filesystem I/O in blocks of - different sizes than the database page size, it may increase the - possibility for database corruption. For example, assume that - Berkeley DB is writing 32KB pages for a database, and the operating - system does filesystem I/O in 16KB blocks. If the operating system - writes the first 16KB of the database page successfully, but - crashes before being able to write the second 16KB of the database, - the database has been corrupted and this corruption may or may not - be detected during recovery. For this reason, it may be important - to select database page sizes that will be written as single block - transfers by the underlying operating system. If you do not select - a page size that the underlying operating system will write as a - single block, you may want to configure the database to use - checksums (see the <a href="../api_reference/C/dbset_flags.html" class="olink">DB->set_flags()</a> flag for more information). By - configuring checksums, you guarantee this kind of corruption will - be detected at the expense of the CPU required to generate the - checksums. When such an error is detected, the only course of - recovery is to perform catastrophic recovery to restore the - database. + <p> + First, it is necessary to consider the underlying page size + of the Berkeley DB databases. The Berkeley DB library performs + all database writes using the page size specified by the + application, and Berkeley DB assumes pages are written + atomically. This means that if the operating system performs + filesystem I/O in blocks of different sizes than the database + page size, it may increase the possibility for database + corruption. For example, assume that Berkeley DB is writing + 32KB pages for a database, and the operating system does + filesystem I/O in 16KB blocks. If the operating system writes + the first 16KB of the database page successfully, but crashes + before being able to write the second 16KB of the database, + the database has been corrupted and this corruption may or may + not be detected during recovery. For this reason, it may be + important to select database page sizes that will be written + as single block transfers by the underlying operating system. + If you do not select a page size that the underlying operating + system will write as a single block, you may want to configure + the database to use checksums (see the <a href="../api_reference/C/dbset_flags.html" class="olink">DB->set_flags()</a> flag for + more information). By configuring checksums, you guarantee + this kind of corruption will be detected at the expense of the + CPU required to generate the checksums. When such an error is + detected, the only course of recovery is to perform + catastrophic recovery to restore the database. </p> - <p> - Second, if you are copying database files (either as part of doing - a hot backup or creation of a hot failover area), there is an - additional question related to the page size of the Berkeley DB - databases. You must copy databases atomically, in units of the - database page size. In other words, the reads made by the copy - program must not be interleaved with writes by other threads of - control, and the copy program must read the databases in multiples - of the underlying database page size. On Unix systems, this is not - a problem, as these operating systems already make this guarantee - and system utilities normally read in power-of-2 sized chunks, - which are larger than the largest possible Berkeley DB database - page size. Other operating systems, particularly those based on - Linux and Windows, do not provide this guarantee and hot backups may - not be performed on these systems by reading data from the file - system. The <a href="../api_reference/C/db_hotbackup.html" class="olink">db_hotbackup</a> utility should be used on these - systems. + <p> + Second, if you are copying database files (either as part + of doing a hot backup or creation of a hot failover area), + there is an additional question related to the page size of + the Berkeley DB databases. You must copy databases atomically, + in units of the database page size. In other words, the reads + made by the copy program must not be interleaved with writes + by other threads of control, and the copy program must read + the databases in multiples of the underlying database page + size. On Unix systems, this is not a problem, as these + operating systems already make this guarantee and system + utilities normally read in power-of-2 sized chunks, which are + larger than the largest possible Berkeley DB database page + size. Other operating systems, particularly those based on + Linux and Windows, do not provide this guarantee and hot + backups may not be performed on these systems by reading data + from the file system. The <a href="../api_reference/C/db_hotbackup.html" class="olink">db_hotbackup</a> utility should be used on + these systems. </p> <p> - An additional problem we have seen in this area was in some - releases of Solaris where the cp utility was implemented using the - mmap system call rather than the read system call. Because the - Solaris' mmap system call did not make the same guarantee of read - atomicity as the read system call, using the cp utility could - create corrupted copies of the databases. Another problem we have - seen is implementations of the tar utility doing 10KB block reads - by default, and even when an output block size was specified to - that utility, not reading from the underlying databases in - multiples of the block size. Using the dd utility instead of the - cp or tar utilities (and specifying an appropriate block size), - fixes these problems. If you plan to use a system utility to copy - database files, you may want to use a system call trace utility - (for example, ktrace or truss) to check for an I/O size smaller - than or not a multiple of the database page size and system calls - other than read. + An additional problem we have seen in this area was in some + releases of Solaris where the cp utility was implemented using + the mmap system call rather than the read system call. Because + the Solaris' mmap system call did not make the same guarantee + of read atomicity as the read system call, using the cp + utility could create corrupted copies of the databases. + Another problem we have seen is implementations of the tar + utility doing 10KB block reads by default, and even when an + output block size was specified to that utility, not reading + from the underlying databases in multiples of the block size. + Using the dd utility instead of the cp or tar utilities (and + specifying an appropriate block size), fixes these problems. + If you plan to use a system utility to copy database files, + you may want to use a system call trace utility (for example, + ktrace or truss) to check for an I/O size smaller than or not + a multiple of the database page size and system calls other + than read. </p> - <p> - Third, it is necessary to consider the behavior of the system's - underlying stable storage hardware. For example, consider a SCSI - controller that has been configured to cache data and return to the - operating system that the data has been written to stable storage, - when, in fact, it has only been written into the controller RAM - cache. If power is lost before the controller is able to flush its - cache to disk, and the controller cache is not stable (that is, the - writes will not be flushed to disk when power returns), the writes - will be lost. If the writes include database blocks, there is no - loss because recovery will correctly update the database. If the - writes include log file blocks, it is possible that transactions - that were already committed may not appear in the recovered - database, although the recovered database will be coherent after a - crash. + <p> + Third, it is necessary to consider the behavior of the + system's underlying stable storage hardware. For example, + consider a SCSI controller that has been configured to cache + data and return to the operating system that the data has been + written to stable storage, when, in fact, it has only been + written into the controller RAM cache. If power is lost before + the controller is able to flush its cache to disk, and the + controller cache is not stable (that is, the writes will not + be flushed to disk when power returns), the writes will be + lost. If the writes include database blocks, there is no loss + because recovery will correctly update the database. If the + writes include log file blocks, it is possible that + transactions that were already committed may not appear in the + recovered database, although the recovered database will be + coherent after a crash. </p> - <p> - If the underlying hardware can fail in any way so that only part of - the block was written, the failure conditions are the same as those - described previously for an operating system failure that writes - only part of a logical database block. In such cases, configuring - the database for checksums will ensure the corruption is - detected. + <p> + If the underlying hardware can fail in any way so that only + part of the block was written, the failure conditions are the + same as those described previously for an operating system + failure that writes only part of a logical database block. In + such cases, configuring the database for checksums will ensure + the corruption is detected. </p> - <p> - For these reasons, it may be important to select hardware that does - not do partial writes and does not cache data writes (or does not - return that the data has been written to stable storage until it - has either been written to stable storage or the actual writing of - all of the data is guaranteed, barring catastrophic hardware - failure — that is, your disk drive exploding). + <p> + For these reasons, it may be important to select hardware + that does not do partial writes and does not cache data writes + (or does not return that the data has been written to stable + storage until it has either been written to stable storage or + the actual writing of all of the data is guaranteed, barring + catastrophic hardware failure — that is, your disk drive + exploding). </p> - <p> - If the disk drive on which you are storing your databases explodes, - you can perform normal Berkeley DB catastrophic recovery, because - it requires only a snapshot of your databases plus the log files - you have archived since those snapshots were taken. In this case, - you should lose no database changes at all. + <p> + If the disk drive on which you are storing your databases + explodes, you can perform normal Berkeley DB catastrophic + recovery, because it requires only a snapshot of your + databases plus the log files you have archived since those + snapshots were taken. In this case, you should lose no + database changes at all. </p> - <p> - If the disk drive on which you are storing your log files explodes, - you can also perform catastrophic recovery, but you will lose any - database changes made as part of transactions committed since your - last archival of the log files. Alternatively, if your database - environment and databases are still available after you lose the - log file disk, you should be able to dump your databases. However, - you may see an inconsistent snapshot of your data after doing the - dump, because changes that were part of transactions that were not - yet committed may appear in the database dump. Depending on the - value of the data, a reasonable alternative may be to perform both - the database dump and the catastrophic recovery and then compare - the databases created by the two methods. + <p> + If the disk drive on which you are storing your log files + explodes, you can also perform catastrophic recovery, but you + will lose any database changes made as part of transactions + committed since your last archival of the log files. + Alternatively, if your database environment and databases are + still available after you lose the log file disk, you should + be able to dump your databases. However, you may see an + inconsistent snapshot of your data after doing the dump, + because changes that were part of transactions that were not + yet committed may appear in the database dump. Depending on + the value of the data, a reasonable alternative may be to + perform both the database dump and the catastrophic recovery + and then compare the databases created by the two methods. </p> - <p> - Regardless, for these reasons, storing your databases and log files - on different disks should be considered a safety measure as well as - a performance enhancement. + <p> + Regardless, for these reasons, storing your databases and + log files on different disks should be considered a safety + measure as well as a performance enhancement. </p> - <p> - Finally, you should be aware that Berkeley DB does not protect - against all cases of stable storage hardware failure, nor does it - protect against simple hardware misbehavior (for example, a disk - controller writing incorrect data to the disk). However, - configuring the database for checksums will ensure that any such - corruption is detected. + <p> + Finally, you should be aware that Berkeley DB does not + protect against all cases of stable storage hardware failure, + nor does it protect against simple hardware misbehavior (for + example, a disk controller writing incorrect data to the + disk). However, configuring the database for checksums will + ensure that any such corruption is detected. </p> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/transapp_recovery.html b/docs/programmer_reference/transapp_recovery.html index c68e0f32..6991eb9c 100644 --- a/docs/programmer_reference/transapp_recovery.html +++ b/docs/programmer_reference/transapp_recovery.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_logfile.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_hotfail.html">Next</a></td> </tr> </table> @@ -38,107 +36,145 @@ </div> </div> </div> - <p>The fifth component of the infrastructure, recovery procedures, concerns -the recoverability of the database. After any application or system -failure, there are two possible approaches to database recovery:</p> + <p> + The fifth component of the infrastructure, recovery + procedures, concerns the recoverability of the database. After + any application or system failure, there are two possible + approaches to database recovery: + </p> <div class="orderedlist"> <ol type="1"> - <li>There is no need for recoverability, and all databases can be re-created -from scratch. Although these applications may still need transaction -protection for other reasons, recovery usually consists of removing the -Berkeley DB environment home directory and all files it contains, and then -restarting the application. -Such an application may use the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_TXN_NOT_DURABLE" class="olink">DB_TXN_NOT_DURABLE</a> flag to avoid -writing log records.</li> <li> - <p> - It is necessary to recover information after system or application - failure. In this case, recovery processing must be performed on - any database environments that were active at the time of the - failure. Recovery processing involves running the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility or - calling the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method with the <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> or - <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER_FATAL" class="olink">DB_RECOVER_FATAL</a> flags. - </p> - <p> - During recovery processing, all database changes made by aborted or - unfinished transactions are undone, and all database changes made - by committed transactions are redone, as necessary. Database - applications must not be restarted until recovery completes. After - recovery finishes, the environment is properly initialized so that - applications may be restarted. - </p> + <p> + There is no need for recoverability, and all + databases can be re-created from scratch. Although + these applications may still need transaction + protection for other reasons, recovery usually + consists of removing the Berkeley DB environment home + directory and all files it contains, and then + restarting the application. Such an application may + use the <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_TXN_NOT_DURABLE" class="olink">DB_TXN_NOT_DURABLE</a> flag to avoid writing log + records. + </p> + </li> + <li> + <p> + It is necessary to recover information after system + or application failure. In this case, recovery + processing must be performed on any database + environments that were active at the time of the + failure. Recovery processing involves running the + <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility or calling the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method with the + <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> or <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER_FATAL" class="olink">DB_RECOVER_FATAL</a> flags. + </p> + <p> + During recovery processing, all database changes + made by aborted or unfinished transactions are undone, + and all database changes made by committed + transactions are redone, as necessary. Database + applications must not be restarted until recovery + completes. After recovery finishes, the environment is + properly initialized so that applications may be + restarted. + </p> </li> </ol> </div> - <p>If performing recovery, there are two types of recovery processing: -<span class="emphasis"><em>normal</em></span> and <span class="emphasis"><em>catastrophic</em></span>. -Which you choose depends on the source for the database and log files you are -using to recover.</p> - <p> - If up-to-the-minute database and log files are accessible on a stable - filesystem, normal recovery is sufficient. Run the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility or - call the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method specifying the <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> flag. However, the - normal recovery case <span class="bold"><strong>never</strong></span> includes - recovery using hot backups of the database environment. For example, - you cannot perform a hot backup of databases and log files, restore the - backup and then run normal recovery — you must always run catastrophic - recovery when using hot backups. -</p> <p> - If the database or log files have been destroyed or corrupted, or - normal recovery fails, catastrophic recovery is required. For example, - catastrophic failure includes the case where the disk drive on which - the database or log files are stored has been physically destroyed, or - when the underlying filesystem is corrupted and the operating system's - normal filesystem checking procedures cannot bring that filesystem to a - consistent state. This is often difficult to detect, and a common sign - of the need for catastrophic recovery is when normal Berkeley DB - recovery procedures fail, or when checksum errors are displayed during - normal database procedures. When catastrophic recovery is necessary, - take the following steps: -</p> - <div class="orderedlist"> - <ol type="1"> - <li>Restore the most recent snapshots of the database and log files from -the backup media into the directory where recovery will be performed.</li> - <li> - <p> - If any log files were archived since the last snapshot was made, - they should be restored into the directory where recovery will be - performed. + If performing recovery, there are two types of recovery + processing: <span class="emphasis"><em>normal</em></span> and + <span class="emphasis"><em>catastrophic</em></span>. Which you choose + depends on the source for the database and log files you are + using to recover. </p> - <p> - If any log files are available from the database environment that - failed (for example, the disk holding the database files crashed, - but the disk holding the log files is fine), those log files should - be copied into the directory where recovery will be performed. + <p> + If up-to-the-minute database and log files are accessible + on a stable filesystem, normal recovery is sufficient. Run the + <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility or call the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method specifying the + <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> flag. However, the normal recovery case <span class="bold"><strong>never</strong></span> includes recovery using hot + backups of the database environment. For example, you cannot + perform a hot backup of databases and log files, restore the + backup and then run normal recovery — you must always + run catastrophic recovery when using hot backups. </p> - <p> - Be sure to restore all log files in the order they were written. - The order is important because it's possible the same log file - appears on multiple backups, and you want to run recovery using the - most recent version of each log file. + <p> + If the database or log files have been destroyed or + corrupted, or normal recovery fails, catastrophic recovery is + required. For example, catastrophic failure includes the case + where the disk drive on which the database or log files are + stored has been physically destroyed, or when the underlying + filesystem is corrupted and the operating system's normal + filesystem checking procedures cannot bring that filesystem to + a consistent state. This is often difficult to detect, and a + common sign of the need for catastrophic recovery is when + normal Berkeley DB recovery procedures fail, or when checksum + errors are displayed during normal database procedures. + </p> + <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> + <h3 class="title">Note</h3> + <p> + Berkeley DB backups (archives) can be recovered using + machines of differing byte order. That is, a backup taken + on a big-endian machine can be used to restore a database + residing on a little-endian machine. + </p> + </div> + <p> + When catastrophic recovery is necessary, take the following + steps: </p> + <div class="orderedlist"> + <ol type="1"> + <li> + <p> + Restore the most recent snapshots of the database + and log files from the backup media into the directory + where recovery will be performed. + </p> </li> <li> + <p> + If any log files were archived since the last + snapshot was made, they should be restored into the + directory where recovery will be performed. + </p> + <p> + If any log files are available from the database + environment that failed (for example, the disk holding + the database files crashed, but the disk holding the + log files is fine), those log files should be copied + into the directory where recovery will be performed. + </p> <p> - Run the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility, specifying its <span class="bold"><strong>-c</strong></span> option; or call the <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method, - specifying the <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER_FATAL" class="olink">DB_RECOVER_FATAL</a> flag. The catastrophic recovery - process will review the logs and database files to bring the - environment databases to a consistent state as of the time of the - last uncorrupted log file that is found. It is important to - realize that only transactions committed before that date will - appear in the databases. - </p> - <p> - It is possible to re-create the database in a location different - from the original by specifying appropriate pathnames to the - <span class="bold"><strong>-h</strong></span> option of the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility. In - order for this to work properly, it is important that your - application refer to files by names relative to the database home - directory or the pathname(s) specified in calls to - <a href="../api_reference/C/envset_data_dir.html" class="olink">DB_ENV->set_data_dir()</a>, instead of using full pathnames. - </p> + Be sure to restore all log files in the order they + were written. The order is important because it's + possible the same log file appears on multiple + backups, and you want to run recovery using the most + recent version of each log file. + </p> + </li> + <li> + <p> + Run the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility, specifying its <span class="bold"><strong>-c</strong></span> option; or call the + <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> method, specifying the <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER_FATAL" class="olink">DB_RECOVER_FATAL</a> + flag. The catastrophic recovery process will review + the logs and database files to bring the environment + databases to a consistent state as of the time of the + last uncorrupted log file that is found. It is + important to realize that only transactions committed + before that date will appear in the databases. + </p> + <p> + It is possible to re-create the database in a + location different from the original by specifying + appropriate pathnames to the <span class="bold"><strong> + -h</strong></span> option of the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility. In + order for this to work properly, it is important that + your application refer to files by names relative to + the database home directory or the pathname(s) + specified in calls to <a href="../api_reference/C/envadd_data_dir.html" class="olink">DB_ENV->add_data_dir()</a>, instead of + using full pathnames. + </p> </li> </ol> </div> diff --git a/docs/programmer_reference/transapp_term.html b/docs/programmer_reference/transapp_term.html index ef80e7d1..9fd1c641 100644 --- a/docs/programmer_reference/transapp_term.html +++ b/docs/programmer_reference/transapp_term.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_why.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_fail.html">Next</a></td> </tr> </table> @@ -38,70 +36,98 @@ </div> </div> </div> - <p>Here are some definitions that will be helpful in understanding -transactions:</p> + <p> + Here are some definitions that will be helpful in + understanding transactions: + </p> <div class="variablelist"> <dl> <dt> <span class="term">Thread of control</span> </dt> - <dd>Berkeley DB is indifferent to the type or style of threads being used by the -application; or, for that matter, if threads are being used at all — -because Berkeley DB supports multiprocess access. In the Berkeley DB documentation, -any time we refer to a <span class="emphasis"><em>thread of control</em></span>, it can be read as -a true thread (one of many in an application's address space) or a -process.</dd> + <dd> + Berkeley DB is indifferent to the type or style + of threads being used by the application; or, for that + matter, if threads are being used at all — + because Berkeley DB supports multiprocess access. In + the Berkeley DB documentation, any time we refer to a + <span class="emphasis"><em>thread of control</em></span>, it can be + read as a true thread (one of many in an application's + address space) or a process. + </dd> <dt> <span class="term">Free-threaded</span> </dt> - <dd>A Berkeley DB handle that can be used by multiple threads simultaneously -without any application-level synchronization is called -<span class="emphasis"><em>free-threaded</em></span>.</dd> + <dd> + A Berkeley DB handle that can be used by + multiple threads simultaneously without any + application-level synchronization is called + <span class="emphasis"><em>free-threaded</em></span>. + </dd> <dt> <span class="term">Transaction</span> </dt> - <dd>A <span class="emphasis"><em>transaction</em></span> is a one or more operations on one or more -databases that should be treated as a single unit of work. For example, -changes to a set of databases, in which either all of the changes must be -applied to the database(s) or none of them should. Applications specify -when each transaction starts, what database operations are included in -it, and when it ends.</dd> + <dd> + A <span class="emphasis"><em>transaction</em></span> is a one or + more operations on one or more databases that should + be treated as a single unit of work. For example, + changes to a set of databases, in which either all of + the changes must be applied to the database(s) or none + of them should. Applications specify when each + transaction starts, what database operations are + included in it, and when it ends. + </dd> <dt> <span class="term">Transaction abort/commit</span> </dt> - <dd>Every transaction ends by <span class="emphasis"><em>committing</em></span> or <span class="emphasis"><em>aborting</em></span>. -If a transaction commits, Berkeley DB guarantees that any database changes -included in the transaction will never be lost, even after system or -application failure. If a transaction aborts, or is uncommitted when -the system or application fails, then the changes involved will never -appear in the database.</dd> + <dd> + Every transaction ends by + <span class="emphasis"><em>committing</em></span> or + <span class="emphasis"><em>aborting</em></span>. If a transaction + commits, Berkeley DB guarantees that any database + changes included in the transaction will never be + lost, even after system or application failure. If a + transaction aborts, or is uncommitted when the system + or application fails, then the changes involved will + never appear in the database. + </dd> <dt> <span class="term">System or application failure</span> </dt> - <dd><span class="emphasis"><em>System or application failure</em></span> is the phrase we use to -describe something bad happening near your data. It can be an -application dumping core, being interrupted by a signal, the disk -filling up, or the entire system crashing. In any case, for whatever -reason, the application can no longer make forward progress, and its -databases are left in an unknown state.</dd> + <dd><span class="emphasis"><em>System or application + failure</em></span> is the phrase we use to + describe something bad happening near your data. It + can be an application dumping core, being interrupted + by a signal, the disk filling up, or the entire system + crashing. In any case, for whatever reason, the + application can no longer make forward progress, and + its databases are left in an unknown state. + </dd> <dt> <span class="term">Recovery</span> </dt> - <dd><span class="emphasis"><em>Recovery</em></span> is what makes the database consistent after a system -or application failure. The recovery process includes review of log -files and databases to ensure that the changes from each committed -transaction appear in the database, and that no changes from an -unfinished (or aborted) transaction do. Whenever system or application -failure occurs, applications must usually run recovery.</dd> + <dd><span class="emphasis"><em>Recovery</em></span> is what makes the + database consistent after a system or application + failure. The recovery process includes review of log + files and databases to ensure that the changes from + each committed transaction appear in the database, and + that no changes from an unfinished (or aborted) + transaction do. Whenever system or application failure + occurs, applications must usually run + recovery. + </dd> <dt> <span class="term">Deadlock</span> </dt> - <dd><span class="emphasis"><em>Deadlock</em></span>, in its simplest form, happens when one thread of -control owns resource A, but needs resource B; while another thread of -control owns resource B, but needs resource A. Neither thread of -control can make progress, and so one has to give up and release all -its resources, at which time the remaining thread of control can make -forward progress.</dd> + <dd><span class="emphasis"><em>Deadlock</em></span>, in its simplest + form, happens when one thread of control owns resource + A, but needs resource B; while another thread of + control owns resource B, but needs resource A. Neither + thread of control can make progress, and so one has to + give up and release all its resources, at which time + the remaining thread of control can make forward + progress. + </dd> </dl> </div> </div> diff --git a/docs/programmer_reference/transapp_throughput.html b/docs/programmer_reference/transapp_throughput.html index 19d4cf6e..c5bc7083 100644 --- a/docs/programmer_reference/transapp_throughput.html +++ b/docs/programmer_reference/transapp_throughput.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Transaction throughput</th> + <th colspan="3" align="center">Transaction + throughput</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_tune.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_faq.html">Next</a></td> </tr> </table> @@ -34,121 +33,196 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="transapp_throughput"></a>Transaction throughput</h2> + <h2 class="title" style="clear: both"><a id="transapp_throughput"></a>Transaction + throughput</h2> </div> </div> </div> - <p>Generally, the speed of a database system is measured by the -<span class="emphasis"><em>transaction throughput</em></span>, expressed as a number of -transactions per second. The two gating factors for Berkeley DB performance -in a transactional system are usually the underlying database files and -the log file. Both are factors because they require disk I/O, which is -slow relative to other system resources such as CPU.</p> - <p>In the worst-case scenario:</p> + <p> + Generally, the speed of a database system is measured by the + <span class="emphasis"><em>transaction throughput</em></span>, expressed as + a number of transactions per second. The two gating factors + for Berkeley DB performance in a transactional system are + usually the underlying database files and the log file. Both + are factors because they require disk I/O, which is slow + relative to other system resources such as CPU. + </p> + <p> + In the worst-case scenario: + </p> <div class="itemizedlist"> <ul type="disc"> - <li>Database access is truly random and the database is too large for any -significant percentage of it to fit into the cache, resulting in a -single I/O per requested key/data pair.</li> - <li>Both the database and the log are on a single disk.</li> + <li> + Database access is truly random and the database is + too large for any significant percentage of it to fit into + the cache, resulting in a single I/O per requested + key/data pair. + </li> + <li> + Both the database and the log are on a single + disk. + </li> </ul> </div> - <p>This means that for each transaction, Berkeley DB is potentially performing -several filesystem operations:</p> + <p> + This means that for each transaction, Berkeley DB is + potentially performing several filesystem operations: + </p> <div class="itemizedlist"> <ul type="disc"> - <li>Disk seek to database file</li> - <li>Database file read</li> - <li>Disk seek to log file</li> - <li>Log file write</li> - <li>Flush log file information to disk</li> - <li>Disk seek to update log file metadata (for example, inode information)</li> - <li>Log metadata write</li> - <li>Flush log file metadata to disk</li> + <li> + Disk seek to database file + </li> + <li> + Database file read + </li> + <li> + Disk seek to log file</li> + <li> + Log file write + </li> + <li> + Flush log file information to disk + </li> + <li> + Disk seek to update log file metadata (for example, + inode information) + </li> + <li> + Log metadata write + </li> + <li> + Flush log file metadata to disk + </li> </ul> </div> - <p>There are a number of ways to increase transactional throughput, all of -which attempt to decrease the number of filesystem operations per -transaction. First, the Berkeley DB software includes support for -<span class="emphasis"><em>group commit</em></span>. Group commit simply means that when the -information about one transaction is flushed to disk, the information -for any other waiting transactions will be flushed to disk at the same -time, potentially amortizing a single log write over a large number of -transactions. There are additional tuning parameters which may be -useful to application writers:</p> + <p> + There are a number of ways to increase transactional + throughput, all of which attempt to decrease the number of + filesystem operations per transaction. First, the Berkeley DB + software includes support for <span class="emphasis"><em>group + commit</em></span>. Group commit simply means that when the + information about one transaction is flushed to disk, the + information for any other waiting transactions will be flushed + to disk at the same time, potentially amortizing a single log + write over a large number of transactions. There are + additional tuning parameters which may be useful to + application writers: + </p> <div class="itemizedlist"> <ul type="disc"> - <li>Tune the size of the database cache. If the Berkeley DB key/data pairs used -during the transaction are found in the database cache, the seek and read -from the database are no longer necessary, resulting in two fewer -filesystem operations per transaction. To determine whether your cache -size is too small, see <a class="xref" href="general_am_conf.html#am_conf_cachesize" title="Selecting a cache size">Selecting a cache size</a>.</li> - <li>Put the database and the log files on different disks. This allows reads -and writes to the log files and the database files to be performed -concurrently.</li> - <li>Set the filesystem configuration so that file access and modification times -are not updated. Note that although the file access and modification times -are not used by Berkeley DB, this may affect other programs -- so be careful.</li> - <li>Upgrade your hardware. When considering the hardware on which to run your -application, however, it is important to consider the entire system. The -controller and bus can have as much to do with the disk performance as -the disk itself. It is also important to remember that throughput is -rarely the limiting factor, and that disk seek times are normally the true -performance issue for Berkeley DB.</li> - <li> - Turn on the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> or <a href="../api_reference/C/envset_flags.html#set_flags_DB_TXN_WRITE_NOSYNC" class="olink">DB_TXN_WRITE_NOSYNC</a> flags. This - changes the Berkeley DB behavior so that the log files are not written - and/or flushed when transactions are committed. Although this change - will greatly increase your transaction throughput, it means that - transactions will exhibit the ACI (atomicity, consistency, and - isolation) properties, but not D (durability). Database integrity will - be maintained, but it is possible that some number of the most recently - committed transactions may be undone during recovery instead of being - redone. -</li> + <li> + Tune the size of the database cache. If the Berkeley + DB key/data pairs used during the transaction are found in + the database cache, the seek and read from the database + are no longer necessary, resulting in two fewer filesystem + operations per transaction. To determine whether your + cache size is too small, see <a class="xref" href="general_am_conf.html#am_conf_cachesize" title="Selecting a cache size">Selecting a cache size</a>. + </li> + <li> + Put the database and the log files on different + disks. This allows reads and writes to the log files and + the database files to be performed + concurrently. + </li> + <li> + Set the filesystem configuration so that file access + and modification times are not updated. Note that although + the file access and modification times are not used by + Berkeley DB, this may affect other programs -- so be + careful. + </li> + <li> + Upgrade your hardware. When considering the hardware + on which to run your application, however, it is important + to consider the entire system. The controller and bus can + have as much to do with the disk performance as the disk + itself. It is also important to remember that throughput + is rarely the limiting factor, and that disk seek times + are normally the true performance issue for Berkeley + DB. + </li> + <li> + Turn on the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> or + <a href="../api_reference/C/envset_flags.html#set_flags_DB_TXN_WRITE_NOSYNC" class="olink">DB_TXN_WRITE_NOSYNC</a> flags. This changes the Berkeley DB + behavior so that the log files are not written and/or + flushed when transactions are committed. Although this + change will greatly increase your transaction throughput, + it means that transactions will exhibit the ACI + (atomicity, consistency, and isolation) properties, but + not D (durability). Database integrity will be maintained, + but it is possible that some number of the most recently + committed transactions may be undone during recovery + instead of being redone. + </li> </ul> </div> - <p>If you are bottlenecked on logging, the following test will help you -confirm that the number of transactions per second that your application -does is reasonable for the hardware on which you are running. Your test -program should repeatedly perform the following operations:</p> + <p> + If you are bottlenecked on logging, the following test will + help you confirm that the number of transactions per second + that your application does is reasonable for the hardware on + which you are running. Your test program should repeatedly + perform the following operations: + </p> <div class="itemizedlist"> <ul type="disc"> - <li>Seek to the beginning of a file</li> - <li>Write to the file</li> - <li>Flush the file write to disk</li> + <li> + Seek to the beginning of a file + </li> + <li> + Write to the file + </li> + <li> + Flush the file write to disk + </li> </ul> </div> - <p>The number of times that you can perform these three operations per -second is a rough measure of the minimum number of transactions per -second of which the hardware is capable. This test simulates the -operations applied to the log file. (As a simplifying assumption in this -experiment, we assume that the database files are either on a separate -disk; or that they fit, with some few exceptions, into the database -cache.) We do not have to directly simulate updating the log file -directory information because it will normally be updated and flushed -to disk as a result of flushing the log file write to disk.</p> - <p>Running this test program, in which we write 256 bytes for 1000 operations -on reasonably standard commodity hardware (Pentium II CPU, SCSI disk), -returned the following results:</p> + <p> + The number of times that you can perform these three + operations per second is a rough measure of the minimum number + of transactions per second of which the hardware is capable. + This test simulates the operations applied to the log file. + (As a simplifying assumption in this experiment, we assume + that the database files are either on a separate disk; or that + they fit, with some few exceptions, into the database cache.) + We do not have to directly simulate updating the log file + directory information because it will normally be updated and + flushed to disk as a result of flushing the log file write to + disk. + </p> + <p> + Running this test program, in which we write 256 bytes for + 1000 operations on reasonably standard commodity hardware + (Pentium II CPU, SCSI disk), returned the following + results: + </p> <pre class="programlisting">% testfile -b256 -o1000 running: 1000 ops Elapsed time: 16.641934 seconds 1000 ops: 60.09 ops per second</pre> - <p>Note that the number of bytes being written to the log as part of each -transaction can dramatically affect the transaction throughput. The -test run used 256, which is a reasonable size log write. Your log -writes may be different. To determine your average log write size, use -the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility to display your log statistics.</p> - <p>As a quick sanity check, the average seek time is 9.4 msec for this -particular disk, and the average latency is 4.17 msec. That results in -a minimum requirement for a data transfer to the disk of 13.57 msec, or -a maximum of 74 transfers per second. This is close enough to the -previous 60 operations per second (which was not done on a quiescent -disk) that the number is believable.</p> - <p>An implementation of the previous <a class="ulink" href="writetest.cs" target="_top">example test -program</a> for IEEE/ANSI Std 1003.1 (POSIX) standard systems is included in the Berkeley DB -distribution.</p> + <p> + Note that the number of bytes being written to the log as + part of each transaction can dramatically affect the + transaction throughput. The test run used 256, which is a + reasonable size log write. Your log writes may be different. + To determine your average log write size, use the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility to + display your log statistics. + </p> + <p> + As a quick sanity check, the average seek time is 9.4 msec + for this particular disk, and the average latency is 4.17 + msec. That results in a minimum requirement for a data + transfer to the disk of 13.57 msec, or a maximum of 74 + transfers per second. This is close enough to the previous 60 + operations per second (which was not done on a quiescent disk) + that the number is believable. + </p> + <p> + An implementation of the previous <a class="ulink" href="writetest.cs" target="_top"> + example test program</a> for IEEE/ANSI Std 1003.1 + (POSIX) standard systems is included in the Berkeley DB + distribution. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/transapp_tune.html b/docs/programmer_reference/transapp_tune.html index 7bdc1049..7f8dfd51 100644 --- a/docs/programmer_reference/transapp_tune.html +++ b/docs/programmer_reference/transapp_tune.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_reclimit.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_throughput.html">Next</a></td> </tr> </table> @@ -38,192 +36,257 @@ </div> </div> </div> - <p>There are a few different issues to consider when tuning the performance -of Berkeley DB transactional applications. First, you should review -<a class="xref" href="am_misc_tune.html" title="Access method tuning">Access method tuning</a>, as the -tuning issues for access method applications are applicable to -transactional applications as well. The following are additional tuning -issues for Berkeley DB transactional applications:</p> + <p> + There are a few different issues to consider when tuning the + performance of Berkeley DB transactional applications. First, + you should review <a class="xref" href="am_misc_tune.html" title="Access method tuning">Access method tuning</a>, as the tuning issues for + access method applications are applicable to transactional + applications as well. The following are additional tuning + issues for Berkeley DB transactional applications: + </p> <div class="variablelist"> <dl> <dt> <span class="term">access method</span> </dt> - <dd>Highly concurrent applications should use the Queue access method, where -possible, as it provides finer-granularity of locking than the other -access methods. Otherwise, applications usually see better concurrency -when using the Btree access method than when using either the Hash or -Recno access methods.</dd> + <dd> + Highly concurrent applications should use the + Queue access method, where possible, as it provides + finer-granularity of locking than the other access + methods. Otherwise, applications usually see better + concurrency when using the Btree access method than + when using either the Hash or Recno access + methods. + </dd> <dt> <span class="term">record numbers</span> </dt> - <dd>Using record numbers outside of the Queue access method will often slow -down concurrent applications as they limit the degree of concurrency -available in the database. Using the Recno access method, or the Btree -access method with retrieval by record number configured can slow -applications down.</dd> + <dd> + Using record numbers outside of the Queue access + method will often slow down concurrent applications as + they limit the degree of concurrency available in the + database. Using the Recno access method, or the Btree + access method with retrieval by record number + configured can slow applications down. + </dd> <dt> <span class="term">Btree database size</span> </dt> - <dd>When using the Btree access method, applications supporting concurrent -access may see excessive numbers of deadlocks in small databases. There -are two different approaches to resolving this problem. First, as the -Btree access method uses page-level locking, decreasing the database -page size can result in fewer lock conflicts. Second, in the case of -databases that are cyclically growing and shrinking, turning off reverse -splits (with <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_REVSPLITOFF" class="olink">DB_REVSPLITOFF</a>) can leave the database with enough -pages that there will be fewer lock conflicts.</dd> + <dd> + When using the Btree access method, applications + supporting concurrent access may see excessive numbers + of deadlocks in small databases. There are two + different approaches to resolving this problem. First, + as the Btree access method uses page-level locking, + decreasing the database page size can result in fewer + lock conflicts. Second, in the case of databases that + are cyclically growing and shrinking, turning off + reverse splits (with <a href="../api_reference/C/dbset_flags.html#dbset_flags_DB_REVSPLITOFF" class="olink">DB_REVSPLITOFF</a>) can leave the + database with enough pages that there will be fewer + lock conflicts. + </dd> <dt> <span class="term">read locks</span> </dt> - <dd>Performing all read operations outside of transactions or at -<a class="xref" href="transapp_read.html" title="Degrees of isolation">Degrees of isolation</a> can often -significantly increase application throughput. In addition, limiting -the lifetime of non-transactional cursors will reduce the length of -times locks are held, thereby improving concurrency.</dd> + <dd> + Performing all read operations outside of + transactions or at <a class="xref" href="transapp_read.html" title="Degrees of isolation">Degrees of isolation</a> can often + significantly increase application throughput. In + addition, limiting the lifetime of non-transactional + cursors will reduce the length of times locks are + held, thereby improving concurrency. + </dd> <dt> <span class="term"><a href="../api_reference/C/envset_flags.html#set_flags_DB_DIRECT_DB" class="olink">DB_DIRECT_DB</a>, <a href="../api_reference/C/envlog_set_config.html#log_set_config_DB_LOG_DIRECT" class="olink">DB_LOG_DIRECT</a></span> </dt> <dd> -On some systems, avoiding caching in the operating system can improve -write throughput and allow the creation of larger Berkeley DB caches.</dd> + On some systems, avoiding caching in the + operating system can improve write throughput and + allow the creation of larger Berkeley DB + caches. + </dd> <dt> <span class="term"><a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a>, <a href="../api_reference/C/dbcget.html#dbcget_DB_READ_COMMITTED" class="olink">DB_READ_COMMITTED</a></span> </dt> <dd> <p> - Consider decreasing the level of isolation of transaction using - the <a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a>, or <a href="../api_reference/C/dbcget.html#dbcget_DB_READ_COMMITTED" class="olink">DB_READ_COMMITTED</a> flags for - transactions or cursors or the <a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a> flag on - individual read operations. The <a href="../api_reference/C/dbcget.html#dbcget_DB_READ_COMMITTED" class="olink">DB_READ_COMMITTED</a> flag will - release read locks on cursors as soon as the data page is - nolonger referenced. This is also called - <span class="emphasis"><em> degree 2 isolation</em></span>. This will - tend to block write operations for shorter periods for - applications that do not need to have repeatable reads for - cursor operations. - </p> - <p> - The <a href="../api_reference/C/dbcget.html#dbcget_DB_READ_COMMITTED" class="olink">DB_READ_COMMITTED</a> flag will allow read operations to - potentially return data which has been modified but not yet - committed, and can significantly increase application - throughput in applications that do not require data be - guaranteed to be permanent in the database. This is also - called <span class="emphasis"><em>degree 1 isolation</em></span>, - or <span class="emphasis"><em>dirty reads</em></span>. - </p> + Consider decreasing the level of isolation of + transaction using the <a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a>, or + <a href="../api_reference/C/dbcget.html#dbcget_DB_READ_COMMITTED" class="olink">DB_READ_COMMITTED</a> flags for transactions or + cursors or the <a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a> flag on + individual read operations. The + <a href="../api_reference/C/dbcget.html#dbcget_DB_READ_COMMITTED" class="olink">DB_READ_COMMITTED</a> flag will release read locks + on cursors as soon as the data page is nolonger + referenced. This is also called <span class="emphasis"><em> degree + 2 isolation</em></span>. This will tend to + block write operations for shorter periods for + applications that do not need to have repeatable + reads for cursor operations. + </p> + <p> + The <a href="../api_reference/C/dbopen.html#dbopen_DB_READ_UNCOMMITTED" class="olink">DB_READ_UNCOMMITTED</a> flag will allow read + operations to potentially return data which has + been modified but not yet committed, and can + significantly increase application throughput in + applications that do not require data be + guaranteed to be permanent in the database. This + is also called <span class="emphasis"><em>degree 1 + isolation</em></span>, or <span class="emphasis"><em>dirty + reads</em></span>. + </p> </dd> <dt> <span class="term"> <a href="../api_reference/C/dbcget.html#dbcget_DB_RMW" class="olink">DB_RMW</a> </span> </dt> - <dd> If there are many deadlocks, consider -using the <a href="../api_reference/C/dbcget.html#dbcget_DB_RMW" class="olink">DB_RMW</a> flag to -immediately acquire write locks when reading data items that will -subsequently be modified. Although this flag may increase contention -(because write locks are held longer than they would otherwise be), it -may decrease the number of deadlocks that occur.</dd> + <dd> + If there are many deadlocks, consider using the + <a href="../api_reference/C/dbcget.html#dbcget_DB_RMW" class="olink">DB_RMW</a> flag to immediately acquire write locks when + reading data items that will subsequently be modified. + Although this flag may increase contention (because + write locks are held longer than they would otherwise + be), it may decrease the number of deadlocks that + occur. + </dd> <dt> <span class="term"><a href="../api_reference/C/envset_flags.html#set_flags_DB_TXN_WRITE_NOSYNC" class="olink">DB_TXN_WRITE_NOSYNC</a>, <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a></span> </dt> - <dd>By default, transactional commit in Berkeley DB implies durability, that is, -all committed operations will be present in the database after recovery -from any application or system failure. For applications not requiring -that level of certainty, specifying the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> flag will -often provide a significant performance improvement. In this case, the -database will still be fully recoverable, but some number of committed -transactions might be lost after application or system failure.</dd> + <dd> + By default, transactional commit in Berkeley DB + implies durability, that is, all committed operations + will be present in the database after recovery from + any application or system failure. For applications + not requiring that level of certainty, specifying the + <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> flag will often provide a significant + performance improvement. In this case, the database + will still be fully recoverable, but some number of + committed transactions might be lost after application + or system failure. + </dd> <dt> <span class="term">access databases in order</span> </dt> - <dd>When modifying multiple databases in a single transaction, always access -physical files and databases within physical files, in the same order -where possible. In addition, avoid returning to a physical file or -database, that is, avoid accessing a database, moving on to another -database and then returning to the first database. This can -significantly reduce the chance of deadlock between threads of -control.</dd> + <dd> + When modifying multiple databases in a single + transaction, always access physical files and + databases within physical files, in the same order + where possible. In addition, avoid returning to a + physical file or database, that is, avoid accessing a + database, moving on to another database and then + returning to the first database. This can + significantly reduce the chance of deadlock between + threads of control. + </dd> <dt> <span class="term">large key/data items</span> </dt> - <dd>Transactional protections in Berkeley DB are guaranteed by before and after -physical image logging. This means applications modifying large -key/data items also write large log records, and, in the case of the -default transaction commit, threads of control must wait until those -log records have been flushed to disk. Applications supporting -concurrent access should try and keep key/data items small wherever -possible.</dd> + <dd> + Transactional protections in Berkeley DB are + guaranteed by before and after physical image logging. + This means applications modifying large key/data items + also write large log records, and, in the case of the + default transaction commit, threads of control must + wait until those log records have been flushed to + disk. Applications supporting concurrent access should + try and keep key/data items small wherever + possible. + </dd> <dt> <span class="term">mutex selection</span> </dt> <dd> <p> - During configuration, Berkeley DB selects a mutex implementation - for the architecture. Berkeley DB normally prefers blocking-mutex - implementations over non-blocking ones. For example, Berkeley DB - will select POSIX pthread mutex interfaces rather than - assembly-code test-and-set spin mutexes because pthread mutexes are - usually more efficient and less likely to waste CPU cycles spinning - without getting any work accomplished. - </p> - <p> - For some applications and systems (generally highly concurrent - applications on large multiprocessor systems), Berkeley DB makes - the wrong choice. In some cases, better performance can be - achieved by configuring with the <a href="../installation/build_unix_conf.html#build_unix_conf.--with-mutex" class="olink">--with-mutex</a> - argument and selecting a different mutex implementation than the - one selected by Berkeley DB. When a test-and-set spin mutex - implementation is selected, it may be useful to tune the number of - spins made before yielding the processor and sleeping. For more - information, see the <a href="../api_reference/C/mutexset_tas_spins.html" class="olink">DB_ENV->mutex_set_tas_spins()</a> method. - </p> + During configuration, Berkeley DB selects a + mutex implementation for the architecture. + Berkeley DB normally prefers blocking-mutex + implementations over non-blocking ones. For + example, Berkeley DB will select POSIX pthread + mutex interfaces rather than assembly-code + test-and-set spin mutexes because pthread mutexes + are usually more efficient and less likely to + waste CPU cycles spinning without getting any work + accomplished. + </p> <p> - Finally, Berkeley DB may put multiple mutexes on individual cache - lines. When tuning Berkeley DB for large multiprocessor systems, - it may be useful to tune mutex alignment using the <a href="../api_reference/C/mutexset_align.html" class="olink">DB_ENV->mutex_set_align()</a> - method. - </p> + For some applications and systems (generally + highly concurrent applications on large + multiprocessor systems), Berkeley DB makes the + wrong choice. In some cases, better performance + can be achieved by configuring with the + <a href="../installation/build_unix_conf.html#build_unix_conf.--with-mutex" class="olink">--with-mutex</a> argument and selecting a different + mutex implementation than the one selected by + Berkeley DB. When a test-and-set spin mutex + implementation is selected, it may be useful to + tune the number of spins made before yielding the + processor and sleeping. This may be particularly + beneficial for systems containing several + hyperthreaded processor cores. For more + information, see the <a href="../api_reference/C/mutexset_tas_spins.html" class="olink">DB_ENV->mutex_set_tas_spins()</a> method. + </p> + <p> + Finally, Berkeley DB may put multiple mutexes + on individual cache lines. When tuning Berkeley DB + for large multiprocessor systems, it may be useful + to tune mutex alignment using the <a href="../api_reference/C/mutexset_align.html" class="olink">DB_ENV->mutex_set_align()</a> + method. + </p> </dd> <dt> <span class="term"> <a href="../installation/build_unix_conf.html#build_unix_conf.--enable-posixmutexes" class="olink">--enable-posix-mutexes</a> </span> </dt> - <dd>By default, the Berkeley DB library will only select the POSIX pthread mutex -implementation if it supports mutexes shared between multiple processes. -If your application does not share its database environment between -processes and your system's POSIX mutex support was not selected because -it did not support inter-process mutexes, you may be able to increase -performance and transactional throughput by configuring with the -<a href="../installation/build_unix_conf.html#build_unix_conf.--enable-posixmutexes" class="olink">--enable-posix-mutexes</a> argument.</dd> + <dd> + By default, the Berkeley DB library will only + select the POSIX pthread mutex implementation if it + supports mutexes shared between multiple processes. If + your application does not share its database + environment between processes and your system's POSIX + mutex support was not selected because it did not + support inter-process mutexes, you may be able to + increase performance and transactional throughput by + configuring with the <a href="../installation/build_unix_conf.html#build_unix_conf.--enable-posixmutexes" class="olink">--enable-posix-mutexes</a> + argument. + </dd> <dt> <span class="term">log buffer size</span> </dt> - <dd>Berkeley DB internally maintains a buffer of log writes. The buffer is -written to disk at transaction commit, by default, or, whenever it -is filled. If it is consistently being filled before transaction -commit, it will be written multiple times per transaction, costing -application performance. In these cases, increasing the size of the -log buffer can increase application throughput.</dd> + <dd> + Berkeley DB internally maintains a buffer of log + writes. The buffer is written to disk at transaction + commit, by default, or, whenever it is filled. If it + is consistently being filled before transaction + commit, it will be written multiple times per + transaction, costing application performance. In these + cases, increasing the size of the log buffer can + increase application throughput. + </dd> <dt> <span class="term">log file location</span> </dt> - <dd>If the database environment's log files are on the same disk as the -databases, the disk arms will have to seek back-and-forth between the -two. Placing the log files and the databases on different disk arms -can often increase application throughput.</dd> + <dd> + If the database environment's log files are on + the same disk as the databases, the disk arms will + have to seek back-and-forth between the two. Placing + the log files and the databases on different disk arms + can often increase application throughput. + </dd> <dt> <span class="term">trickle write</span> </dt> - <dd>In some applications, the cache is sufficiently active and dirty that -readers frequently need to write a dirty page in order to have space in -which to read a new page from the backing database file. You can use -the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility (or the statistics returned by the -<a href="../api_reference/C/mempstat.html" class="olink">DB_ENV->memp_stat()</a> method) to see how often this is happening in your -application's cache. In this case, using a separate thread of control -and the <a href="../api_reference/C/memptrickle.html" class="olink">DB_ENV->memp_trickle()</a> method to trickle-write pages can often increase -the overall throughput of the application.</dd> + <dd> + In some applications, the cache is sufficiently + active and dirty that readers frequently need to write + a dirty page in order to have space in which to read a + new page from the backing database file. You can use + the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility (or the statistics returned by the + <a href="../api_reference/C/mempstat.html" class="olink">DB_ENV->memp_stat()</a> method) to see how often this is happening + in your application's cache. In this case, using a + separate thread of control and the <a href="../api_reference/C/memptrickle.html" class="olink">DB_ENV->memp_trickle()</a> + method to trickle-write pages can often increase the + overall throughput of the application. + </dd> </dl> </div> </div> @@ -242,7 +305,8 @@ the overall throughput of the application.</dd> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Transaction throughput</td> + <td width="40%" align="right" valign="top"> Transaction + throughput</td> </tr> </table> </div> diff --git a/docs/programmer_reference/transapp_why.html b/docs/programmer_reference/transapp_why.html index b8573e72..f3f381b2 100644 --- a/docs/programmer_reference/transapp_why.html +++ b/docs/programmer_reference/transapp_why.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_term.html">Next</a></td> </tr> </table> @@ -38,33 +36,47 @@ </div> </div> </div> - <p>Perhaps the first question to answer is "Why transactions?" There are -a number of reasons to include transactional support in your applications. -The most common ones are the following:</p> + <p> + Perhaps the first question to answer is "Why transactions?" + There are a number of reasons to include transactional support + in your applications. The most common ones are the + following: + </p> <div class="variablelist"> <dl> <dt> <span class="term">Recoverability</span> </dt> - <dd>Applications often need to ensure that no matter how the system or -application fails, previously saved data is available the next time the -application runs. This is often called Durability.</dd> + <dd> + Applications often need to ensure that no matter + how the system or application fails, previously saved + data is available the next time the application runs. + This is often called Durability. + </dd> <dt> <span class="term">Atomicity</span> </dt> - <dd>Applications may need to make multiple changes to one or more databases, -but ensure that either all of the changes happen, or none of them -happens. Transactions guarantee that a group of changes are atomic; -that is, if the application or system fails, either all of the changes -to the databases will appear when the application next runs, or none of -them.</dd> + <dd> + Applications may need to make multiple changes + to one or more databases, but ensure that either all + of the changes happen, or none of them happens. + Transactions guarantee that a group of changes are + atomic; that is, if the application or system fails, + either all of the changes to the databases will appear + when the application next runs, or none of + them. + </dd> <dt> <span class="term">Isolation</span> </dt> - <dd>Applications may need to make changes in isolation, that is, ensure that -only a single thread of control is modifying a key/data pair at a time. -Transactions ensure each thread of control sees all records as if all -other transactions either completed before or after its transaction.</dd> + <dd> + Applications may need to make changes in + isolation, that is, ensure that only a single thread + of control is modifying a key/data pair at a time. + Transactions ensure each thread of control sees all + records as if all other transactions either completed + before or after its transaction. + </dd> </dl> </div> </div> @@ -79,9 +91,7 @@ other transactions either completed before or after its transaction.</dd> <td width="40%" align="right"> <a accesskey="n" href="transapp_term.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 11. - Berkeley DB Transactional Data Store Applications - </td> + <td width="40%" align="left" valign="top">Chapter 11. Berkeley DB Transactional Data Store Applications </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/txn.html b/docs/programmer_reference/txn.html index 3dd7eed7..35cded33 100644 --- a/docs/programmer_reference/txn.html +++ b/docs/programmer_reference/txn.html @@ -14,13 +14,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 19. - The Transaction Subsystem - </th> + <th colspan="3" align="center">Chapter 19. The Transaction Subsystem </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="mp_warm.html">Prev</a> </td> @@ -34,9 +32,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="txn"></a>Chapter 19. - The Transaction Subsystem - </h2> + <h2 class="title"><a id="txn"></a>Chapter 19. The Transaction Subsystem </h2> </div> </div> </div> @@ -64,17 +60,17 @@ <dl> <dt> <span class="sect2"> - <a href="txn_limits.html#idp3173248">Transaction IDs</a> + <a href="txn_limits.html#idp2936928">Transaction IDs</a> </span> </dt> <dt> <span class="sect2"> - <a href="txn_limits.html#idp3049928">Cursors</a> + <a href="txn_limits.html#idp2856976">Cursors</a> </span> </dt> <dt> <span class="sect2"> - <a href="txn_limits.html#idp3082656">Multiple Threads of Control</a> + <a href="txn_limits.html#idp2907576">Multiple Threads of Control</a> </span> </dt> </dl> @@ -89,64 +85,100 @@ </div> </div> </div> - <p>The Transaction subsystem makes operations atomic, consistent, isolated, -and durable in the face of system and application failures. The subsystem -requires that the data be properly logged and locked in order to attain -these properties. Berkeley DB contains all the components necessary to -transaction-protect the Berkeley DB access methods, and other forms of data may -be protected if they are logged and locked appropriately.</p> - <p>The Transaction subsystem is created, initialized, and opened by calls to -<a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> with the <a href="../api_reference/C/envopen.html#envopen_DB_INIT_TXN" class="olink">DB_INIT_TXN</a> flag specified. Note -that enabling transactions automatically enables logging, but does not -enable locking because a single thread of control that needed atomicity -and recoverability would not require it.</p> - <p>The <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a> function starts a transaction, returning an opaque -handle to a transaction. If the parent parameter to <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a> is -non-NULL, the new transaction is a child of the designated parent -transaction.</p> - <p>The <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> function ends the designated transaction and causes -all updates performed by the transaction to be undone. The end result is -that the database is left in a state identical to the state that existed -prior to the <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a>. If the aborting transaction has any child -transactions associated with it (even ones that have already been -committed), they are also aborted. Any transactions that are unresolved -(neither committed nor aborted) when the application or system fails -are aborted during recovery.</p> - <p>The <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a> function ends the designated transaction and makes -all the updates performed by the transaction permanent, even in the face -of application or system failure. If this is a parent transaction -committing, all child transactions that individually committed or -had not been resolved are also committed.</p> - <p>Transactions are identified by 32-bit unsigned integers. The ID -associated with any transaction can be obtained using the <a href="../api_reference/C/txnid.html" class="olink">DB_TXN->id()</a> -function. If an application is maintaining information outside of Berkeley DB -it wants to transaction-protect, it should use this transaction ID as -the locking ID.</p> - <p>The <a href="../api_reference/C/txncheckpoint.html" class="olink">DB_ENV->txn_checkpoint()</a> function causes a transaction checkpoint. A -checkpoint is performed using to a specific log sequence number (LSN), -referred to as the checkpoint LSN. When a checkpoint completes -successfully, it means that all data buffers whose updates are described -by LSNs less than the checkpoint LSN have been written to disk. This, in -turn, means that the log records less than the checkpoint LSN are no -longer necessary for normal recovery (although they would be required for -catastrophic recovery if the database files were lost), and all log files -containing only records prior to the checkpoint LSN may be safely archived -and removed.</p> - <p>The time required to run normal recovery is proportional to the amount -of work done between checkpoints. If a large number of modifications -happen between checkpoints, many updates recorded in the log may -not have been written to disk when failure occurred, and recovery may -take longer to run. Generally, if the interval between checkpoints is -short, data may be being written to disk more frequently, but the -recovery time will be shorter. Often, the checkpoint interval is tuned -for each specific application.</p> - <p>The <a href="../api_reference/C/txnstat.html" class="olink">DB_TXN->stat()</a> method returns information about the status of the -transaction subsystem. It is the programmatic interface used by the -<a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility.</p> - <p>The transaction system is closed by a call to <a href="../api_reference/C/envclose.html" class="olink">DB_ENV->close()</a>.</p> - <p>Finally, the entire transaction system may be removed using the -<a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> method.</p> - <p>For more information on the transaction subsystem methods, see the <a href="../api_reference/C/txn.html#txnlist" class="olink">Transaction Subsystem and Related Methods</a> section in the <em class="citetitle">Berkeley DB C API Reference Guide.</em> </p> + <p> + The Transaction subsystem makes operations atomic, + consistent, isolated, and durable in the face of system and + application failures. The subsystem requires that the data be + properly logged and locked in order to attain these + properties. Berkeley DB contains all the components necessary + to transaction-protect the Berkeley DB access methods, and + other forms of data may be protected if they are logged and + locked appropriately. + </p> + <p> + The Transaction subsystem is created, initialized, and + opened by calls to <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> with the <a href="../api_reference/C/envopen.html#envopen_DB_INIT_TXN" class="olink">DB_INIT_TXN</a> flag + specified. Note that enabling transactions automatically + enables logging, but does not enable locking because a single + thread of control that needed atomicity and recoverability + would not require it. + </p> + <p> + The <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a> function starts a transaction, returning an + opaque handle to a transaction. If the parent parameter to + <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a> is non-NULL, the new transaction is a child of the + designated parent transaction. + </p> + <p> + The <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> function ends the designated transaction and + causes all updates performed by the transaction to be undone. + The end result is that the database is left in a state + identical to the state that existed prior to the <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a>. + If the aborting transaction has any child transactions + associated with it (even ones that have already been + committed), they are also aborted. Any transactions that are + unresolved (neither committed nor aborted) when the + application or system fails are aborted during + recovery. + </p> + <p> + The <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a> function ends the designated transaction and + makes all the updates performed by the transaction permanent, + even in the face of application or system failure. If this is + a parent transaction committing, all child transactions that + individually committed or had not been resolved are also + committed. + </p> + <p> + Transactions are identified by 32-bit unsigned integers. The + ID associated with any transaction can be obtained using the + <a href="../api_reference/C/txnid.html" class="olink">DB_TXN->id()</a> function. If an application is maintaining information + outside of Berkeley DB it wants to transaction-protect, it + should use this transaction ID as the locking ID. + </p> + <p> + The <a href="../api_reference/C/txncheckpoint.html" class="olink">DB_ENV->txn_checkpoint()</a> function causes a transaction + checkpoint. A checkpoint is performed using to a specific log + sequence number (LSN), referred to as the checkpoint LSN. When + a checkpoint completes successfully, it means that all data + buffers whose updates are described by LSNs less than the + checkpoint LSN have been written to disk. This, in turn, means + that the log records less than the checkpoint LSN are no + longer necessary for normal recovery (although they would be + required for catastrophic recovery if the database files were + lost), and all log files containing only records prior to the + checkpoint LSN may be safely archived and removed. + </p> + <p> + The time required to run normal recovery is proportional to + the amount of work done between checkpoints. If a large number + of modifications happen between checkpoints, many updates + recorded in the log may not have been written to disk when + failure occurred, and recovery may take longer to run. + Generally, if the interval between checkpoints is short, data + may be being written to disk more frequently, but the recovery + time will be shorter. Often, the checkpoint interval is tuned + for each specific application. + </p> + <p> + The <a href="../api_reference/C/txnstat.html" class="olink">DB_TXN->stat()</a> method returns information about the status of + the transaction subsystem. It is the programmatic interface + used by the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility. + </p> + <p> + The transaction system is closed by a call to + <a href="../api_reference/C/envclose.html" class="olink">DB_ENV->close()</a>. + </p> + <p> + Finally, the entire transaction system may be removed using + the <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> method. + </p> + <p> + For more information on the transaction subsystem methods, + see the <a href="../api_reference/C/txn.html#txnlist" class="olink"> + Transaction Subsystem and Related Methods</a> section + in the <em class="citetitle">Berkeley DB C API Reference Guide.</em> + </p> </div> </div> <div class="navfooter"> diff --git a/docs/programmer_reference/txn_config.html b/docs/programmer_reference/txn_config.html index a7b3c5f1..69783439 100644 --- a/docs/programmer_reference/txn_config.html +++ b/docs/programmer_reference/txn_config.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="txn.html">Prev</a> </td> - <th width="60%" align="center">Chapter 19. - The Transaction Subsystem - </th> + <th width="60%" align="center">Chapter 19. The Transaction Subsystem </th> <td width="20%" align="right"> <a accesskey="n" href="txn_limits.html">Next</a></td> </tr> </table> @@ -38,33 +36,43 @@ </div> </div> </div> - <p>The application may change the number of simultaneous outstanding -transactions supported by the Berkeley DB environment by calling the -<a href="../api_reference/C/envset_tx_max.html" class="olink">DB_ENV->set_tx_max()</a> method. This will also set the size of the -underlying transaction subsystem's region. When the number of -outstanding transactions is reached, additional calls to -<a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a> will fail until some active transactions complete.</p> - <p>The application can limit how long a transaction runs or blocks on -contested resources. -The <a href="../api_reference/C/envset_timeout.html" class="olink">DB_ENV->set_timeout()</a> method specifies the length of the timeout. -This value is checked whenever deadlock detection is performed or -when the transaction is about to block on a lock that cannot be -immediately granted. -Because timeouts are only checked at these times, the accuracy of the -timeout depends on how often deadlock detection is performed or how -frequently the transaction blocks.</p> - <p>There is an additional parameter used in configuring transactions; the -<a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a>. Setting the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> flag to -<a href="../api_reference/C/envset_flags.html" class="olink">DB_ENV->set_flags()</a> when opening a transaction region changes the -behavior of transactions to not write or synchronously flush the log -during transaction commit.</p> - <p>This change may significantly increase application transactional -throughput. However, it means that although transactions will continue -to exhibit the ACI (atomicity, consistency, and isolation) properties, -they will not have D (durability). Database integrity will be -maintained, but it is possible that some number of the most recently -committed transactions may be undone during recovery instead of being -redone.</p> + <p> + The application may change the number of simultaneous + outstanding transactions supported by the Berkeley DB + environment by calling the <a href="../api_reference/C/envset_tx_max.html" class="olink">DB_ENV->set_tx_max()</a> method. This will + also set the size of the underlying transaction subsystem's + region. When the number of outstanding transactions is + reached, additional calls to <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a> will fail until some + active transactions complete. + </p> + <p> + The application can limit how long a transaction runs or + blocks on contested resources. The <a href="../api_reference/C/envset_timeout.html" class="olink">DB_ENV->set_timeout()</a> method + specifies the length of the timeout. This value is checked + whenever deadlock detection is performed or when the + transaction is about to block on a lock that cannot be + immediately granted. Because timeouts are only checked at + these times, the accuracy of the timeout depends on how often + deadlock detection is performed or how frequently the + transaction blocks. + </p> + <p> + There is an additional parameter used in configuring + transactions; the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a>. Setting the <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a> + flag to <a href="../api_reference/C/envset_flags.html" class="olink">DB_ENV->set_flags()</a> when opening a transaction region + changes the behavior of transactions to not write or + synchronously flush the log during transaction commit. + </p> + <p> + This change may significantly increase application + transactional throughput. However, it means that although + transactions will continue to exhibit the ACI (atomicity, + consistency, and isolation) properties, they will not have D + (durability). Database integrity will be maintained, but it is + possible that some number of the most recently committed + transactions may be undone during recovery instead of being + redone. + </p> </div> <div class="navfooter"> <hr /> @@ -77,9 +85,7 @@ redone.</p> <td width="40%" align="right"> <a accesskey="n" href="txn_limits.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Chapter 19. - The Transaction Subsystem - </td> + <td width="40%" align="left" valign="top">Chapter 19. The Transaction Subsystem </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/txn_limits.html b/docs/programmer_reference/txn_limits.html index 5c8e87f8..c30639c6 100644 --- a/docs/programmer_reference/txn_limits.html +++ b/docs/programmer_reference/txn_limits.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="txn_config.html">Prev</a> </td> - <th width="60%" align="center">Chapter 19. - The Transaction Subsystem - </th> + <th width="60%" align="center">Chapter 19. The Transaction Subsystem </th> <td width="20%" align="right"> <a accesskey="n" href="sequence.html">Next</a></td> </tr> </table> @@ -42,17 +40,17 @@ <dl> <dt> <span class="sect2"> - <a href="txn_limits.html#idp3173248">Transaction IDs</a> + <a href="txn_limits.html#idp2936928">Transaction IDs</a> </span> </dt> <dt> <span class="sect2"> - <a href="txn_limits.html#idp3049928">Cursors</a> + <a href="txn_limits.html#idp2856976">Cursors</a> </span> </dt> <dt> <span class="sect2"> - <a href="txn_limits.html#idp3082656">Multiple Threads of Control</a> + <a href="txn_limits.html#idp2907576">Multiple Threads of Control</a> </span> </dt> </dl> @@ -61,58 +59,71 @@ <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp3173248"></a>Transaction IDs</h3> + <h3 class="title"><a id="idp2936928"></a>Transaction IDs</h3> </div> </div> </div> - <p>Transactions are identified by 31-bit unsigned integers, which means -there are just over two billion unique transaction IDs. When a database -environment is initially created or recovery is run, the transaction ID -name space is reset, and new transactions are numbered starting from -0x80000000 (2,147,483,648). The IDs will wrap if the maximum -transaction ID is reached, starting again from 0x80000000. The most -recently allocated transaction ID is the <span class="bold"><strong>st_last_txnid</strong></span> value in -the transaction statistics information, and can be displayed by the -<a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility.</p> + <p> + Transactions are identified by 31-bit unsigned integers, + which means there are just over two billion unique + transaction IDs. When a database environment is initially + created or recovery is run, the transaction ID name space + is reset, and new transactions are numbered starting from + 0x80000000 (2,147,483,648). The IDs will wrap if the + maximum transaction ID is reached, starting again from + 0x80000000. The most recently allocated transaction ID is + the <span class="bold"><strong>st_last_txnid</strong></span> value + in the transaction statistics information, and can be + displayed by the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp3049928"></a>Cursors</h3> + <h3 class="title"><a id="idp2856976"></a>Cursors</h3> </div> </div> </div> - <p>When using transactions, cursors are localized to a single transaction. -That is, a cursor may not span transactions, and must be opened and -closed within a single transaction. In addition, intermingling -transaction-protected cursor operations and non-transaction-protected -cursor operations on the same database in a single thread of control is -practically guaranteed to deadlock because the locks obtained for -transactions and non-transactions can conflict.</p> + <p> + When using transactions, cursors are localized to a + single transaction. That is, a cursor may not span + transactions, and must be opened and closed within a + single transaction. In addition, intermingling + transaction-protected cursor operations and + non-transaction-protected cursor operations on the same + database in a single thread of control is practically + guaranteed to deadlock because the locks obtained for + transactions and non-transactions can conflict. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp3082656"></a>Multiple Threads of Control</h3> + <h3 class="title"><a id="idp2907576"></a>Multiple Threads of Control</h3> </div> </div> </div> - <p>Because transactions must hold all their locks until commit, a single -transaction may accumulate a large number of long-term locks during its -lifetime. As a result, when two concurrently running transactions -access the same database, there is strong potential for conflict. -Although Berkeley DB allows an application to have multiple outstanding -transactions active within a single thread of control, great care must -be taken to ensure that the transactions do not block each other (for -example, attempt to obtain conflicting locks on the same data). If two -concurrently active transactions in the same thread of control do -encounter a lock conflict, the thread of control will deadlock so that -the deadlock detector cannot detect the problem. In this case, there -is no true deadlock, but because the transaction on which a transaction -is waiting is in the same thread of control, no forward progress can be -made.</p> + <p> + Because transactions must hold all their locks until + commit, a single transaction may accumulate a large number + of long-term locks during its lifetime. As a result, when + two concurrently running transactions access the same + database, there is strong potential for conflict. Although + Berkeley DB allows an application to have multiple + outstanding transactions active within a single thread of + control, great care must be taken to ensure that the + transactions do not block each other (for example, attempt + to obtain conflicting locks on the same data). If two + concurrently active transactions in the same thread of + control do encounter a lock conflict, the thread of + control will deadlock so that the deadlock detector cannot + detect the problem. In this case, there is no true + deadlock, but because the transaction on which a + transaction is waiting is in the same thread of control, + no forward progress can be made. + </p> </div> </div> <div class="navfooter"> @@ -130,9 +141,7 @@ made.</p> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 20. - Sequences - </td> + <td width="40%" align="right" valign="top"> Chapter 20. Sequences </td> </tr> </table> </div> diff --git a/docs/programmer_reference/witold.html b/docs/programmer_reference/witold.html index b597466f..b18eef88 100644 --- a/docs/programmer_reference/witold.html +++ b/docs/programmer_reference/witold.html @@ -1,5 +1,5 @@ <!--$Id: witold.so,v 10.5 2001/06/09 14:34:45 bostic Exp $--> -<!--Copyright (c) 1997,2009 Oracle. All rights reserved.--> +<!--Copyright (c) 1997, 2014 Oracle. All rights reserved.--> <!--See the file LICENSE for redistribution information.--> <html> <head> diff --git a/docs/programmer_reference/xa.html b/docs/programmer_reference/xa.html index ab53b95e..e5294b9c 100644 --- a/docs/programmer_reference/xa.html +++ b/docs/programmer_reference/xa.html @@ -14,13 +14,11 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Chapter 13. - Distributed Transactions - </th> + <th colspan="3" align="center">Chapter 13. Distributed Transactions </th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="rep_ex_chan.html">Prev</a> </td> @@ -34,9 +32,7 @@ <div class="titlepage"> <div> <div> - <h2 class="title"><a id="xa"></a>Chapter 13. - Distributed Transactions - </h2> + <h2 class="title"><a id="xa"></a>Chapter 13. Distributed Transactions </h2> </div> </div> </div> @@ -64,32 +60,27 @@ <dl> <dt> <span class="sect2"> - <a href="xa_build.html#idp2599152">Communicating with multiple Berkeley DB environments</a> + <a href="xa_build.html#idp2310216">Communicating with multiple Berkeley DB environments</a> </span> </dt> <dt> <span class="sect2"> - <a href="xa_build.html#idp2600096">Recovering from GTM failure</a> + <a href="xa_build.html#idp2311288">Managing the Global Transaction ID (GID) name space</a> </span> </dt> <dt> <span class="sect2"> - <a href="xa_build.html#idp2584512">Managing the Global Transaction ID (GID) name space</a> + <a href="xa_build.html#idp2293376">Maintaining state for each distributed transaction.</a> </span> </dt> <dt> <span class="sect2"> - <a href="xa_build.html#idp2523640">Maintaining state for each distributed transaction.</a> + <a href="xa_build.html#idp2226880">Recovering from the failure of a single environment</a> </span> </dt> <dt> <span class="sect2"> - <a href="xa_build.html#idp2597672">Recovering from the failure of a single environment</a> - </span> - </dt> - <dt> - <span class="sect2"> - <a href="xa_build.html#idp2600560">Recovering from GTM failure</a> + <a href="xa_build.html#idp2287824">Recovering from GTM failure</a> </span> </dt> </dl> @@ -108,17 +99,17 @@ <dl> <dt> <span class="sect2"> - <a href="xa_xa_config.html#idp2607440">Update the Resource Manager File in Tuxedo</a> + <a href="xa_xa_config.html#idp2312920">Update the Resource Manager File in Tuxedo</a> </span> </dt> <dt> <span class="sect2"> - <a href="xa_xa_config.html#idp2633056">Build the Transaction Manager Server</a> + <a href="xa_xa_config.html#idp2341840">Build the Transaction Manager Server</a> </span> </dt> <dt> <span class="sect2"> - <a href="xa_xa_config.html#idp2580168">Update the UBBCONFIG File</a> + <a href="xa_xa_config.html#idp1702176">Update the UBBCONFIG File</a> </span> </dt> </dl> @@ -143,19 +134,25 @@ </div> </div> </div> - <p>An application must use distributed transactions whenever it wants transactional semantics either -across operations in multiple Berkeley DB environments (even if they are on the same machine) -or across operations in Berkeley DB and some other database systems (for example, Oracle server). -Berkeley DB provides support for distributed transactions using a two-phase -commit protocol. In order to use the two-phase commit feature of Berkeley DB, an application -must either implement its own global transaction manager or use an -XA-compliant transaction manager such as Oracle Tuxedo (as Berkeley DB can act as an XA-compliant -resource manager). -</p> - <p>This chapter explains Berkeley DB's XA-compliant resource manager, which can be used in any X/Open -distributed transaction processing system, and explains how to configure -Oracle Tuxedo to use the Berkeley DB resource manager. -</p> + <p> + An application must use distributed transactions whenever + it wants transactional semantics either across operations in + multiple Berkeley DB environments (even if they are on the + same machine) or across operations in Berkeley DB and some + other database systems (for example, Oracle server). Berkeley DB + provides support for distributed transactions using a + two-phase commit protocol. In order to use the two-phase + commit feature of Berkeley DB, an application must either + implement its own global transaction manager or use an + XA-compliant transaction manager such as Oracle Tuxedo + (as Berkeley DB can act as an XA-compliant resource manager). + </p> + <p> + This chapter explains Berkeley DB's XA-compliant resource manager, + which can be used in any X/Open distributed transaction + processing system, and explains how to configure Oracle Tuxedo + to use the Berkeley DB resource manager. + </p> </div> </div> <div class="navfooter"> @@ -167,8 +164,7 @@ Oracle Tuxedo to use the Berkeley DB resource manager. <td width="40%" align="right"> <a accesskey="n" href="ch13s02.html">Next</a></td> </tr> <tr> - <td width="40%" align="left" valign="top">Ex_rep_chan: a Replication Manager -channel example </td> + <td width="40%" align="left" valign="top">Ex_rep_chan: a Replication Manager channel example </td> <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> diff --git a/docs/programmer_reference/xa_build.html b/docs/programmer_reference/xa_build.html index 09ae7049..9085e6f4 100644 --- a/docs/programmer_reference/xa_build.html +++ b/docs/programmer_reference/xa_build.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="ch13s02.html">Prev</a> </td> - <th width="60%" align="center">Chapter 13. - Distributed Transactions - </th> + <th width="60%" align="center">Chapter 13. Distributed Transactions </th> <td width="20%" align="right"> <a accesskey="n" href="xa_xa_intro.html">Next</a></td> </tr> </table> @@ -42,166 +40,161 @@ <dl> <dt> <span class="sect2"> - <a href="xa_build.html#idp2599152">Communicating with multiple Berkeley DB environments</a> + <a href="xa_build.html#idp2310216">Communicating with multiple Berkeley DB environments</a> </span> </dt> <dt> <span class="sect2"> - <a href="xa_build.html#idp2600096">Recovering from GTM failure</a> + <a href="xa_build.html#idp2311288">Managing the Global Transaction ID (GID) name space</a> </span> </dt> <dt> <span class="sect2"> - <a href="xa_build.html#idp2584512">Managing the Global Transaction ID (GID) name space</a> + <a href="xa_build.html#idp2293376">Maintaining state for each distributed transaction.</a> </span> </dt> <dt> <span class="sect2"> - <a href="xa_build.html#idp2523640">Maintaining state for each distributed transaction.</a> + <a href="xa_build.html#idp2226880">Recovering from the failure of a single environment</a> </span> </dt> <dt> <span class="sect2"> - <a href="xa_build.html#idp2597672">Recovering from the failure of a single environment</a> - </span> - </dt> - <dt> - <span class="sect2"> - <a href="xa_build.html#idp2600560">Recovering from GTM failure</a> + <a href="xa_build.html#idp2287824">Recovering from GTM failure</a> </span> </dt> </dl> </div> - <p>Managing distributed transactions and using the two-phase commit -protocol of Berkeley DB from an application requires the application to provide -the functionality of a global transaction manager (GTM). The GTM is -responsible for the following:</p> + <p> + Managing distributed transactions and using the two-phase commit + protocol of Berkeley DB from an application requires the + application to provide the functionality of a global transaction + manager (GTM). The GTM is responsible for the following: + </p> <div class="itemizedlist"> <ul type="disc"> - <li>Communicating with the multiple environments (potentially on separate -systems).</li> - <li>Managing the global transaction ID name space.</li> - <li>Maintaining state information about each distributed transaction.</li> - <li>Recovering from failures of individual environments.</li> - <li>Recovering the global transaction state after failure of the global -transaction manager.</li> + <li> + Communicating with the multiple environments (potentially on separate + systems). + </li> + <li> + Managing the global transaction ID name space. + </li> + <li> + Maintaining state information about each distributed transaction. + </li> + <li> + Recovering from failures of individual environments. + </li> + <li> + Recovering the global transaction state after failure of the global + transaction manager. + </li> </ul> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2599152"></a>Communicating with multiple Berkeley DB environments</h3> + <h3 class="title"><a id="idp2310216"></a>Communicating with multiple Berkeley DB environments</h3> </div> </div> </div> - <p>Two-phase commit is required if a transaction spans operations -in multiple Berkeley DB environments or across operations in Berkeley DB -and some other database systems. If the environments -reside on the same machine, the application can communicate with each -environment through its own address space with no additional complexity. -If the environments reside on separate machines, the application - -may use its own messaging capability, translating messages on the remote -machine into calls into the Berkeley DB library (including the recovery -calls). </p> - </div> - <div class="sect2" lang="en" xml:lang="en"> - <div class="titlepage"> - <div> - <div> - <h3 class="title"><a id="idp2600096"></a>Recovering from GTM failure</h3> - </div> - </div> - </div> - <p>If the GTM fails, it must first recover its local state. Assuming the -GTM uses Berkeley DB tables to maintain state, it should run the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility (or the DB_RECOVER option to <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a>) upon startup. Once the GTM is back up and running, it needs to review all its outstanding global transactions, that is, all transactions that are recorded but not yet completed.</p> <p> -The global transactions that have not yet reached the prepare phase should be aborted. For each global transaction that has not yet prepared, the GTM should send a message to each participant telling it to abort its transaction.</p> - <p>Next the GTM should review its log to identify all participating environments that have transactions in the preparing, aborting, or committing states. For each such participant, the GTM should issue a <a href="../api_reference/C/txnrecover.html" class="olink">DB_ENV->txn_recover()</a> call. Upon receiving responses from each participant, the GTM must decide the fate of each transaction and issue appropriate calls. The correct behavior is defined as follows:</p> - <div class="variablelist"> - <dl> - <dt> - <span class="term">preparing</span> - </dt> - <dd>if all participating environments return the transaction in the list of prepared but not yet completed transactions, then the GTM should commit the transaction. If any participating environment fails to return the transaction in this list, then the GTM must issue an abort to all environments participating in that global transaction.</dd> - <dt> - <span class="term">committing</span> - </dt> - <dd>the GTM should send a commit to any environment that returned this -transaction in its list of prepared but not yet completed -transactions.</dd> - <dt> - <span class="term">aborting</span> - </dt> - <dd>the GTM should send an abort to any environment that returned this -transaction in its list of prepared but not yet completed -transactions.</dd> - </dl> - </div> + Two-phase commit is required if a transaction spans operations + in multiple Berkeley DB environments or across operations in + Berkeley DB and some other database systems. If the + environments reside on the same machine, the application can + communicate with each environment through its own address space + with no additional complexity. If the environments reside on + separate machines, the application may use its own messaging + capability, translating messages on the remote machine into + calls into the Berkeley DB library (including the recovery + calls). + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2584512"></a>Managing the Global Transaction ID (GID) name space</h3> + <h3 class="title"><a id="idp2311288"></a>Managing the Global Transaction ID (GID) name space</h3> </div> </div> </div> - <p>A global transaction is a transaction that spans multiple environments. -Each global transaction must have a unique transaction ID. This unique -ID is the global transaction ID (GID). In Berkeley DB, global transaction IDs -must be represented by the character array, <code class="literal">DB_GID_SIZE</code> (currently 128 bytes). -It is the responsibility of the -global transaction manager to assign GIDs, guarantee their uniqueness, -and manage the mapping of local transactions to GID. That is, for each -GID, the GTM should know which local transaction managers participated. -The Berkeley DB logging system or a Berkeley DB table could be used to record this -information.</p> + <p> + A global transaction is a transaction that spans multiple + environments. Each global transaction must have a unique + transaction ID. This unique ID is the global transaction ID + (GID). In Berkeley DB, global transaction IDs must be + represented by the character array, + <code class="literal">DB_GID_SIZE</code> (currently 128 bytes). It is + the responsibility of the global transaction manager to assign + GIDs, guarantee their uniqueness, and manage the mapping of + local transactions to GID. That is, for each GID, the GTM + should know which local transaction managers participated. The + Berkeley DB logging system or a Berkeley DB table could be used + to record this information. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2523640"></a>Maintaining state for each distributed transaction.</h3> + <h3 class="title"><a id="idp2293376"></a>Maintaining state for each distributed transaction.</h3> </div> </div> </div> - <p>In addition to knowing which local environments participate in each -global transaction, the GTM must also know the state of each active -global transaction. As soon as a transaction becomes distributed (that -is, a second environment participates), the GTM must record the -existence of the global transaction and all participants (whether this -must reside on stable storage or not depends on the exact configuration -of the system). As new environments participate, the GTM must keep this -information up to date.</p> - <p>When the GTM is ready to begin commit processing, it should issue -<a href="../api_reference/C/txnprepare.html" class="olink">DB_TXN->prepare()</a> calls to each participating environment, indicating -the GID of the global transaction. Once all the participants have -successfully prepared, then the GTM must record that the global -transaction will be committed. This record should go to stable -storage. Once written to stable storage, the GTM can send -<a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a> requests to each participating environment. Once -all environments have successfully completed the commit, the GTM can -either record the successful commit or can somehow "forget" the global -transaction.</p> <p> -If an application uses nested transactions (that is, the -parent parameter is non-NULL in a call to <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a>) - then, only the parent transaction should call <a href="../api_reference/C/txnprepare.html" class="olink">DB_TXN->prepare()</a>, not any of the child transactions. -</p> - <p>Should any participant fail to prepare, then the GTM must abort the -global transaction. The fact that the transaction is going to be -aborted should be written to stable storage. Once written, the GTM can -then issue <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> requests to each environment. When all -aborts have returned successfully, the GTM can either record the -successful abort or "forget" the global transaction.</p> - <p>In summary, for each transaction, the GTM must maintain the following:</p> + In addition to knowing which local environments participate in + each global transaction, the GTM must also know the state of + each active global transaction. As soon as a transaction + becomes distributed (that is, a second environment + participates), the GTM must record the existence of the global + transaction and all participants (whether this must reside on + stable storage or not depends on the exact configuration of the + system). As new environments participate, the GTM must keep + this information up to date. + </p> + <p> + When the GTM is ready to begin commit processing, it should + issue <a href="../api_reference/C/txnprepare.html" class="olink">DB_TXN->prepare()</a> calls to each participating environment, + indicating the GID of the global transaction. Once all the + participants have successfully prepared, then the GTM must + record that the global transaction will be committed. This + record should go to stable storage. Once written to stable + storage, the GTM can send <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a> requests to each + participating environment. Once all environments have + successfully completed the commit, the GTM can either record + the successful commit or can somehow "forget" the global + transaction. + </p> + <p> + If an application uses nested transactions (that is, the parent + parameter is non-NULL in a call to <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a>) then, only the + parent transaction should call <a href="../api_reference/C/txnprepare.html" class="olink">DB_TXN->prepare()</a>, not any of the + child transactions. + </p> + <p> + Should any participant fail to prepare, then the GTM must abort + the global transaction. The fact that the transaction is going + to be aborted should be written to stable storage. Once + written, the GTM can then issue <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a> requests to each + environment. When all aborts have returned successfully, the + GTM can either record the successful abort or "forget" the + global transaction. + </p> + <p> + In summary, for each transaction, the GTM must maintain the following: + </p> <div class="itemizedlist"> <ul type="disc"> - <li>A list of participating environments</li> - <li>The current state of each transaction (pre-prepare, preparing, -committing, aborting, done)</li> + <li> + A list of participating environments + </li> + <li> + The current state of each transaction (pre-prepare, preparing, + committing, aborting, done) + </li> </ul> </div> </div> @@ -209,50 +202,151 @@ committing, aborting, done)</li> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2597672"></a>Recovering from the failure of a single environment</h3> + <h3 class="title"><a id="idp2226880"></a>Recovering from the failure of a single environment</h3> </div> </div> </div> - <p>If a single environment fails, there is no need to bring down or recover other environments (the only exception to this is if all environments are managed in the same application address space and there is a risk that the failure of the environment corrupted other environments). Instead, once the failing environment comes back up, it should be recovered (that is, conventional recovery, via the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility or by specifying the <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> flag to <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> should be run). If the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility is used, then the -e option must be specified. In this case, the application will almost certainly want to specify environmental parameters via a <a class="xref" href="env_db_config.html" title="DB_CONFIG configuration file">DB_CONFIG configuration file</a> in the environment's home directory, so that the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility can create an appropriately configured environment. -If the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility is not used, then the GTM should call <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> specifying the <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> flag. It should then call <a href="../api_reference/C/txnrecover.html" class="olink">DB_ENV->txn_recover()</a>, which will return an array of DB_TXN handles for the set of prepared, but not yet completed transactions. For each transaction, the GTM should combine this knowledge with its transaction state table and call either <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a> or <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a>. After that process is complete, the environment is ready to participate in new transactions. -</p> - <p>If the GTM is running in a system with multiple GTMs, it is possible that some of the transactions returned via <a href="../api_reference/C/txnrecover.html" class="olink">DB_ENV->txn_recover()</a> do not belong to the current environment. The GTM should detect this and call <a href="../api_reference/C/txndiscard.html" class="olink">DB_TXN->discard()</a> on each such transaction handle. Furthermore, it is important to note the environment does not retain information about which GTM has issued <a href="../api_reference/C/txnrecover.html" class="olink">DB_ENV->txn_recover()</a> operations. Therefore, each GTM should issue all its <a href="../api_reference/C/txnrecover.html" class="olink">DB_ENV->txn_recover()</a> calls before another GTM issues its calls. If the calls are interleaved, each GTM may not get a complete and consistent set of transactions. The simplest way to enforce this is for each GTM to make sure it can receive all its outstanding transactions in a single <a href="../api_reference/C/txnrecover.html" class="olink">DB_ENV->txn_recover()</a> call. The maximum number of possible outstanding transactions is roughly the maximum number of active transactions in the environment (whose value can be obtained using the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility). To simplify this procedure, the caller should allocate an array large enough to hold the list of transactions (for example, allocate an array able to hold three times the maximum number of transactions). If that is not possible, callers should check that the array was not completely filled in when <a href="../api_reference/C/txnrecover.html" class="olink">DB_ENV->txn_recover()</a> returns. If the array was completely filled in, each transaction should be explicitly discarded, and the call repeated with a larger array.</p> - <p>The newly recovered environment will forbid any new transactions from being started until the prepared but not yet completed transactions have been resolved. In the multiple GTM case, this means that all GTMs must recover before any GTM can begin issuing new transactions.</p> <p> -The GTM must determine how long it needs to retain global transaction commit and abort records. If the participating environments are following a DB_TXN_SYNC policy, that is, they are forcing commit and abort records to disk before replying to the GTM, then once the GTM has heard from all participants, it need not retain its persistent log records. However, if participating environments are running at weaker durability levels, such as <a href="../api_reference/C/envset_flags.html#set_flags_DB_TXN_WRITE_NOSYNC" class="olink">DB_TXN_WRITE_NOSYNC</a> or <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a>, then the GTM must retain all commit and abort records until all participants have completed a checkpoint following the completion of a transaction. -</p> + If a single environment fails, there is no need to bring down + or recover other environments (the only exception to this is if + all environments are managed in the same application address + space and there is a risk that the failure of the environment + corrupted other environments). Instead, once the failing + environment comes back up, it should be conventionally recovered + using the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility or by specifying + the <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> flag to <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a>. + </p> + <p> + If the + <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility is used, then the -e option must be specified. In + this case, the application will almost certainly want to + specify environmental parameters using a + <a class="xref" href="env_db_config.html" title="DB_CONFIG configuration file">DB_CONFIG configuration + file</a> in + the environment's home directory, so that the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility can + create an appropriately configured environment. + </p> + <p> + If the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility is not used, then the GTM should call + <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> specifying the <a href="../api_reference/C/envopen.html#envopen_DB_RECOVER" class="olink">DB_RECOVER</a> flag. It should then + call <a href="../api_reference/C/txnrecover.html" class="olink">DB_ENV->txn_recover()</a>, which will return an array of DB_TXN handles + for the set of prepared, but not yet completed, transactions. + For each transaction, the GTM should combine this knowledge + with its transaction state table and call either <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a> + or <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a>. After that process is complete, the environment + is ready to participate in new transactions. + </p> + <p> + If the GTM is running in a system with multiple GTMs, it is + possible that some of the transactions returned by + <a href="../api_reference/C/txnrecover.html" class="olink">DB_ENV->txn_recover()</a> do not belong to the current environment. The GTM + should detect this and call <a href="../api_reference/C/txndiscard.html" class="olink">DB_TXN->discard()</a> on each such + transaction handle. + </p> + <p> + Be aware that the + environment does not retain information about which GTM has + issued <a href="../api_reference/C/txnrecover.html" class="olink">DB_ENV->txn_recover()</a> operations. Therefore, each GTM should + issue all its <a href="../api_reference/C/txnrecover.html" class="olink">DB_ENV->txn_recover()</a> calls before another GTM issues its + calls. If the calls are interleaved, each GTM may not get a + complete and consistent set of transactions. The simplest way + to enforce this is for each GTM to make sure it can receive all + its outstanding transactions in a single <a href="../api_reference/C/txnrecover.html" class="olink">DB_ENV->txn_recover()</a> call. + The maximum number of possible outstanding transactions is + roughly the maximum number of active transactions in the + environment (whose value can be obtained using the <a href="../api_reference/C/db_stat.html" class="olink">db_stat</a> utility). + To simplify this procedure, the caller should allocate an array + large enough to hold the list of transactions (for example, + allocate an array able to hold three times the maximum number + of transactions). If that is not possible, callers should + check that the array was not completely filled in when + <a href="../api_reference/C/txnrecover.html" class="olink">DB_ENV->txn_recover()</a> returns. If the array was completely filled in, + each transaction should be explicitly discarded, and the call + repeated with a larger array. + </p> + <p> + The newly recovered environment will forbid any new + transactions from being started until the prepared but not yet + completed transactions have been resolved. In the multiple GTM + case, this means that all GTMs must recover before any GTM can + begin issuing new transactions. + </p> + <p> + The GTM must determine how long it needs to retain global + transaction commit and abort records. If the participating + environments are following a DB_TXN_SYNC policy, that is, they + are forcing commit and abort records to disk before replying to + the GTM, then once the GTM has heard from all participants, it + need not retain its persistent log records. However, if + participating environments are running at weaker durability + levels, such as <a href="../api_reference/C/envset_flags.html#set_flags_DB_TXN_WRITE_NOSYNC" class="olink">DB_TXN_WRITE_NOSYNC</a> or <a href="../api_reference/C/envset_flags.html#envset_flags_DB_TXN_NOSYNC" class="olink">DB_TXN_NOSYNC</a>, then + the GTM must retain all commit and abort records until all + participants have completed a checkpoint following the + completion of a transaction. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2600560"></a>Recovering from GTM failure</h3> + <h3 class="title"><a id="idp2287824"></a>Recovering from GTM failure</h3> </div> </div> </div> - <p>If the GTM fails, it must first recover its local state. Assuming the -GTM uses Berkeley DB tables to maintain state, it should run the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility (or the DB_RECOVER option to <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a>) upon startup. Once the GTM is back up and running, it needs to review all its outstanding global transactions, that is, all transactions that are recorded but not yet completed.</p> <p> -The global transactions that have not yet reached the prepare phase should be aborted. For each global transaction that has not yet prepared, the GTM should send a message to each participant telling it to abort its transaction.</p> - <p>Next the GTM should review its log to identify all participating environments that have transactions in the preparing, aborting, or committing states. For each such participant, the GTM should issue a <a href="../api_reference/C/txnrecover.html" class="olink">DB_ENV->txn_recover()</a> call. Upon receiving responses from each participant, the GTM must decide the fate of each transaction and issue appropriate calls. The correct behavior is defined as follows:</p> + If the GTM fails, it must first recover its local state. + Assuming the GTM uses Berkeley DB tables to maintain state, it + should run the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility (or the DB_RECOVER option to + <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a>) upon startup. Once the GTM is back up and running, + it needs to review all its outstanding global transactions. + That is, all transactions that are recorded but not yet + completed. + </p> + <p> + The global transactions that have not yet reached the prepare + phase should be aborted. For each global transaction that has + not yet prepared, the GTM should send a message to each + participant telling it to abort its transaction. + </p> + <p> + Next the GTM should review its log to identify all + participating environments that have transactions in the + preparing, aborting, or committing states. For each such + participant, the GTM should issue a <a href="../api_reference/C/txnrecover.html" class="olink">DB_ENV->txn_recover()</a> call. Upon + receiving responses from each participant, the GTM must decide + the fate of each transaction and issue appropriate calls. The + correct behavior is defined as follows: + </p> <div class="variablelist"> <dl> <dt> <span class="term">preparing</span> </dt> - <dd>if all participating environments return the transaction in the list of prepared but not yet completed transactions, then the GTM should commit the transaction. If any participating environment fails to return the transaction in this list, then the GTM must issue an abort to all environments participating in that global transaction.</dd> + <dd> + If all participating environments return the + transaction in the list of prepared but not yet + completed transactions, then the GTM should commit the + transaction. If any participating environment fails to + return the transaction in this list, then the GTM must + issue an abort to all environments participating in + that global transaction. + </dd> <dt> <span class="term">committing</span> </dt> - <dd>the GTM should send a commit to any environment that returned this -transaction in its list of prepared but not yet completed -transactions.</dd> + <dd> + The GTM should send a commit to any environment + that returned this transaction in its list of prepared + but not yet completed transactions. + </dd> <dt> <span class="term">aborting</span> </dt> - <dd>the GTM should send an abort to any environment that returned this -transaction in its list of prepared but not yet completed -transactions.</dd> + <dd> + The GTM should send an abort to any environment + that returned this transaction in its list of prepared + but not yet completed transactions. + </dd> </dl> </div> </div> diff --git a/docs/programmer_reference/xa_faq.html b/docs/programmer_reference/xa_faq.html index cbc84922..29026a0a 100644 --- a/docs/programmer_reference/xa_faq.html +++ b/docs/programmer_reference/xa_faq.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="xa_xa_restrict.html">Prev</a> </td> - <th width="60%" align="center">Chapter 13. - Distributed Transactions - </th> + <th width="60%" align="center">Chapter 13. Distributed Transactions </th> <td width="20%" align="right"> <a accesskey="n" href="apprec.html">Next</a></td> </tr> </table> @@ -42,62 +40,105 @@ <ol type="1"> <li> <span class="bold"> - <strong>Is it possible to mix XA and non-XA transactions?</strong> + <strong>Is it possible to mix XA and non-XA + transactions?</strong> </span> - <p>Yes. It is also possible for XA and non-XA transactions to coexist in -the same Berkeley DB environment. To do this, specify the same environment -to the non-XA <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> calls as was specified in the Tuxedo -configuration file.</p> + <p> + Yes. It is also possible for XA and non-XA + transactions to coexist in the same Berkeley DB + environment. To do this, specify the same environment + to the non-XA <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> calls as was specified in the + Tuxedo configuration file. + </p> </li> <li> <span class="bold"> - <strong>Does converting an application to run within XA change any of -the already existing C/C++ API calls it does?</strong> + <strong>Does converting an application to + run within XA change any of the already existing C/C++ + API calls it does?</strong> </span> - <p>When converting an application to run under XA, the application's Berkeley DB calls are unchanged, with three exceptions:</p> + <p> + When converting an application to run under XA, the + application's Berkeley DB calls are unchanged, with + three exceptions: + </p> <div class="orderedlist"> <ol type="a"> - <li>The application must specify the <a href="../api_reference/C/dbcreate.html#dbcreate_DB_XA_CREATE" class="olink">DB_XA_CREATE</a> flag to -the <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a> function.</li> - <li>Unless the application is performing an operation for a non-XA transaction, the application should never explicitly call <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a>, <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a>, and <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a>, and those calls should be replaced by calls into the Tuxedo transaction manager.</li> - <li>Unless the application is performing an operation for a non-XA transaction, the application should specify a transaction argument of NULL to Berkeley DB methods taking transaction arguments (for example, <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> or <a href="../api_reference/C/dbcursor.html" class="olink">DB->cursor()</a>).</li> + <li> + The application must specify the + <a href="../api_reference/C/dbcreate.html#dbcreate_DB_XA_CREATE" class="olink">DB_XA_CREATE</a> flag to the <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a> + function. + </li> + <li> + Unless the application is performing an + operation for a non-XA transaction, the + application should never explicitly call + <a href="../api_reference/C/txncommit.html" class="olink">DB_TXN->commit()</a>, <a href="../api_reference/C/txnabort.html" class="olink">DB_TXN->abort()</a>, and <a href="../api_reference/C/txnbegin.html" class="olink">DB_ENV->txn_begin()</a>, and those + calls should be replaced by calls into the Tuxedo + transaction manager. + </li> + <li> + Unless the application is performing an + operation for a non-XA transaction, the + application should specify a transaction argument + of NULL to Berkeley DB methods taking transaction + arguments (for example, <a href="../api_reference/C/dbput.html" class="olink">DB->put()</a> or + <a href="../api_reference/C/dbcursor.html" class="olink">DB->cursor()</a>). + </li> </ol> </div> - <p>Otherwise, the application should be unchanged.</p> + <p> + Otherwise, the application should be + unchanged. + </p> </li> <li> <span class="bold"> - <strong>How does Berkeley DB recovery interact with recovery by the Tuxedo -transaction manager?</strong> + <strong>How does Berkeley DB recovery + interact with recovery by the Tuxedo transaction + manager?</strong> </span> - <p>Recovery is completed in two steps. First, each resource manager should -recover its environment(s). This can be done via a program that calls -<a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> or by calling the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility. If using the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility, then the option -should be specified so that the regions that are recovered persist after -the utility exits. Any transactions that were prepared, but neither -completed nor aborted, are restored to their prepared state so that they -may be aborted or committed via the Tuxedo recovery mechanisms. After -each resource manager has recovered, then Tuxedo recovery may begin. -Tuxedo will interact with each resource manager via the __db_xa_recover -function which returns the list of prepared, but not yet completed -transactions. It should issue a commit or abort for each one, and only -after having completed each transaction will normal processing resume.</p> - <p>Finally, standard log file archival and catastrophic recovery procedures -should occur independently of XA operation.</p> + <p> + Recovery is completed in two steps. First, each + resource manager should recover its environment(s). + This can be done via a program that calls <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> or + by calling the <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility. If using the + <a href="../api_reference/C/db_recover.html" class="olink">db_recover</a> utility, then the + teoption should be specified so that + the regions that are recovered persist after the + utility exits. Any transactions that were prepared, + but neither completed nor aborted, are restored to + their prepared state so that they may be aborted or + committed via the Tuxedo recovery mechanisms. After + each resource manager has recovered, then Tuxedo + recovery may begin. Tuxedo will interact with each + resource manager via the __db_xa_recover function + which returns the list of prepared, but not yet + completed transactions. It should issue a commit or + abort for each one, and only after having completed + each transaction will normal processing resume. + </p> + <p> + Finally, standard log file archival and catastrophic + recovery procedures should occur independently of XA + operation. + </p> </li> <li> <span class="bold"> - <strong>Does Berkeley DB provide multi-threaded support for XA transactions?</strong> + <strong>Does Berkeley DB provide + multi-threaded support for XA transactions?</strong> </span> - <p> -Yes. -For information on how to build multi-threaded servers for XA transactions, -see <a class="ulink" href="http://download.oracle.com/docs/cd/E13161_01/tuxedo/docs10gr3/pgc/pgthr.html" target="_top">http://download.oracle.com/docs/cd/E13161_01/tuxedo/docs10gr3/pgc/pgthr.html</a>. -All databases used by servers should be opened with handles created with the <a href="../api_reference/C/dbcreate.html#dbcreate_DB_XA_CREATE" class="olink">DB_XA_CREATE</a> flag in the <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a> method and must be opened in the tpsvrinit routine. -Note that the environment parameter of the <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a> method must be assigned NULL. -For more information on the tpsvrinit routine, see <a class="ulink" href="http://download.oracle.com/docs/cd/E13161_01/tuxedo/docs10gr3/pgc/pgthr.html" target="_top">http://download.oracle.com/docs/cd/E13161_01/tuxedo/docs10gr3/pgc/pgthr.html</a>. - -</p> + <p> + Yes. For information on how to build multi-threaded + servers for XA transactions, see <a class="ulink" href="http://download.oracle.com/docs/cd/E13161_01/tuxedo/docs10gr3/pgc/pgthr.html" target="_top">http://download.oracle.com/docs/cd/E13161_01/tuxedo/docs10gr3/pgc/pgthr.html</a>. + All databases used by servers should be opened + with handles created with the <a href="../api_reference/C/dbcreate.html#dbcreate_DB_XA_CREATE" class="olink">DB_XA_CREATE</a> flag in + the <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a> method and must be opened in the + tpsvrinit routine. Note that the environment parameter + of the <a href="../api_reference/C/dbcreate.html" class="olink">db_create()</a> method must be assigned NULL. For + more information on the tpsvrinit routine, see <a class="ulink" href="http://download.oracle.com/docs/cd/E13161_01/tuxedo/docs10gr3/pgc/pgthr.html" target="_top">http://download.oracle.com/docs/cd/E13161_01/tuxedo/docs10gr3/pgc/pgthr.html</a>. + </p> </li> </ol> </div> @@ -117,9 +158,8 @@ For more information on the tpsvrinit routine, see <a class="ulink" href="http:/ <td width="20%" align="center"> <a accesskey="h" href="index.html">Home</a> </td> - <td width="40%" align="right" valign="top"> Chapter 14. - Application Specific Logging and Recovery - </td> + <td width="40%" align="right" valign="top"> Chapter 14. Application Specific Logging and + Recovery </td> </tr> </table> </div> diff --git a/docs/programmer_reference/xa_xa_config.html b/docs/programmer_reference/xa_xa_config.html index 35f9c916..a00d4a3d 100644 --- a/docs/programmer_reference/xa_xa_config.html +++ b/docs/programmer_reference/xa_xa_config.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="xa_xa_intro.html">Prev</a> </td> - <th width="60%" align="center">Chapter 13. - Distributed Transactions - </th> + <th width="60%" align="center">Chapter 13. Distributed Transactions </th> <td width="20%" align="right"> <a accesskey="n" href="xa_xa_restrict.html">Next</a></td> </tr> </table> @@ -42,91 +40,160 @@ <dl> <dt> <span class="sect2"> - <a href="xa_xa_config.html#idp2607440">Update the Resource Manager File in Tuxedo</a> + <a href="xa_xa_config.html#idp2312920">Update the Resource Manager File in Tuxedo</a> </span> </dt> <dt> <span class="sect2"> - <a href="xa_xa_config.html#idp2633056">Build the Transaction Manager Server</a> + <a href="xa_xa_config.html#idp2341840">Build the Transaction Manager Server</a> </span> </dt> <dt> <span class="sect2"> - <a href="xa_xa_config.html#idp2580168">Update the UBBCONFIG File</a> + <a href="xa_xa_config.html#idp1702176">Update the UBBCONFIG File</a> </span> </dt> </dl> </div> - <p> -To configure the Tuxedo system to use Berkeley DB resource managers, do the following: -</p> - <div class="sect2" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="idp2607440"></a>Update the Resource Manager File in Tuxedo</h3></div></div></div><p>For the purposes of this discussion, assume that the Tuxedo home directory is in</p><pre class="programlisting">/home/tuxedo</pre> -In that case, the resource manager file will be located in -<pre class="programlisting">/home/tuxedo/udataobj/RM</pre> -Edit the resource manager file to identify the Berkeley DB resource manager, the name of the resource manager switch, and the name of the library for the resource manager. -<p>For example, on a RedHat Linux Enterprise (64-bit) installation of Oracle Tuxedo 11gR1, you can update the resource manager file by adding the following line:</p><pre class="programlisting">BERKELEY-DB:db_xa_switch:-L${DB_INSTALL}/lib -ldb </pre><p>where <code class="literal">${DB_INSTALL}</code> is the directory into which you installed the Berkeley DB -library.</p><p>Note that the load options may differ depending on the platform of -your system.</p></div> + <p> + To configure the Tuxedo system to use Berkeley DB resource + managers, do the following: + </p> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2633056"></a>Build the Transaction Manager Server</h3> + <h3 class="title"><a id="idp2312920"></a>Update the Resource Manager File in Tuxedo</h3> </div> </div> </div> - <p>To do this, use the -Tuxedo <span class="bold"><strong>buildtms(1)</strong></span> utility. The <span class="bold"><strong>buildtms</strong></span> command will create -the <code class="literal">Berkeley-DB</code> resource manager in the directory from which it was run. -The parameters to the <span class="bold"><strong>buildtms</strong></span> command should be:</p> + <p> + For the purposes of this discussion, assume that the + Tuxedo home directory is in + </p> + <pre class="programlisting">/home/tuxedo</pre> + <p> + In that case, the resource manager file will be located + in + </p> + <pre class="programlisting">/home/tuxedo/udataobj/RM</pre> + <p> + Edit the resource manager file to identify the Berkeley + DB resource manager, the name of the resource manager + switch, and the name of the library for the resource + manager. + </p> + <p> + For example, on a RedHat Linux Enterprise (64-bit) + installation of Oracle Tuxedo 11gR1, you can update the + resource manager file by adding the following line: + </p> + <pre class="programlisting">BERKELEY-DB:db_xa_switch:-L${DB_INSTALL}/lib -ldb </pre> + <p> + where <code class="literal">${DB_INSTALL}</code> is the directory + into which you installed the Berkeley DB library. + </p> + <p> + Note that the load options may differ depending on the + platform of your system. + </p> + </div> + <div class="sect2" lang="en" xml:lang="en"> + <div class="titlepage"> + <div> + <div> + <h3 class="title"><a id="idp2341840"></a>Build the Transaction Manager Server</h3> + </div> + </div> + </div> + <p> + To do this, use the Tuxedo <span class="bold"><strong> + buildtms(1) </strong></span> utility. The <span class="bold"><strong>buildtms</strong></span> command will create + the <code class="literal">Berkeley-DB</code> resource manager in the + directory from which it was run. The parameters to the + <span class="bold"><strong>buildtms</strong></span> command + should be: + </p> <pre class="programlisting">buildtms -v -o DBRM -r BERKELEY-DB</pre> - <p>This will create an executable transaction manager server, <code class="literal">DBRM</code>, which is -called by Tuxedo to process begins, commits, and aborts.</p> + <p> + This will create an executable transaction manager + server, <code class="literal">DBRM</code>, which is called by Tuxedo + to process begins, commits, and aborts. + </p> </div> <div class="sect2" lang="en" xml:lang="en"> <div class="titlepage"> <div> <div> - <h3 class="title"><a id="idp2580168"></a>Update the UBBCONFIG File</h3> + <h3 class="title"><a id="idp1702176"></a>Update the UBBCONFIG File</h3> </div> </div> </div> - <p>You must make sure that your TUXCONFIG environment variable -identifies an UBBCONFIG file that properly identifies your resource -managers. In the GROUPS section of the UBBCONFIG file, you should identify the -group's LMID and GRPNO, as well as the transaction manager server name -"TMSNAME=DBRM." You must also specify the OPENINFO parameter, setting it -equal to the string</p> + <p> + You must make sure that your TUXCONFIG environment + variable identifies an UBBCONFIG file that properly + identifies your resource managers. In the GROUPS section + of the UBBCONFIG file, you should identify the group's + LMID and GRPNO, as well as the transaction manager server + name "TMSNAME=DBRM." You must also specify the OPENINFO + parameter, setting it equal to the string + </p> <pre class="programlisting">rm_name:dir</pre> - <p>where rm_name is the resource name specified in the RM file (that is, -BERKELEY-DB) and dir is the directory for the Berkeley DB home environment -(see <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> for a discussion of Berkeley DB environments).</p> - <p>Because Tuxedo resource manager startup accepts only a single string -for configuration, any environment customization that might have been -done via the config parameter to <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> must instead be done -by placing a <a class="xref" href="env_db_config.html" title="DB_CONFIG configuration file">DB_CONFIG configuration file</a> in the Berkeley DB environment directory. -See <a class="xref" href="env_naming.html" title="File naming">File naming</a> for further -information.</p> - <p>Consider the following configuration. We have built a transaction -manager server, as described previously. We want the Berkeley DB environment -to be <code class="literal">/home/dbhome</code>, our database files to be maintained in -<code class="literal">/home/datafiles</code>, our log files to be maintained in -<code class="literal">/home/log</code>, and we want a duplexed server.</p> - <p>The GROUPS section of the ubb file might look like the following:</p> + <p> + where rm_name is the resource name specified in the RM + file (that is, BERKELEY-DB) and dir is the directory for + the Berkeley DB home environment (see <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> for a + discussion of Berkeley DB environments). + </p> + <p> + Because Tuxedo resource manager startup accepts only a + single string for configuration, any environment + customization that might have been done via the config + parameter to <a href="../api_reference/C/envopen.html" class="olink">DB_ENV->open()</a> must instead be done by placing a + <a class="xref" href="env_db_config.html" title="DB_CONFIG configuration file">DB_CONFIG configuration + file</a> in the Berkeley DB + environment directory. See <a class="xref" href="env_naming.html" title="File naming">File naming</a> for further information. + </p> + <p> + Consider the following configuration. We have built a + transaction manager server, as described previously. We + want the Berkeley DB environment to be + <code class="literal">/home/dbhome</code>, our database files to + be maintained in <code class="literal">/home/datafiles</code>, our + log files to be maintained in + <code class="literal">/home/log</code>, and we want a duplexed + server. + </p> + <p> + The GROUPS section of the ubb file might look like the + following: + </p> <pre class="programlisting">group_tm LMID=myname GRPNO=1 TMSNAME=DBRM TMSCOUNT=2 \ - OPENINFO="BERKELEY-DB:/home/dbhome"</pre> - <p>There would be a <a class="xref" href="env_db_config.html" title="DB_CONFIG configuration file">DB_CONFIG configuration file</a> in the directory -<code class="literal">/home/dbhome</code> that contained the following two lines:</p> - <pre class="programlisting">set_data_dir /home/datafiles -set_lg_dir /home/log</pre> - <p>Finally, the UBBCONFIG file must be translated into a binary version using -Tuxedo's <span class="bold"><strong>tmloadcf</strong></span>(1) utility, and then the pathname of that -binary file must be specified as your TUXCONFIG environment variable.</p> + OPENINFO="BERKELEY-DB:/home/dbhome"</pre> + <p> + There would be a <a class="xref" href="env_db_config.html" title="DB_CONFIG configuration file">DB_CONFIG configuration + file</a> in the directory + <code class="literal">/home/dbhome</code> that contained the + following two lines: + </p> + <pre class="programlisting">add_data_dir /home/datafiles +set_lg_dir /home/log</pre> + <p> + Finally, the UBBCONFIG file must be translated into a + binary version using Tuxedo's <span class="bold"><strong>tmloadcf </strong></span>(1) + utility, and then the + pathname of that binary file must be specified as your + TUXCONFIG environment variable. + </p> </div> - <p>At this point, your system is properly initialized to use the Berkeley DB -resource manager.</p> - <p>See <a href="../api_reference/C/db.html" class="olink">DB class</a> for further information on accessing data files -using XA.</p> + <p> + At this point, your system is properly initialized to use + the Berkeley DB resource manager. + </p> + <p> + See <a href="../api_reference/C/db.html" class="olink">DB class</a> for further information on accessing data + files using XA. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/xa_xa_intro.html b/docs/programmer_reference/xa_xa_intro.html index dc74b265..6b3ec733 100644 --- a/docs/programmer_reference/xa_xa_intro.html +++ b/docs/programmer_reference/xa_xa_intro.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="xa_build.html">Prev</a> </td> - <th width="60%" align="center">Chapter 13. - Distributed Transactions - </th> + <th width="60%" align="center">Chapter 13. Distributed Transactions </th> <td width="20%" align="right"> <a accesskey="n" href="xa_xa_config.html">Next</a></td> </tr> </table> @@ -38,34 +36,72 @@ </div> </div> </div> - <p>Berkeley DB can be used as an XA-compliant resource manager. The XA -implementation is known to work with the Tuxedo transaction -manager.</p> - <p>The XA support is encapsulated in the resource manager switch -db_xa_switch, which defines the following functions:</p> + <p> + Berkeley DB can be used as an XA-compliant resource manager. + The XA implementation is known to work with the Tuxedo + transaction manager. + </p> + <p> + The XA support is encapsulated in the resource manager + switch db_xa_switch, which defines the following + functions: + </p> <div class="itemizedlist"> <ul type="disc"> - <li><span class="emphasis"><em>__db_xa_close.</em></span> Close the resource manager.</li> - <li><span class="emphasis"><em>__db_xa_commit. </em></span> Commit the specified transaction.</li> - <li><span class="emphasis"><em>__db_xa_complete.</em></span> Wait for asynchronous operations to complete.</li> - <li><span class="emphasis"><em>__db_xa_end. </em></span> Disassociate the application from a transaction.</li> - <li><span class="emphasis"><em>__db_xa_forget.</em></span> Forget about a transaction that was heuristically completed. (Berkeley DB does not support heuristic completion.)</li> - <li><span class="emphasis"><em>__db_xa_open.</em></span> Open the resource manager.</li> - <li><span class="emphasis"><em>__db_xa_prepare.</em></span> Prepare the specified transaction.</li> - <li><span class="emphasis"><em>__db_xa_recover.</em></span> Return a list of prepared, but not yet committed transactions.</li> - <li><span class="emphasis"><em>__db_xa_rollback.</em></span> Abort the specified transaction.</li> - <li><span class="emphasis"><em>__db_xa_start.</em></span> Associate the application with a transaction.</li> + <li><span class="emphasis"><em>__db_xa_close.</em></span> + Close the resource manager. + </li> + <li><span class="emphasis"><em>__db_xa_commit. </em></span> + Commit the specified transaction. + </li> + <li><span class="emphasis"><em>__db_xa_complete.</em></span> + Wait for asynchronous operations to complete. + </li> + <li><span class="emphasis"><em>__db_xa_end. </em></span> + Disassociate the application from a + transaction. + </li> + <li><span class="emphasis"><em>__db_xa_forget.</em></span> + Forget about a transaction that was heuristically + completed. (Berkeley DB does not support heuristic + completion.) + </li> + <li><span class="emphasis"><em>__db_xa_open.</em></span> Open + the resource manager. + </li> + <li><span class="emphasis"><em>__db_xa_prepare.</em></span> + Prepare the specified transaction. + </li> + <li><span class="emphasis"><em>__db_xa_recover.</em></span> + Return a list of prepared, but not yet committed + transactions. + </li> + <li><span class="emphasis"><em>__db_xa_rollback.</em></span> + Abort the specified transaction. + </li> + <li><span class="emphasis"><em>__db_xa_start.</em></span> + Associate the application with a transaction. + </li> </ul> </div> - <p>The Berkeley DB resource manager does not support the following optional -XA features:</p> + <p> + The Berkeley DB resource manager does not support the + following optional XA features: + </p> <div class="itemizedlist"> <ul type="disc"> - <li>Asynchronous operations</li> - <li>Transaction migration</li> + <li> + Asynchronous operations + </li> + <li> + Transaction migration + </li> </ul> </div> - <p>The Tuxedo System is available from <a class="ulink" href="http://www.oracle.com/us/bea/index.html" target="_top">Oracle BEA Systems</a>.</p> + <p> + The Tuxedo System is available from <a class="ulink" href="http://www.oracle.com/us/corporate/Acquisitions/bea/index.html" target="_top">Oracle BEA + Systems</a>. + </p> </div> <div class="navfooter"> <hr /> diff --git a/docs/programmer_reference/xa_xa_restrict.html b/docs/programmer_reference/xa_xa_restrict.html index 9f1292a2..9c44bd39 100644 --- a/docs/programmer_reference/xa_xa_restrict.html +++ b/docs/programmer_reference/xa_xa_restrict.html @@ -14,7 +14,7 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> @@ -22,9 +22,7 @@ </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="xa_xa_config.html">Prev</a> </td> - <th width="60%" align="center">Chapter 13. - Distributed Transactions - </th> + <th width="60%" align="center">Chapter 13. Distributed Transactions </th> <td width="20%" align="right"> <a accesskey="n" href="xa_faq.html">Next</a></td> </tr> </table> @@ -38,84 +36,90 @@ </div> </div> </div> - <p> - When you are using Berkeley DB for XA transactions, there are a few - restrictions you should be aware of: + <p> + When you are using Berkeley DB for XA transactions, there + are a few restrictions you should be aware of: </p> <div class="itemizedlist"> <ul type="disc"> <li> <p> - Configure environment using the <a href="../api_reference/C/configuration_reference.html" class="olink">DB_CONFIG</a> file + Configure environment using the <a href="../api_reference/C/configuration_reference.html" class="olink">DB_CONFIG</a> file </p> - <p> - For most options, you must configure your environment via - the <a href="../api_reference/C/configuration_reference.html" class="olink">DB_CONFIG</a> file because an XA application or server cannot - control the environment creation. + <p> + For most options, you must configure your + environment via the <a href="../api_reference/C/configuration_reference.html" class="olink">DB_CONFIG</a> file because an XA + application or server cannot control the environment + creation. </p> </li> <li> - <p> - Snapshot isolation must be configured for the entire - environment. + <p> + Snapshot isolation must be configured for the + entire environment. </p> - <p> - Transactions managed by the Berkeley DB X/open compliant XA - resource manager can be configured for transaction - snapshots using either database open flags or the <a href="../api_reference/C/configuration_reference.html" class="olink">DB_CONFIG</a> file - file. To configure using database open flags, open the XA - managed database with the flag <a href="../api_reference/C/dbopen.html#dbopen_DB_MULTIVERSION" class="olink">DB_MULTIVERSION</a>. When using - DB_CONFIG, include both of the following lines: + <p> + Transactions managed by the Berkeley DB X/open + compliant XA resource manager can be configured for + transaction snapshots using either database open flags + or the <a href="../api_reference/C/configuration_reference.html" class="olink">DB_CONFIG</a> file file. To configure using database + open flags, open the XA managed database with the flag + <a href="../api_reference/C/dbopen.html#dbopen_DB_MULTIVERSION" class="olink">DB_MULTIVERSION</a>. When using DB_CONFIG, include both + of the following lines: </p> <pre class="programlisting">set_flags DB_MULTIVERSION set_flags DB_TXN_SNAPSHOT</pre> - <p> - Note that both methods will results in all transactions - using transaction snapshots, there is no way to enable - transaction snapshots in just a subset of XA managed - transactions. + <p> + Note that both methods will results in all + transactions using transaction snapshots, there is no + way to enable transaction snapshots in just a subset + of XA managed transactions. </p> </li> <li> - <p>No in-memory logging</p> <p> - Upon return from xa_open, Berkeley DB checks to ensure - there is no in-memory logging. If in-memory logging is - detected, a FAILURE message is returned to the application. + No in-memory logging + </p> + <p> + Upon return from xa_open, Berkeley DB checks to + ensure there is no in-memory logging. If in-memory + logging is detected, a FAILURE message is returned to + the application. </p> </li> <li> <p> - No application-level child transactions + No application-level child transactions </p> - <p> - Berkeley DB verifies in the xa_start and xa_end calls that no - XA transaction has a parent. If application-level child - transactions are detected, a FAILURE message is returned to the - application. + <p> + Berkeley DB verifies in the xa_start and xa_end + calls that no XA transaction has a parent. If + application-level child transactions are detected, a + FAILURE message is returned to the application. </p> </li> <li> - <p> - All database-level operations, such as create, rename, and - remove, must be performed in local BDB transactions, not - distributed XA transactions + <p> + All database-level operations, such as create, + rename, and remove, must be performed in local BDB + transactions, not distributed XA transactions </p> - <p> + <p> Berkeley DB checks that there is no XA transaction - currently active during these operations, and if detected, - a FAILURE message is returned to the application. + currently active during these operations, and if + detected, a FAILURE message is returned to the + application. </p> </li> <li> <p> - Close cursors before a service invocation returns + Close cursors before a service invocation returns </p> - <p> - Berkeley DB checks in the <code class="literal">xa_end</code> call - that the <code class="literal">DB_TXN</code> has no active cursors - open and and if detected, a FAILURE message is returned to - the application. + <p> + Berkeley DB checks in the <code class="literal">xa_end</code> + call that the <code class="literal">DB_TXN</code> has no active + cursors open and and if detected, a FAILURE message is + returned to the application. </p> </li> </ul> |