diff options
| author | Lorry Tar Creator <lorry-tar-importer@baserock.org> | 2015-02-17 17:25:57 +0000 |
|---|---|---|
| committer | <> | 2015-03-17 16:26:24 +0000 |
| commit | 780b92ada9afcf1d58085a83a0b9e6bc982203d1 (patch) | |
| tree | 598f8b9fa431b228d29897e798de4ac0c1d3d970 /docs/programmer_reference/transapp_archival.html | |
| parent | 7a2660ba9cc2dc03a69ddfcfd95369395cc87444 (diff) | |
| download | berkeleydb-master.tar.gz | |
Diffstat (limited to 'docs/programmer_reference/transapp_archival.html')
| -rw-r--r-- | docs/programmer_reference/transapp_archival.html | 334 |
1 files changed, 176 insertions, 158 deletions
diff --git a/docs/programmer_reference/transapp_archival.html b/docs/programmer_reference/transapp_archival.html index 2923d54c..e007b697 100644 --- a/docs/programmer_reference/transapp_archival.html +++ b/docs/programmer_reference/transapp_archival.html @@ -14,17 +14,16 @@ <body> <div xmlns="" class="navheader"> <div class="libver"> - <p>Library Version 11.2.5.3</p> + <p>Library Version 12.1.6.1</p> </div> <table width="100%" summary="Navigation header"> <tr> - <th colspan="3" align="center">Database and log file archival</th> + <th colspan="3" align="center">Database and log file + archival</th> </tr> <tr> <td width="20%" align="left"><a accesskey="p" href="transapp_checkpoint.html">Prev</a> </td> - <th width="60%" align="center">Chapter 11. - Berkeley DB Transactional Data Store Applications - </th> + <th width="60%" align="center">Chapter 11. Berkeley DB Transactional Data Store Applications </th> <td width="20%" align="right"> <a accesskey="n" href="transapp_logfile.html">Next</a></td> </tr> </table> @@ -34,218 +33,237 @@ <div class="titlepage"> <div> <div> - <h2 class="title" style="clear: both"><a id="transapp_archival"></a>Database and log file archival</h2> + <h2 class="title" style="clear: both"><a id="transapp_archival"></a>Database and log file + archival</h2> </div> </div> </div> - <p> - The third component of the administrative infrastructure, archival - for catastrophic recovery, concerns the recoverability of the - database in the face of catastrophic failure. Recovery after - catastrophic failure is intended to minimize data loss when - physical hardware has been destroyed — for example, loss of a - disk that contains databases or log files. Although the - application may still experience data loss in this case, it is - possible to minimize it. + <p> + The third component of the administrative infrastructure, + archival for catastrophic recovery, concerns the + recoverability of the database in the face of catastrophic + failure. Recovery after catastrophic failure is intended to + minimize data loss when physical hardware has been destroyed + — for example, loss of a disk that contains databases or + log files. Although the application may still experience data + loss in this case, it is possible to minimize it. </p> - <p> - First, you may want to periodically create snapshots (that is, - backups) of your databases to make it possible to recover from - catastrophic failure. These snapshots are either a standard - backup, which creates a consistent picture of the databases as of a - single instant in time; or an on-line backup (also known as a - <span class="emphasis"><em>hot</em></span> backup), which creates a consistent - picture of the databases as of an unspecified instant during the - period of time when the snapshot was made. The advantage of a hot - backup is that applications may continue to read and write the - databases while the snapshot is being taken. The disadvantage of a - hot backup is that more information must be archived, and recovery - based on a hot backup is to an unspecified time between the start - of the backup and when the backup is completed. + <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> + <h3 class="title">Note</h3> + <p> + Berkeley DB backups (archives) can be recovered using + machines of differing byte order. That is, a backup taken + on a big-endian machine can be used to restore a database + residing on a little-endian machine. + </p> + </div> + <p> + First, you may want to periodically create snapshots (that + is, backups) of your databases to make it possible to recover + from catastrophic failure. These snapshots are either a + standard backup, which creates a consistent picture of the + databases as of a single instant in time; or an on-line backup + (also known as a <span class="emphasis"><em>hot</em></span> backup), which + creates a consistent picture of the databases as of an + unspecified instant during the period of time when the + snapshot was made. The advantage of a hot backup is that + applications may continue to read and write the databases + while the snapshot is being taken. The disadvantage of a hot + backup is that more information must be archived, and recovery + based on a hot backup is to an unspecified time between the + start of the backup and when the backup is completed. </p> <p> - Second, after taking a snapshot, you should periodically archive - the log files being created in the environment. It is often - helpful to think of database archival in terms of full and - incremental filesystem backups. A snapshot is a full backup, - whereas the periodic archival of the current log files is an - incremental backup. For example, it might be reasonable to take a - full snapshot of a database environment weekly or monthly, and - archive additional log files daily. Using both the snapshot and - the log files, a catastrophic crash at any time can be recovered to - the time of the most recent log archival; a time long after the - original snapshot. + Second, after taking a snapshot, you should periodically + archive the log files being created in the environment. It is + often helpful to think of database archival in terms of full + and incremental filesystem backups. A snapshot is a full + backup, whereas the periodic archival of the current log files + is an incremental backup. For example, it might be reasonable + to take a full snapshot of a database environment weekly or + monthly, and archive additional log files daily. Using both + the snapshot and the log files, a catastrophic crash at any + time can be recovered to the time of the most recent log + archival; a time long after the original snapshot. </p> <p> - When incremental backups are implemented using this procedure, it - is important to know that a database copy taken prior to a bulk - loading event (that is, a transaction started with the - <a href="../api_reference/C/txnbegin.html#txnbegin_DB_TXN_BULK" class="olink">DB_TXN_BULK</a> flag) can no longer be used as the target of an - incremental backup. This is true because bulk loading omits - logging of some record insertions, so these insertions cannot be - rolled forward by recovery. It is recommended that a full backup - be scheduled following a bulk loading event. + When incremental backups are implemented using this + procedure, it is important to know that a database copy taken + prior to a bulk loading event (that is, a transaction started + with the <a href="../api_reference/C/txnbegin.html#txnbegin_DB_TXN_BULK" class="olink">DB_TXN_BULK</a> flag) can no longer be used as the + target of an incremental backup. This is true because bulk + loading omits logging of some record insertions, so these + insertions cannot be rolled forward by recovery. It is + recommended that a full backup be scheduled following a bulk + loading event. </p> - <p> - To create a standard backup of your database that can be used to - recover from catastrophic failure, take the following steps: + <p> + To create a standard backup of your database that can be + used to recover from catastrophic failure, take the following + steps: </p> <div class="orderedlist"> <ol type="1"> <li> - Commit or abort all ongoing transactions. + Commit or abort all ongoing transactions. </li> - <li> - Stop writing your databases until the backup has completed. - Read-only operations are permitted, but no write operations and - no filesystem operations may be performed (for example, the - <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> and <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> methods may not be called). + <li> + Stop writing your databases until the backup has + completed. Read-only operations are permitted, but no + write operations and no filesystem operations may be + performed (for example, the <a href="../api_reference/C/envremove.html" class="olink">DB_ENV->remove()</a> and <a href="../api_reference/C/dbopen.html" class="olink">DB->open()</a> + methods may not be called). </li> - <li> - Force an environment checkpoint (see the <a href="../api_reference/C/db_checkpoint.html" class="olink">db_checkpoint</a> utility for - more information). + <li> + Force an environment checkpoint (see the + <a href="../api_reference/C/db_checkpoint.html" class="olink">db_checkpoint</a> utility for more information). </li> <li> - <p> - Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility with option <span class="bold"><strong>-s</strong></span> to identify all the database data - files, and copy them to a backup device such as CD-ROM, alternate - disk, or tape. + <p> + Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility with option <span class="bold"><strong>-s</strong></span> to identify all the + database data files, and copy them to a backup device + such as CD-ROM, alternate disk, or tape. </p> <p> - If the database files are stored in a separate directory from the - other Berkeley DB files, it may be simpler to archive the directory - itself instead of the individual files (see <a href="../api_reference/C/envset_data_dir.html" class="olink">DB_ENV->set_data_dir()</a> for - additional information). + If the database files are stored in a separate + directory from the other Berkeley DB files, it may be + simpler to archive the directory itself instead of the + individual files (see <a href="../api_reference/C/envadd_data_dir.html" class="olink">DB_ENV->add_data_dir()</a> for additional + information on storing database files in separate + directories). </p> <div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"> <h3 class="title">Note</h3> <p> - If any of the database files did not have an open <a href="../api_reference/C/db.html" class="olink">DB</a> - handle during the lifetime of the current log files, - the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility will not list them in its output. This - is another reason it may be simpler to use a separate - database file directory and archive the entire - directory instead of archiving only the files listed by - the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility. + If any of the database files did not have an + open <a href="../api_reference/C/db.html" class="olink">DB</a> handle during the lifetime of the + current log files, the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility will not list + them in its output. This is another reason it may + be simpler to use a separate database file + directory and archive the entire directory instead + of archiving only the files listed by the + <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility. </p> </div> </li> <li> - Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility with option <span class="bold"><strong>-l</strong></span> to identify all the log files, - and copy the last one (that is, the one with the highest - number) to a backup device such as CD-ROM, alternate disk, or - tape. + Run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility with option <span class="bold"><strong>-l</strong></span> to identify all the log + files, and copy the last one (that is, the one with the + highest number) to a backup device such as CD-ROM, + alternate disk, or tape. </li> </ol> </div> <p> - To create a <span class="emphasis"><em>hot</em></span> backup of your database that - can be used to recover from catastrophic failure, take the - following steps: + To create a <span class="emphasis"><em>hot</em></span> backup of your + database that can be used to recover from catastrophic + failure, take the following steps: </p> <div class="orderedlist"> <ol type="1"> <li> - Set the <a href="../api_reference/C/envset_flags.html#set_flags_DB_HOTBACKUP_IN_PROGRESS" class="olink">DB_HOTBACKUP_IN_PROGRESS</a> flag in the environment. - This affects the behavior of transactions started with the - <a href="../api_reference/C/txnbegin.html#txnbegin_DB_TXN_BULK" class="olink">DB_TXN_BULK</a> flag. + Set the <a href="../api_reference/C/envset_flags.html#set_flags_DB_HOTBACKUP_IN_PROGRESS" class="olink">DB_HOTBACKUP_IN_PROGRESS</a> flag in the + environment. This affects the behavior of transactions + started with the <a href="../api_reference/C/txnbegin.html#txnbegin_DB_TXN_BULK" class="olink">DB_TXN_BULK</a> flag. </li> <li> <p> - Archive your databases, as described in the previous step - #4. You do not have to halt ongoing transactions or force - a checkpoint. As this is a hot backup, and the databases - may be modified during the copy, it is critical that - database pages be read atomically as described by - <a class="xref" href="transapp_reclimit.html" title="Berkeley DB recoverability">Berkeley DB recoverability</a>. - </p> + Archive your databases, as described in the + previous step #4. You do not have to halt ongoing + transactions or force a checkpoint. As this is a hot + backup, and the databases may be modified during the + copy, it is critical that database pages be read + atomically as described by <a class="xref" href="transapp_reclimit.html" title="Berkeley DB recoverability">Berkeley DB recoverability</a>. </p> <p> - Note that only UNIX based systems are known to support the - atomicity of reads. These systems include: Solaris, Mac OSX, - HPUX and various BSD based systems. Linux and Windows - based systems do not support atomic filesystem reads - directly. The XFS file system supports atomic reads - despite the lack of it in Linux. On systems that do not - support atomic file system reads, the <a href="../api_reference/C/db_hotbackup.html" class="olink">db_hotbackup</a> utility - should be used or a tool can be constructed - using the <a href="../api_reference/C/envbackup.html" class="olink">DB_ENV->backup()</a> method. Alternatively, you can - construct a tool using the the <a href="../api_reference/C/db_copy.html" class="olink">db_copy()</a> method. You can - also perform a hot backup of just a single database in your - environment using the <a href="../api_reference/C/envdbbackup.html" class="olink">DB_ENV->dbbackup()</a> method. + Note that only UNIX based systems are known to + support the atomicity of reads. These systems include: + Solaris, Mac OSX, HPUX and various BSD based systems. + Linux and Windows based systems do not support atomic + filesystem reads directly. The XFS file system + supports atomic reads despite the lack of it in Linux. + On systems that do not support atomic file system + reads, the <a href="../api_reference/C/db_hotbackup.html" class="olink">db_hotbackup</a> utility should be used or a tool can + be constructed using the <a href="../api_reference/C/envbackup.html" class="olink">DB_ENV->backup()</a> method. + Alternatively, you can construct a tool using the the + <a href="../api_reference/C/db_copy.html" class="olink">db_copy()</a> method. You can also perform a hot backup of + just a single database in your environment using the + <a href="../api_reference/C/envdbbackup.html" class="olink">DB_ENV->dbbackup()</a> method. </p> </li> <li> - <p> - Archive <span class="bold"><strong>all</strong></span> of the log files. - The order of these two operations is required, and the database - files must be archived <span class="bold"><strong>before</strong></span> - the log files. This means that if the database files and log - files are in the same directory, you cannot simply archive the - directory; you must make sure that the correct order of - archival is maintained. + <p> + Archive <span class="bold"><strong>all</strong></span> of the + log files. The order of these two operations is + required, and the database files must be archived + <span class="bold"><strong>before</strong></span> the log + files. This means that if the database files and log + files are in the same directory, you cannot simply + archive the directory; you must make sure that the + correct order of archival is maintained. </p> <p> - To archive your log files, run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility using the - <span class="bold"><strong>-l</strong></span> option to identify all the - database log files, and copy them to your backup media. If the - database log files are stored in a separate directory from the - other database files, it may be simpler to archive the - directory itself instead of the individual files (see the + To archive your log files, run the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility + using the <span class="bold"><strong>-l</strong></span> option + to identify all the database log files, and copy them + to your backup media. If the database log files are + stored in a separate directory from the other database + files, it may be simpler to archive the directory + itself instead of the individual files (see the <a href="../api_reference/C/envset_lg_dir.html" class="olink">DB_ENV->set_lg_dir()</a> method for more information). </p> </li> - <li> - Reset the <a href="../api_reference/C/envset_flags.html#set_flags_DB_HOTBACKUP_IN_PROGRESS" class="olink">DB_HOTBACKUP_IN_PROGRESS</a> flag. + <li> Reset the <a href="../api_reference/C/envset_flags.html#set_flags_DB_HOTBACKUP_IN_PROGRESS" class="olink">DB_HOTBACKUP_IN_PROGRESS</a> flag. </li> </ol> </div> <p> - To minimize the archival space needed for log files when doing a - hot backup, run db_archive to identify those log files which are - not in use. Log files which are not in use do not need to be - included when creating a hot backup, and you can discard them or - move them aside for use with previous backups (whichever is - appropriate), before beginning the hot backup. + To minimize the archival space needed for log files when + doing a hot backup, run db_archive to identify those log files + which are not in use. Log files which are not in use do not + need to be included when creating a hot backup, and you can + discard them or move them aside for use with previous backups + (whichever is appropriate), before beginning the hot backup. </p> - <p> - After completing one of these two sets of steps, the database - environment can be recovered from catastrophic failure (see - <a class="xref" href="transapp_recovery.html" title="Recovery procedures">Recovery procedures</a> - for more information). + <p> + After completing one of these two sets of steps, the + database environment can be recovered from catastrophic + failure (see <a class="xref" href="transapp_recovery.html" title="Recovery procedures">Recovery procedures</a> for more information). </p> <p> To update either a hot or cold backup so that recovery from - catastrophic failure is possible to a new point in time, repeat - step #2 under the hot backup instructions and archive - <span class="bold"><strong>all</strong></span> of the log files in the - database environment. Each time both the database and log files - are copied to backup media, you may discard all previous database - snapshots and saved log files. Archiving additional log files does - not allow you to discard either previous database snapshots or log - files. Generally, updating a backup must be integrated with the - application's log file removal procedures. + catastrophic failure is possible to a new point in time, + repeat step #2 under the hot backup instructions and archive + <span class="bold"><strong>all</strong></span> of the log files in + the database environment. Each time both the database and log + files are copied to backup media, you may discard all previous + database snapshots and saved log files. Archiving additional + log files does not allow you to discard either previous + database snapshots or log files. Generally, updating a backup + must be integrated with the application's log file removal + procedures. </p> - <p> - The time to restore from catastrophic failure is a function of the - number of log records that have been written since the snapshot was - originally created. Perhaps more importantly, the more separate - pieces of backup media you use, the more likely it is that you will - have a problem reading from one of them. For these reasons, it is - often best to make snapshots on a regular basis. + <p> + The time to restore from catastrophic failure is a function + of the number of log records that have been written since the + snapshot was originally created. Perhaps more importantly, the + more separate pieces of backup media you use, the more likely + it is that you will have a problem reading from one of them. + For these reasons, it is often best to make snapshots on a + regular basis. </p> <p> - <span class="bold"><strong>Obviously, the reliability of your archive - media will affect the safety of your data. For archival - safety, ensure that you have multiple copies of your database - backups, verify that your archival media is error-free and - readable, and that copies of your backups are stored - offsite!</strong></span> + <span class="bold"><strong>Obviously, the reliability of your + archive media will affect the safety of your data. For + archival safety, ensure that you have multiple copies of + your database backups, verify that your archival media is + error-free and readable, and that copies of your backups + are stored offsite!</strong></span> </p> - <p> - The functionality provided by the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility is also available - directly from the Berkeley DB library. The following code fragment - prints out a list of log and database files that need to be - archived: + <p> + The functionality provided by the <a href="../api_reference/C/db_archive.html" class="olink">db_archive</a> utility is also + available directly from the Berkeley DB library. The following + code fragment prints out a list of log and database files that + need to be archived: </p> <pre class="programlisting">void log_archlist(DB_ENV *dbenv) |