Running Replication using the db

Date: Tue, 17 Feb 2015 17:25:57 +0000 Subject: Imported from /home/lorry/working-area/delta_berkeleydb/db-6.1.23.tar.gz. --- docs/programmer_reference/rep_replicate.html | 538 +++++++++++++++------------ 1 file changed, 297 insertions(+), 241 deletions(-) (limited to 'docs/programmer_reference/rep_replicate.html') diff --git a/docs/programmer_reference/rep_replicate.html b/docs/programmer_reference/rep_replicate.html index 4068e278..dd848b85 100644 --- a/docs/programmer_reference/rep_replicate.html +++ b/docs/programmer_reference/rep_replicate.html @@ -3,28 +3,27 @@ - Running Replication using the db_replicate Utility + Running replication using the db_replicate utility - +

Library Version 11.2.5.3

Library Version 12.1.6.1

- + - +

Running Replication using the db_replicate Utility			Running replication using the + db_replicate utility
Prev	Chapter 12. - Berkeley DB Replication -	Chapter 12. Berkeley DB Replication	Next

@@ -34,7 +33,8 @@

Running Replication using the db_replicate Utility

Running replication using the + db_replicate utility

@@ -42,125 +42,159 @@

- One Replication Process and Multiple Subordinate Processes + One replication process and multiple subordinate processes
- Common Use Case + Common use case
- Avoiding Rollback + Avoiding rollback
- When to Consider an Integrated HA Application + When to consider an integrated replication application

Replication Manager supports shared access to a database -environment from multiple processes. Berkeley DB provides a -replication-aware utility, db_replicate, that enables you to upgrade an existing Transactional Data Store -application, as discussed in the Transactional Data Store introduction section, -to an HA application with -minor modifications. While the db_replicate utility simplifies the use -of replication with a TDS application, you -must still understand replication and its impact on the application. -

+ Replication Manager supports shared access to a database + environment from multiple processes. Berkeley DB provides a + replication-aware utility, db_replicate, that enables you to + upgrade an existing Transactional Data Store application, as + discussed in the Transactional Data Store introduction section, to a replication + application with minor modifications. While the db_replicate utility + simplifies the use of replication with a Transactional Data + Store application, you must still understand replication and + its impact on the application. +

One Replication Process and Multiple Subordinate Processes

One replication process and multiple subordinate processes

Based on the terminology introduced in the Running Replication Manager in multiple processes section, -application processes are "subordinate processes" and the db_replicate utility -is the "primary replication process".

-You must consider the following items when -planning to use the db_replicate utility in combination with -a TDS application. -

+ Based on the terminology introduced in the Running Replication Manager in + multiple processes + section, application processes are subordinate processes + and the db_replicate utility is the replication process. +

+ You must consider the following items when planning to + use the db_replicate utility in combination with a Transactional + Data Store application. +

Memory regions -
- The db_replicate utility requires shared memory access among - separate processes, and therefore cannot be used with DB_PRIVATE. -
Multi-process implications -
- You must understand and accept all of the TDS implications - of multi-process use as specified in Architecting Transactional Data Store applications. Special attention should be - paid to the coordination needed for unrelated processes to - start up correctly. -
Replication configuration -
- Several configuration settings are required for replication. - You must set the DB_INIT_REP - and DB_THREAD flags for the DB_ENV->open() method. - Another required configuration item is the local address. You identify - this by creating a DB_SITE handle and then setting the - DB_LOCAL_SITE parameter using the - DB_SITE->set_config() method. You also tell sites how to contact other - sites by creating DB_SITE handles for those sites. - Most replication configuration options start with reasonable - defaults, but applications have to customize - at least some of them. You can set all replication related configuration options - either programmatically or in the DB_CONFIG file. -
Starting the application and replication -
- The db_replicate utility assumes that an - environment exists and that the application has run recovery, - if necessary, and created and configured the environment. - The startup flow of a typical TDS application may not be the - best flow for a replication application and you - must understand the issues involved. - For instance, if an application starts, runs recovery, - and performs update operations before starting the db_replicate utility, - then if that site becomes a client when replication starts, - those update operations will be rolled back. -
Handling events -
- Almost all of the replication-specific events are handled - by the db_replicate utility process, and therefore the application - process does not see them. If the application needs to know - the information from those replication-specific events, such as - role changes, the application must call the rep_stat() method method. - The one replication-specific event the application can now - receive is the DB_EVENT_REP_PERM_FAILED event. - See Choosing a Replication Manager Ack Policy - for additional information about this event. -
Handling errors -
- There are some error return values that relate only to replication. - Specifically, the DB_REP_HANDLE_DEAD error should now be handled - by the application. Also, if master leases are in use, then the - application also needs to consider the DB_REP_LEASE_EXPIRED error. -
Flexibility tradeoff -
- You are giving up flexibility - for the ease of use of the utility. - Application complexity or requirements may eventually - dictate integrating HA calls into the application - over using the db_replicate utility.
Read-only client application -
- The application requires additional changes to manage - the read-only status when the application takes on the role of - a client. -
+ Memory regions +
+ The db_replicate utility + requires shared memory access among separate + processes, and therefore cannot be used with + DB_PRIVATE. +
+ Multi-process implications +
+ You must + understand and accept all of the Transactional + Data Store implications of multi-process use as + specified in Architecting Transactional Data + Store applications. Special + attention should be paid to the coordination + needed for unrelated processes to start up + correctly. +
+ Replication configuration +
+ Several + configuration settings are required for + replication. You must set the DB_INIT_REP and + DB_THREAD flags for the DB_ENV->open() method. + Another required configuration item is the local + address. You identify this by creating a DB_SITE + handle and then setting the + DB_LOCAL_SITE parameter + using the DB_SITE->set_config() method. You also + tell sites how to contact other sites by creating + DB_SITE handles for those sites. Most replication + configuration options start with reasonable + defaults, but applications have to customize at + least some of them. You can set all replication + related configuration options either + programmatically or in the DB_CONFIG file. +
+ Starting the application and replication +
+ The db_replicate utility assumes that an environment + exists and that the application has run recovery, + if necessary, and created and configured the + environment. The startup flow of a typical + Transactional Data Store application may not be + the best flow for a replication application and + you must understand the issues involved. For + instance, if an application starts, runs recovery, + and performs update operations before starting the + db_replicate utility, then if that site becomes a client + when replication starts, those update operations + will be rolled back. +
+ Handling events +
+ Almost all of the + replication-specific events are handled by the + db_replicate utility process, and therefore the + application process does not see them. If the + application needs to know the information from + those replication-specific events, such as role + changes, the application must call the rep_stat() method + method. The one replication-specific event the + application can now receive is the + DB_EVENT_REP_PERM_FAILED event. See Choosing a Replication Manager acknowledgement policy for additional + information about this event. +
+ Handling errors +
+ There are some error + return values that relate only to replication. + Specifically, the + DB_REP_HANDLE_DEAD error + should now be handled by the application. Also, if + master leases are in use, then the application + also needs to consider the + DB_REP_LEASE_EXPIRED error. +
+ Flexibility tradeoff +
+ You are giving up + flexibility for the ease of use of the utility. + Application complexity or requirements may + eventually dictate integrating replication calls + into the application instead of using the + db_replicate utility. +
+ Read-only client application +
+ The + application requires additional changes to manage + the read-only status when the application takes on + the role of a client. +

@@ -168,197 +202,218 @@ a TDS application.

Common Use Case

Common use case

-This section lists the steps needed to get replication running for -a common use case of the db_replicate utility. -The use case presented is an existing TDS application that -already has its environment and databases created and is up and running. -At some point, HA is considered because failover protection -or balancing the read load may now be desired. -

+ This section lists the steps needed to get replication + running for a common use case of the db_replicate utility. The + use case presented is an existing Transactional Data Store + application that already has its environment and databases + created and is up and running. At some point, replication + is considered because failover protection or balancing the + read load may now be desired. +

-
- To understand the issues involved in a replication/HA application, see - the db_replicate utility section in the API Reference Guide, the Replication Chapter in the - Programmer's Reference Guide, and the source code of the ex_rep_mgr - example program. -
+
+ To understand the issues involved in a + replication application, see the db_replicate utility + section in the API Reference + Guide, the Replication Chapter in the + Programmer's Reference + Guide, and the source code of the + ex_rep_mgr example program. +
-
- Make a local hot backup of the current - application environment to a new location to use as a testing area. -
+
+ Make a local hot backup of the current + application environment to a new location to use + as a testing area. +
-
- Add the DB_INIT_REP and DB_THREAD flags (if not already - being used) to the application or the DB_CONFIG file. -
+
+ Add the DB_INIT_REP and DB_THREAD flags (if + not already being used) to the application or the + DB_CONFIG file. +
- Modify the DB_CONFIG file to add the necessary replication - configuration values. At a minimum, the local - host and port information must be added using - the repmgr_site method parameter. - As more sites are added to the group, remote host and port - information can optionally also be added by adding - more repmgr_site method parameters to the DB_CONFIG file file. -
+ Modify the DB_CONFIG file to add the necessary + replication configuration values. At a minimum, + the local host and port information must be added + using the repmgr_site method parameter. As + more sites are added to the group, remote host and + port information can optionally also be added by + adding more repmgr_site method parameters to + the DB_CONFIG file. +
-
- Rebuild the application and restart - it in the current testing directory. -
+
+ Rebuild the application and restart it in the + current testing directory. +
- Start the db_replicate utility on the master site with the -M option and - any other options needed such as -h for the home directory. - At this point you have a lone master site running in an - environment with no other replicated sites in the group. -
+ Start the db_replicate utility on the master site + with the -M option and any other options needed + such as -h for the home directory. At this point + you have a lone master site running in an + environment with no other replicated sites in the + group. +
-
- Optionally, prepare to start a client site by performing a - manual hot backup of the running master environment to - initialize a client target directory. While replication - can make its own copy, the hot backup will expedite the - synchronization process. Also, if the application assumes - the existence of a database and the client site is started - without data, the application may have errors or incorrectly - attempt to create the database. -
+
+ Optionally, prepare to start a client site by + performing a manual hot backup of the running + master environment to initialize a client target + directory. While replication can make its own + copy, the hot backup will expedite the + synchronization process. Also, if the application + assumes the existence of a database and the client + site is started without data, the application may + have errors or incorrectly attempt to create the + database. +
- Copy the application to the client target. -
+ Copy the application to the client target. +
-
- Modify the client environment's DB_CONFIG file to set the client's local host and port values and to add - remote site information for the master site and any - other replication configuration choices necessary. -
+
+ Modify the client environment's DB_CONFIG file to + set the client's local host and port values and to + add remote site information for the master site + and any other replication configuration choices + necessary. +
-
- Start the application on the client. The client application - should not update data at this point, as explained previously. -
+
+ Start the application on the client. The client + application should not update data at this point, + as explained previously. +
-
- Start the db_replicate utility specifying the - client environment's home directory using the -h option. Omit the -M - option in this case, because the utility defaults to - starting in the client role. -
+
+ Start the db_replicate utility specifying the client + environment's home directory using the -h option. + Omit the -M option in this case, because the + utility defaults to starting in the client role. +

-Once the initial replication group is established, do not use the -M option with -the db_replicate utility. -After the initial start, db_replicate utility assumes the use of -elections. If a site crashes, it should rejoin the group as -a client so that it can synchronize with the rest of the group. -

+ Once the initial replication group is established, do + not use the -M option with the db_replicate utility. After the + initial start, the db_replicate utility assumes the use of + elections. If a site crashes, it should rejoin the group + as a client so that it can synchronize with the rest of + the group. +

Avoiding Rollback

Avoiding rollback

+ Depending on how an application is structured, + transactional rollback can occur. If this is possible, + then you must make application changes or be prepared for + successful transactions to disappear. Consider a common + program flow where the application first creates and opens + the environment with recovery. Then, immediately after + that, the application opens up the databases it expects to + use. Often an application will use the DB_CREATE flag so + that if the database does not exist it is created, + otherwise the existing one is used automatically. Then the + application begins servicing transactions to write and + read data. +

-Depending on how an application is structured, transactional rollback -can occur. If this is possible, then you must make application -changes or be prepared for successful transactions to disappear. -Consider a common program flow where the application first creates -and opens the environment with recovery. Then, immediately after -that, the application opens up the databases it expects to use. -Often an application will use the DB_CREATE flag so that if the -database does not exist it is created, otherwise the existing one is -used automatically. Then the application begins servicing transactions -to write and read data. -

+ When replication is introduced, particularly via the + db_replicate utility, the possibility of rollback exists unless + the application takes steps to prevent it. In the + situation described above, if all of the above steps occur + before the db_replicate utility process starts, and the site is + started as a client, then all the operations will be + rolled back when the site finds the master. The client + site will synchronize with the log and operations on the + master site, so any operations that occurred in the client + application before it knew it was a client will be + discarded. +

-When replication is introduced, particularly via the db_replicate utility, the -possibility of rollback exists unless the application takes steps -to prevent it. In the situation described above, if all of the -above steps occur before the db_replicate utility process starts, and -the site is started as a client, then all the operations will be -rolled back when the site finds the master. The client site will -synchronize with the log and operations on the master site, so -any operations that occurred in the client application before it -knew it was a client will be discarded. -

-One way to reduce the possibility of rollback is to modify the -application so that it only performs update operations (including -creation of a database) if it is the master site. If the application -refrains from updating until it is the master, then it will not perform -operations when it is in the undefined state before replication -has been started. -The event indicating a site is master will be delivered to the -db_replicate utility process, so the application process must look -for that information via the rep_stat() method. -A site that is expecting to perform updates may need to poll -via the rep_stat() method to see the state change from an undefined -role to either the master or client role. Similarly, since a -client site cannot create a database, it may need to poll for -the database's existence while the client synchronizes with the master -until the database is created at the client site. -

+ One way to reduce the possibility of rollback is to + modify the application so that it only performs update + operations (including creation of a database) if it is the + master site. If the application refrains from updating + until it is the master, then it will not perform + operations when it is in the undefined state before + replication has been started. The event indicating a site + is master will be delivered to the db_replicate utility process, + so the application process must look for that information + via the rep_stat() method. A site that is expecting to perform + updates may need to poll via the rep_stat() method to see the + state change from an undefined role to either the master + or client role. Similarly, since a client site cannot + create a database, it may need to poll for the database's + existence while the client synchronizes with the master + until the database is created at the client site. +

When to Consider an Integrated HA Application

When to consider an integrated replication application

+ The db_replicate utility provides the means to achieve a + replicated application quickly. However, the trade-off for + this rapid implementation is that the full flexibility of + replication is not available. Some applications may + eventually need to consider integrating directly with + replication rather than using the db_replicate utility if + greater flexibility is desired. +

-The db_replicate utility provides the means to achieve -a replicated application quickly. However, -the trade-off for this rapid implementation is that the full -flexibility of replication is not available. Some applications -may eventually need to consider integrating directly with -replication rather than using the db_replicate utility if greater flexibility is desired. -

-One likely reason for considering integration would be the -convenience of receiving all replication-related events in -the application process and gaining direct knowledge of such -things as role changes. -Using the event callback is cleaner and easier than -polling for state changes via the rep_stat() method. -

+ One likely reason for considering integration would be + the convenience of receiving all replication-related + events in the application process and gaining direct + knowledge of such things as role changes. Using the event + callback is cleaner and easier than polling for state + changes via the rep_stat() method. +

-A second likely reason for integrating replication directly -into the application is the multi-process aspect of the -utility program. The developer may find it easier to insert -the start of replication directly into the code once the -environment is created, recovered, or opened, and avoid -the scenario where the application is running in the -undefined state. Also it may simply be easier to start -the application once than to coordinate different processes -and their startup order in the system. -

+ A second likely reason for integrating replication + directly into the application is the multi-process aspect + of the utility program. The developer may find it easier + to insert the start of replication directly into the code + once the environment is created, recovered, or opened, and + avoid the scenario where the application is running in the + undefined state. Also it may simply be easier to start the + application once than to coordinate different processes + and their startup order in the system. +

@@ -372,11 +427,12 @@ and their startup order in the system. Next - Running Replication Manager in multiple processes + Running Replication Manager in + multiple processes Home - Choosing a Replication Manager Ack Policy + Choosing a Replication Manager acknowledgement policy

-- cgit v1.2.1