diff options
Diffstat (limited to 'qpid/doc/book/src/cpp-broker/queue-state-replication.xml')
| -rw-r--r-- | qpid/doc/book/src/cpp-broker/queue-state-replication.xml | 333 |
1 files changed, 333 insertions, 0 deletions
diff --git a/qpid/doc/book/src/cpp-broker/queue-state-replication.xml b/qpid/doc/book/src/cpp-broker/queue-state-replication.xml new file mode 100644 index 0000000000..3ffac805eb --- /dev/null +++ b/qpid/doc/book/src/cpp-broker/queue-state-replication.xml @@ -0,0 +1,333 @@ +<?xml version="1.0" encoding="utf-8"?> +<!-- + + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +--> + +<section id="queue-state-replication"> + <title> + Queue State Replication + </title> + + <section role="h2" id="queuestatereplication-AsynchronousReplicationofQueueState"> + <title> + Asynchronous + Replication of Queue State + </title> + + <section role="h3" id="queuestatereplication-Overview"> + <title> + Overview + </title> + <para> + There is support in qpidd for selective asynchronous replication + of queue state. This is achieved by: + </para> + <para> + (a) enabling event generation for the queues in question + </para> + <para> + (b) loading a plugin on the 'source' broker to encode those + events as messages on a replication queue (this plugin is + called + replicating_listener.so) + </para> + <para> + (c) loading a custom exchange plugin on the 'backup' broker (this + plugin is called replication_exchange.so) + </para> + <para> + (d) creating an instance of the replication exchange type on the + backup broker + </para> + <para> + (e) establishing a federation bridge between the replication + queue on the source broker and the replication exchange on the + backup broker + </para> + <para> + The bridge established between the source and backup brokers for + replication (step (e) above) should have acknowledgements turned + on (this may be done through the --ack N option to qpid-route). + This ensures that replication events are not lost if the bridge + fails. + </para> + <para> + The replication protocol will also eliminate duplicates to ensure + reliably replicated state. Note though that only one bridge per + replication exchange is supported. If clients try to publish to + the replication exchange or if more than a the single required + bridge from the replication queue on the source broker is + created, replication will be corrupted. (Access control may be + used to restrict access and help prevent this). + </para> + <para> + The replicating event listener plugin (step (b) above) has the + following options: + </para> + <programlisting> +Queue Replication Options: + --replication-queue QUEUE Queue on which events for + other queues are recorded + --replication-listener-name NAME (replicator) name by which to register the + replicating event listener + --create-replication-queue if set, the replication will + be created if it does not + exist + </programlisting> + <para> + The name of the queue is required. It can either point to a + durable queue whose definition has been previously recorded, or + the --create-replication-queue option can be specified in which + case the queue will be created a simple non-durable queue if it + does not already exist. + </para> + <!--h3--> + </section> + + <section role="h3" id="queuestatereplication-UsewithClustering"> + <title> + Use with + Clustering + </title> + <para> + The source and/or backup brokers may also be clustered brokers. + In this case the federated bridge will be re-established between + replicas should either of the originally connected nodes fail. + There are however the following limitations at present: + </para> + <itemizedlist> + <listitem> + <para>The backup site does not process membership updates after it + establishes the first connection. In order for newly added + members on a source cluster to be eligible as failover targets, + the bridge must be recreated after those members have been added + to the source cluster. + </para> + </listitem> + </itemizedlist> + <itemizedlist> + <listitem> + <para>New members added to a backup cluster will not receive + information about currently established bridges. Therefore in + order to allow the bridge to be re-established from these members + in the event of failure of older nodes, the bridge must be + recreated after the new members have joined. + </para> + </listitem> + </itemizedlist> + <itemizedlist> + <listitem> + <para>Only a single URL can be passed to create the initial link + from backup site to the primary site. this means that at the time + of creating the initial connection the initial node in the + primary site to which the connection is made needs to be running. + Once connected the backup site will receive a membership update + of all the nodes in the primary site, and if the initial + connection node in the primary fails, the link will be + re-established on the next node that was started (time) on the + primary site. + </para> + </listitem> + </itemizedlist> + <para> + Due to the acknowledged transfer of events over the bridge (see + note above) manual recreation of the bridge and automatic + re-establishment of te bridge after connection failure (including + failover where either or both ends are clustered brokers) will + not result in event loss. + </para> + <!--h3--> + </section> + + <section role="h3" id="queuestatereplication-OperationsonBackupQueues"> + <title> + Operations + on Backup Queues + </title> + <para> + When replicating the state of a queue to a backup broker it is + important to recognise that any other operations performed + directly on the backup queue may break the replication. + </para> + <para> + If the backup queue is to be an active (i.e. accessed by clients + while replication is on) only enqueues should be selected + for + replication. In this mode, any message enqueued on the source + brokers copy of the queue will also be enqueued on the backup + brokers copy. However not attempt will be made to remove messages + from the backup queue in response to removal of messages from the + source queue. + </para> + <!--h3--> + </section> + + <section role="h3" id="queuestatereplication-SelectingQueuesforReplication"> + <title> + Selecting + Queues for Replication + </title> + <para> + Queues are selected for replication by specifying the types of + events they should generate (it is from these events that the + replicating plugin constructs messages which are then pulled and + processed by the backup site). This is done through options + passed to the initial queue-declare command that creates the + queue and may be done either through qpid-config or similar + tools, or by the application. + </para> + <para> + With qpid-config, the --generate-queue-events options is used: + </para> + <programlisting> + --generate-queue-events N + If set to 1, every enqueue will generate an event that can be processed by + registered listeners (e.g. for replication). If set to 2, events will be + generated for enqueues and dequeues + </programlisting> + <para> + From an application, the arguments field of the queue-declare + AMQP command is used to convey this information. An entry should + be added to the map with key 'qpid.queue_event_generation' and an + integer value of 1 (to replicate only enqueue events) or 2 (to + replicate both enqueue and dequeue events). + </para> + <para> + Applications written using the c++ client API may fine the + qpid::client::QueueOptions class convenient. This has a + enableQueueEvents() method on it that can be used to set the + option (the instance of QueueOptions is then passed as the value + of the arguments field in the queue-declare command. The boolean + option to that method should be set to true if only enequeue + events should be replicated; by default it is false meaning that + both enqueues and dequeues will be replicated. E.g. + </para> + <programlisting> + QueueOptions options; + options.enableQueueEvents(false); + session.queueDeclare(arg::queue="my-queue", arg::arguments=options); + </programlisting> + <!--h3--> + </section> + + <section role="h3" id="queuestatereplication-Example"> + <title> + Example + </title> + <para> + Lets assume we will run the primary broker on host1 and the + backup on host2, have installed qpidd on both and have the + replicating_listener and replication_exchange plugins in qpidd's + module directory(*1). + </para> + <para> + On host1 we start the source broker and specifcy that a queue + called 'replication' should be used for storing the events until + consumed by the backup. We also request that this queue be + created (as transient) if not already specified: + </para> + <programlisting> + qpidd --replication-queue replication-queue --create-replication-queue true --log-enable info+ + </programlisting> + <para> + On host2 we start up the backup broker ensuring that the + replication exchange module is loaded: + </para> + <programlisting> + qpidd + </programlisting> + <para> + We can then create the instance of that replication exchange that + we will use to process the events: + </para> + <programlisting> + qpid-config -a host2 add exchange replication replication-exchange + </programlisting> + <para> + If this fails with the message "Exchange type not implemented: + replication", it means the replication exchange module was + not + loaded. Check that the module is installed on your system and if + necessary provide the full path to the library. + </para> + <para> + We then connect the replication queue on the source broker with + the replication exchange on the backup broker using the + qpid-route command: + </para> + <programlisting> + qpid-route --ack 50 queue add host2 host1 replication-exchange replication-queue +</programlisting> + <para> + The example above configures the bridge to acknowledge messages + in batches of 50. + </para> + <para> + Now create two queues (on both source and backup brokers), one + replicating both enqueues and dequeues (queue-a) and the + other + replicating only dequeues (queue-b): + </para> + <programlisting> + qpid-config -a host1 add queue queue-a --generate-queue-events 2 + qpid-config -a host1 add queue queue-b --generate-queue-events 1 + + qpid-config -a host2 add queue queue-a + qpid-config -a host2 add queue queue-b + </programlisting> + <para> + We are now ready to use the queues and see the replication. + </para> + <para> + Any message enqueued on queue-a will be replicated to the backup + broker. When the message is acknowledged by a client connected to + host1 (and thus dequeued), that message will be removed from the + copy of the queue on host2. The state of queue-a on host2 will + thus mirror that of the equivalent queue on host1, albeit with a + small lag. (Note + however that we must not have clients connected to host2 publish + to-or consume from- queue-a or the state will fail to replicate + correctly due to conflicts). + </para> + <para> + Any message enqueued on queue-b on host1 will also be enqueued on + the equivalent queue on host2. However the acknowledgement and + consequent dequeuing of messages from queue-b on host1 will have + no effect on the state of queue-b on host2. + </para> + <para> + (*1) If not the paths in the above may need to be modified. E.g. + if using modules built from a qpid svn checkout, the following + would be added to the command line used to start qpidd on host1: + </para> + <programlisting> + --load-module <path-to-qpid-dir>/src/.libs/replicating_listener.so + </programlisting> + <para> + and the following for the equivalent command line on host2: + </para> + <programlisting> + --load-module <path-to-qpid-dir>/src/.libs/replication_exchange.so + </programlisting> + <!--h3--> + </section> + <!--h2--> + </section> +</section> |