diff options
Diffstat (limited to 'qpid/doc/book/src/cpp-broker/queue-state-replication.xml')
| -rw-r--r-- | qpid/doc/book/src/cpp-broker/queue-state-replication.xml | 333 |
1 files changed, 0 insertions, 333 deletions
diff --git a/qpid/doc/book/src/cpp-broker/queue-state-replication.xml b/qpid/doc/book/src/cpp-broker/queue-state-replication.xml deleted file mode 100644 index 3ffac805eb..0000000000 --- a/qpid/doc/book/src/cpp-broker/queue-state-replication.xml +++ /dev/null @@ -1,333 +0,0 @@ -<?xml version="1.0" encoding="utf-8"?> -<!-- - - Licensed to the Apache Software Foundation (ASF) under one - or more contributor license agreements. See the NOTICE file - distributed with this work for additional information - regarding copyright ownership. The ASF licenses this file - to you under the Apache License, Version 2.0 (the - "License"); you may not use this file except in compliance - with the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - KIND, either express or implied. See the License for the - specific language governing permissions and limitations - under the License. - ---> - -<section id="queue-state-replication"> - <title> - Queue State Replication - </title> - - <section role="h2" id="queuestatereplication-AsynchronousReplicationofQueueState"> - <title> - Asynchronous - Replication of Queue State - </title> - - <section role="h3" id="queuestatereplication-Overview"> - <title> - Overview - </title> - <para> - There is support in qpidd for selective asynchronous replication - of queue state. This is achieved by: - </para> - <para> - (a) enabling event generation for the queues in question - </para> - <para> - (b) loading a plugin on the 'source' broker to encode those - events as messages on a replication queue (this plugin is - called - replicating_listener.so) - </para> - <para> - (c) loading a custom exchange plugin on the 'backup' broker (this - plugin is called replication_exchange.so) - </para> - <para> - (d) creating an instance of the replication exchange type on the - backup broker - </para> - <para> - (e) establishing a federation bridge between the replication - queue on the source broker and the replication exchange on the - backup broker - </para> - <para> - The bridge established between the source and backup brokers for - replication (step (e) above) should have acknowledgements turned - on (this may be done through the --ack N option to qpid-route). - This ensures that replication events are not lost if the bridge - fails. - </para> - <para> - The replication protocol will also eliminate duplicates to ensure - reliably replicated state. Note though that only one bridge per - replication exchange is supported. If clients try to publish to - the replication exchange or if more than a the single required - bridge from the replication queue on the source broker is - created, replication will be corrupted. (Access control may be - used to restrict access and help prevent this). - </para> - <para> - The replicating event listener plugin (step (b) above) has the - following options: - </para> - <programlisting> -Queue Replication Options: - --replication-queue QUEUE Queue on which events for - other queues are recorded - --replication-listener-name NAME (replicator) name by which to register the - replicating event listener - --create-replication-queue if set, the replication will - be created if it does not - exist - </programlisting> - <para> - The name of the queue is required. It can either point to a - durable queue whose definition has been previously recorded, or - the --create-replication-queue option can be specified in which - case the queue will be created a simple non-durable queue if it - does not already exist. - </para> - <!--h3--> - </section> - - <section role="h3" id="queuestatereplication-UsewithClustering"> - <title> - Use with - Clustering - </title> - <para> - The source and/or backup brokers may also be clustered brokers. - In this case the federated bridge will be re-established between - replicas should either of the originally connected nodes fail. - There are however the following limitations at present: - </para> - <itemizedlist> - <listitem> - <para>The backup site does not process membership updates after it - establishes the first connection. In order for newly added - members on a source cluster to be eligible as failover targets, - the bridge must be recreated after those members have been added - to the source cluster. - </para> - </listitem> - </itemizedlist> - <itemizedlist> - <listitem> - <para>New members added to a backup cluster will not receive - information about currently established bridges. Therefore in - order to allow the bridge to be re-established from these members - in the event of failure of older nodes, the bridge must be - recreated after the new members have joined. - </para> - </listitem> - </itemizedlist> - <itemizedlist> - <listitem> - <para>Only a single URL can be passed to create the initial link - from backup site to the primary site. this means that at the time - of creating the initial connection the initial node in the - primary site to which the connection is made needs to be running. - Once connected the backup site will receive a membership update - of all the nodes in the primary site, and if the initial - connection node in the primary fails, the link will be - re-established on the next node that was started (time) on the - primary site. - </para> - </listitem> - </itemizedlist> - <para> - Due to the acknowledged transfer of events over the bridge (see - note above) manual recreation of the bridge and automatic - re-establishment of te bridge after connection failure (including - failover where either or both ends are clustered brokers) will - not result in event loss. - </para> - <!--h3--> - </section> - - <section role="h3" id="queuestatereplication-OperationsonBackupQueues"> - <title> - Operations - on Backup Queues - </title> - <para> - When replicating the state of a queue to a backup broker it is - important to recognise that any other operations performed - directly on the backup queue may break the replication. - </para> - <para> - If the backup queue is to be an active (i.e. accessed by clients - while replication is on) only enqueues should be selected - for - replication. In this mode, any message enqueued on the source - brokers copy of the queue will also be enqueued on the backup - brokers copy. However not attempt will be made to remove messages - from the backup queue in response to removal of messages from the - source queue. - </para> - <!--h3--> - </section> - - <section role="h3" id="queuestatereplication-SelectingQueuesforReplication"> - <title> - Selecting - Queues for Replication - </title> - <para> - Queues are selected for replication by specifying the types of - events they should generate (it is from these events that the - replicating plugin constructs messages which are then pulled and - processed by the backup site). This is done through options - passed to the initial queue-declare command that creates the - queue and may be done either through qpid-config or similar - tools, or by the application. - </para> - <para> - With qpid-config, the --generate-queue-events options is used: - </para> - <programlisting> - --generate-queue-events N - If set to 1, every enqueue will generate an event that can be processed by - registered listeners (e.g. for replication). If set to 2, events will be - generated for enqueues and dequeues - </programlisting> - <para> - From an application, the arguments field of the queue-declare - AMQP command is used to convey this information. An entry should - be added to the map with key 'qpid.queue_event_generation' and an - integer value of 1 (to replicate only enqueue events) or 2 (to - replicate both enqueue and dequeue events). - </para> - <para> - Applications written using the c++ client API may fine the - qpid::client::QueueOptions class convenient. This has a - enableQueueEvents() method on it that can be used to set the - option (the instance of QueueOptions is then passed as the value - of the arguments field in the queue-declare command. The boolean - option to that method should be set to true if only enequeue - events should be replicated; by default it is false meaning that - both enqueues and dequeues will be replicated. E.g. - </para> - <programlisting> - QueueOptions options; - options.enableQueueEvents(false); - session.queueDeclare(arg::queue="my-queue", arg::arguments=options); - </programlisting> - <!--h3--> - </section> - - <section role="h3" id="queuestatereplication-Example"> - <title> - Example - </title> - <para> - Lets assume we will run the primary broker on host1 and the - backup on host2, have installed qpidd on both and have the - replicating_listener and replication_exchange plugins in qpidd's - module directory(*1). - </para> - <para> - On host1 we start the source broker and specifcy that a queue - called 'replication' should be used for storing the events until - consumed by the backup. We also request that this queue be - created (as transient) if not already specified: - </para> - <programlisting> - qpidd --replication-queue replication-queue --create-replication-queue true --log-enable info+ - </programlisting> - <para> - On host2 we start up the backup broker ensuring that the - replication exchange module is loaded: - </para> - <programlisting> - qpidd - </programlisting> - <para> - We can then create the instance of that replication exchange that - we will use to process the events: - </para> - <programlisting> - qpid-config -a host2 add exchange replication replication-exchange - </programlisting> - <para> - If this fails with the message "Exchange type not implemented: - replication", it means the replication exchange module was - not - loaded. Check that the module is installed on your system and if - necessary provide the full path to the library. - </para> - <para> - We then connect the replication queue on the source broker with - the replication exchange on the backup broker using the - qpid-route command: - </para> - <programlisting> - qpid-route --ack 50 queue add host2 host1 replication-exchange replication-queue -</programlisting> - <para> - The example above configures the bridge to acknowledge messages - in batches of 50. - </para> - <para> - Now create two queues (on both source and backup brokers), one - replicating both enqueues and dequeues (queue-a) and the - other - replicating only dequeues (queue-b): - </para> - <programlisting> - qpid-config -a host1 add queue queue-a --generate-queue-events 2 - qpid-config -a host1 add queue queue-b --generate-queue-events 1 - - qpid-config -a host2 add queue queue-a - qpid-config -a host2 add queue queue-b - </programlisting> - <para> - We are now ready to use the queues and see the replication. - </para> - <para> - Any message enqueued on queue-a will be replicated to the backup - broker. When the message is acknowledged by a client connected to - host1 (and thus dequeued), that message will be removed from the - copy of the queue on host2. The state of queue-a on host2 will - thus mirror that of the equivalent queue on host1, albeit with a - small lag. (Note - however that we must not have clients connected to host2 publish - to-or consume from- queue-a or the state will fail to replicate - correctly due to conflicts). - </para> - <para> - Any message enqueued on queue-b on host1 will also be enqueued on - the equivalent queue on host2. However the acknowledgement and - consequent dequeuing of messages from queue-b on host1 will have - no effect on the state of queue-b on host2. - </para> - <para> - (*1) If not the paths in the above may need to be modified. E.g. - if using modules built from a qpid svn checkout, the following - would be added to the command line used to start qpidd on host1: - </para> - <programlisting> - --load-module <path-to-qpid-dir>/src/.libs/replicating_listener.so - </programlisting> - <para> - and the following for the equivalent command line on host2: - </para> - <programlisting> - --load-module <path-to-qpid-dir>/src/.libs/replication_exchange.so - </programlisting> - <!--h3--> - </section> - <!--h2--> - </section> -</section> |