diff options
author | Kenneth Anthony Giusti <kgiusti@apache.org> | 2011-03-09 15:17:52 +0000 |
---|---|---|
committer | Kenneth Anthony Giusti <kgiusti@apache.org> | 2011-03-09 15:17:52 +0000 |
commit | 1d5a430601df6b557b06f4858a91e1c0eb239e1c (patch) | |
tree | 221e8925d9c3d3eaf7ce6b7f8e0fd331d8a3ef11 /qpid/doc | |
parent | 2856fb36dd246f77ff8e209eb4441b537e368696 (diff) | |
download | qpid-python-1d5a430601df6b557b06f4858a91e1c0eb239e1c.tar.gz |
QPID-3128: add producer flow control info to C++ Broker docs
git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk@1079830 13f79535-47bb-0310-9956-ffa450edef68
Diffstat (limited to 'qpid/doc')
4 files changed, 360 insertions, 1 deletions
diff --git a/qpid/doc/book/src/AMQP-Messaging-Broker-CPP-Book.xml b/qpid/doc/book/src/AMQP-Messaging-Broker-CPP-Book.xml index 309492262d..6c27d7c668 100644 --- a/qpid/doc/book/src/AMQP-Messaging-Broker-CPP-Book.xml +++ b/qpid/doc/book/src/AMQP-Messaging-Broker-CPP-Book.xml @@ -56,6 +56,7 @@ <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="LVQ.xml"/> <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="queue-state-replication.xml"/> <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="Starting-a-cluster.xml"/> + <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="producer-flow-control.xml"/> <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="AMQP-Compatibility.xml"/> <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="Qpid-Interoperability-Documentation.xml"/> diff --git a/qpid/doc/book/src/AMQP-Messaging-Broker-CPP.xml b/qpid/doc/book/src/AMQP-Messaging-Broker-CPP.xml index b9c513e511..15f5660455 100644 --- a/qpid/doc/book/src/AMQP-Messaging-Broker-CPP.xml +++ b/qpid/doc/book/src/AMQP-Messaging-Broker-CPP.xml @@ -50,7 +50,8 @@ <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="LVQ.xml"/> <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="queue-state-replication.xml"/> <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="Starting-a-cluster.xml"/> - <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="ACL.xml"/> + <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="ACL.xml"/> + <xi:include xmlns:xi="http://www.w3.org/2001/XInclude" href="producer-flow-control.xml"/> </chapter> diff --git a/qpid/doc/book/src/Cheat-Sheet-for-configuring-Queue-Options.xml b/qpid/doc/book/src/Cheat-Sheet-for-configuring-Queue-Options.xml index 60f1a4103d..d50948e0cc 100644 --- a/qpid/doc/book/src/Cheat-Sheet-for-configuring-Queue-Options.xml +++ b/qpid/doc/book/src/Cheat-Sheet-for-configuring-Queue-Options.xml @@ -65,6 +65,16 @@ </itemizedlist> </para></listitem> </itemizedlist> + + <para> + The 0.10 C++ Broker supports the following additional Queue configuration options: + </para> + <itemizedlist> + <listitem><para> + <xref linkend="producer-flow-control"/> + </para></listitem> + </itemizedlist> + <section role="h3" id="CheatSheetforconfiguringQueueOptions-ApplyingQueueSizingConstraints"><title> Applying Queue Sizing Constraints </title> diff --git a/qpid/doc/book/src/producer-flow-control.xml b/qpid/doc/book/src/producer-flow-control.xml new file mode 100644 index 0000000000..01ca4f8ac9 --- /dev/null +++ b/qpid/doc/book/src/producer-flow-control.xml @@ -0,0 +1,347 @@ +<?xml version="1.0" encoding="utf-8"?> +<!-- + + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +--> + +<section id="producer-flow-control"> + <title> + Producer Flow Control + </title> + + <section role="h2" id="producerflowcontrol-Overview"> + <title> + Overview + </title> + <para> + As of release 0.10, the C++ broker supports the use of flow control to + throttle back message producers that are at risk of overflowing a + destination queue. + </para> + + <para> + Each queue in the C++ broker has two threshold values associated with it: + </para> + + <para> + Flow Stop Threshold: this is the level of queue resource + utilization above which flow control will be enabled. Once this + threshold is crossed, the queue is considered in danger of overflow. + </para> + + <para> + Flow Resume Threshold - this is the level of queue resource utilization + below which flow control will be disabled. Once this threshold is + crossed, the queue is no longer considered in danger of overflow. + </para> + + <para> + In the above description, queue resource utilization may be + defined as the total count of messages currently enqueued, or the total + sum of all message content in bytes. + </para> + + <para> + The value for a queue's Flow Stop Threshold must be greater than or + equal to the value of the queue's Flow Resume Threshold. + </para> + + <section role="h3" id="producerflowcontrol-QueueThresholdsExample"> + <title> + Example + </title> + + <para> + Let's consider a queue with a maximum limit set on the total number of + messages that may be enqueued to that queue. Assume this maximum + message limit is 1000 messages. Assume also that the user configures a + Flow Stop Threshold of 900 messages, and a Flow Resume Threshold of 500 + messages. Then the following holds: + </para> + + <para> + When the total number of enqueued messages is greater than or equal to + 900, the queue's flow control state is OFF. + </para> + + <para> + When the total number of enqueued messages is greater than 900, the + queue's flow control state transitions to ON. + </para> + + <para> + When the queue's flow control state is "ON", it remains "ON" until the + total number of enqueued messages is less than 500. At that point, the queue's + flow control state transitions to "OFF". + </para> + + <para> + A similar example using total enqueued content bytes as the threshold + units are permitted. + </para> + </section> + + <para> + Thresholds may be set using both total message counts and total byte + counts. In this case, the following rules apply: + </para> + + <para> + 1) Flow control is "ON" when either stop threshold value is crossed. + </para> + <para> + 2) Flow control remains "ON" until both resume thresholds are satisfied. + </para> + + <section role="h3" id="producerflowcontro-MultiThresholdExample"> + <title> + Example + </title> + + <para> + Let's consider a queue with a maximum size limit of 10K bytes, and 5000 + messages. A user may assign a Flow Stop Threshold based on a total + message count of 4000 messages. They may also assigne a Flow Stop + Threshold of 8K bytes. The queue's flow control state transitions to + "ON" if either threshold is crossed: (total-msgs greater-than 4000 OR total-bytes + greater-than 8K). + </para> + + <para> + Assume the user has assigned Flow Resume threshold's of 3000 messages and + 6K bytes. Then the queue's flow control will remain active until both + thresholds are satified: (total-msg less-than 3000 AND total-bytes less-than 6K). + </para> + </section> + + <para> + The Broker enforces flow control by delaying the completion of the + Message.Transfer command that causes a message to be delivered to a queue + with active flow control. The completion of the Message.Transfer command + is held off until flow control state transitions to "OFF" for all queues + that are a destination for that command. + </para> + + <para> + A message producing client is permitted to have a finite number of + commands pending completion. When the total number of these outstanding + commands reaches the limit, the client must not issue further commands + until one or more of the outstanding commands have completed. This + window of outstanding commands is considered the sender's "capacity". + This allows any given producer to have a "capacity's" worth of messages + blocked due to flow control before the sender must stop sending further + messages. + </para> + + <para> + This capacity window must be considered when determining a suitable + flow stop threshold for a given queue, as a producer may send its + capacity worth of messages _after_ a queue has reached the flow stop + threshold. Therefore, a flow stop threshould should be set such that + the queue can accomodate more messages without overflowing. + </para> + + <para> + For example, assume two clients, C1 and C2, are producing messages to + one particular destination queue. Assume client C1 has a configured + capacity of 50 messages, and client C2's capacity is 15 messages. In + this example, assume C1 and C2 are the only clients queuing messages to + a given queue. If this queue has a Flow Stop Threshold of 100 + messages, then, worst-case, the queue may receive up to 165 messages + before clients C1 and C2 are blocked from sending further messages. + This is due to the fact that the queue will enable flow control on + receipt of its 101'st message - preventing the completion of the + Message.Transfer command that carried the 101'st message. However, C1 + and C2 are allowed to have a total of 65 (50 for C1 and 15 for C2) + messages pending completion of Message.Transfer before they will stop + producing messages. Thus, up to 65 messages may be enqueued beyond the + flow stop threshold before the producers will be blocked. + </para> + </section> + + <section role="h2" id="producerflowcontrol-UserInterface"> + <title> + User Interface + </title> + + <para> + By default, the C++ broker assigns a queue's flow stop and flow resume + thresholds when the queue is created. The C++ broker also allows the + user to manually specify the flow control thresholds on a per queue + basis. + </para> + + <para> + However, queues that have been configured with a Limit Policy of type + RING or RING-STRICT do NOT have queue flow thresholds enabled by + default. The nature of a RING queue defines its behavior when its + capacity is reach: replace the oldest message. + </para> + + <para> + The flow control state of a queue can be determined by the "flowState" + boolean in the queue's QMF management object. The queue's management + object also contains a counter that increments each time flow control + becomes active for the queue. + </para> + + <para> + The broker applies a threshold ratio to compute a queue's default flow + control configuration. These thresholds are expressed as a percentage + of a queue's maximum capacity. There is one value for determining the + stop threshold, and another for determining the resume threshold. The + user may configure these percentages using the following broker + configuration options: + </para> + + <programlisting> + --default-flow-stop-threshold ("Queue capacity level at which flow control is activated.") + --default-flow-resume-threshold ("Queue capacity level at which flow control is de-activated.") + </programlisting> + + <para> + For example: + </para> + + <programlisting> + qpidd --default-flow-stop-threshold=90 --default-flow-resume-threshold=75 + </programlisting> + + <para> + Sets the default flow stop threshold to 90% of a queue's maximum + capacity and the flow resume threshold to 75% of the maximum capacity. + If a queue is created with a default-queue-limit of 10000 bytes, then + the default flow stop threshold would be 90% of 10000 = 9000 bytes and + the flow resume threshold would be 75% of 10000 = 7500. The same + computation is performed should a queue be created with a maximum size + expressed as a message count instead of a byte count. + </para> + + <para> + If not overridden by the user, the value of the + default-flow-stop-threshold is 80% and the value of the + default-flow-resume-threshold is 70%. + </para> + + <para> + The user may disable default queue flow control broker-wide by + specifying the value 0 for both of these configuration options. Note + that flow control may still be applied manually on a per-queue basis in + this case. + </para> + + <para> + The user may manually set the flow thresholds when creating a queue. + The following options may be provided when adding a queue using the + <command>qpid-config</command> command line tool: + </para> + + <programlisting> + --flow-stop-size=<replaceable>N</replaceable> Sets the queue's flow stop threshold to <replaceable>N</replaceable> total bytes. + --flow-resume-size=<replaceable>N</replaceable> Sets the queue's flow resume threshold to <replaceable>N</replaceable> total bytes. + --flow-stop-count=<replaceable>N</replaceable> Sets the queue's flow stop threshold to <replaceable>N</replaceable> total messages. + --flow-resume-count=<replaceable>N</replaceable> Sets the queue's flow resume threshold to <replaceable>N</replaceable> total messages. + </programlisting> + + <para> + Flow thresholds may also be specified in the + <command>queue.declare</command> method, via the + <command>arguments</command> parameter map. The following keys can be + provided in the arguments map for setting flow thresholds: + </para> + + <table> + <title>Queue Declare Method Flow Control Arguments</title> + <tgroup cols="2"> + <thead> + <row> + <entry>Key</entry> + <entry>Value</entry> + </row> + </thead> + <tbody> + <row> + <entry>qpid.flow_stop_size</entry> + <entry>integer - queue's flow stop threshold value in bytes</entry> + </row> + <row> + <entry>qpid.flow_resume_size</entry> + <entry>integer - queue's flow resume threshold value in bytes</entry> + </row> + <row> + <entry>qpid.flow_stop_count</entry> + <entry>integer - queue's flow stop threshold value as a message count</entry> + </row> + <row> + <entry>qpid.flow_resume_count</entry> + <entry>integer - queue's flow resume threshold value as a message count</entry> + </row> + </tbody> + </tgroup> + </table> + + <para> + The user may disable flow control on a per queue basis by setting + the flow-stop-size and flow-stop-count to zero for the queue. + </para> + + <para> + The current state of flow control for a given queue can be + determined by the "flowStopped" statistic. This statistic is + available in the queue's QMF management object. The value of + flowStopped is True when the queue's capacity has exceeded the + flow stop threshold. The value of flowStopped is False when the + queue is no longer blocking due to flow control. + </para> + + <para> + A queue will also track the number of times flow control has been + activated. The "flowStoppedCount" statistic is incremented each time + the queue's capacity exceeds a flow stop threshold. This statistic can + be used to monitor the activity of flow control for any given queue + over time. + </para> + + <table> + <title>Flow Control Statistics available in Queue's QMF Class</title> + <tgroup cols="3"> + <thead> + <row> + <entry>Statistic Name</entry> + <entry>Type</entry> + <entry>Description</entry> + </row> + </thead> + <tbody> + <row> + <entry>flowStopped</entry> + <entry>Boolean</entry> + <entry>If true, producers are blocked by flow control.</entry> + </row> + <row> + <entry>flowStoppedCount</entry> + <entry>count32</entry> + <entry>Number of times flow control was activated for this queue</entry> + </row> + </tbody> + </tgroup> + </table> + </section> + <!--h2--> + </section> |