path: root/doc/book/src/How-to-Use-SlowConsumerDisconnect.xml

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.1.2//EN"
	"http://www.oasis-open.org/docbook/xml/4.1.2/docbookx.dtd">
<!--
 
 Licensed to the Apache Software Foundation (ASF) under one
 or more contributor license agreements.  See the NOTICE file
 distributed with this work for additional information
 regarding copyright ownership.  The ASF licenses this file
 to you under the Apache License, Version 2.0 (the
 "License"); you may not use this file except in compliance
 with the License.  You may obtain a copy of the License at
 
   http://www.apache.org/licenses/LICENSE-2.0
 
 Unless required by applicable law or agreed to in writing,
 software distributed under the License is distributed on an
 "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 KIND, either express or implied.  See the License for the
 specific language governing permissions and limitations
 under the License.
 
-->
<section>
  <title>Slow Consumer Disconnect - User Guide</title>

  <section>
<title>Introduction</title>
   <para>Slow Consumer Disconnect (SCD) is a new feature in Qpid that provides a configurable
        mechanism to prevent a single slow consumer from causing a back up of unconsumed messages on
        the broker. </para>

   <para>This is most relevant where Topics are in use, since a published message is not removed
        from the broker's memory until all subscribers have acknowledged that message. </para>

   <para>Cases where a consumer is 'slow' can arise due to one of the following: poor network
        connectivity exists; a transient system issue affects a single client; a single subscriber
        written by a client team is behaving incorrectly and not acknowledging messages; a
        downstream resource such as a database is non-responsive. </para>

   <para>SCD will enable the application owner to configure limits for a given consumer's queue and
        the behaviour to execute when those limits are reached. </para>

  </section>

  <section>
<title>What can it do?</title>
   <para>SCD is only applicable to topics or durable subscriptions and can be configured on either
        a topic or a subscription name. </para>

   <para>On triggering of a specified threshold the offending client will be disconnected from the
        broker with a 506 error code wrapped in a JMSException returned to the client via the
        ExceptionListener registered on the Connection object. </para>

   <para>Note that it is essential that an ExceptionListener be specified by the client on
        creation of the connection and that exceptions coming back on that listener are handled
        correctly. </para>

  </section>

  <section>
<title>Frequency of SCD Checking</title>
   <section>
<title><emphasis role='bold'>Configuring Frequency</emphasis></title>
    <para>You can configure the frequency with which the SCD process will check for slow consumers,
          along with the unit of time used to specify that frequency. </para>

    <para>The <emphasis role="italic">virtualhosts.virtualhost.hostname.slow-consumer-detection</emphasis>
          elements <emphasis role="italic">delay</emphasis> and <emphasis role="italic">timeunit</emphasis>
          are used to specify the frequency and timeunit respectively in the virtualhosts.xml
          file e.g. </para>

<programlisting>
&lt;virtualhosts&gt;
	&lt;default&gt;test&lt;/default&gt;
	&lt;virtualhost&gt;
		&lt;name&gt;test&lt;/name&gt;
		&lt;test&gt;
		   &lt;slow-consumer-detection&gt;
			&lt;delay&gt;60&lt;delay/&gt;
			&lt;timeunit&gt;seconds&lt;timeunit/&gt;
		&lt;slow-consumer-detection/&gt;
		&lt;/test&gt;
	&lt;/virtualhost&gt;
&lt;/virtualhosts&gt;
</programlisting>

   </section>

   <section>
<title><emphasis role='bold'>SCD Log output</emphasis></title>
    <para>When the SCD component finds a queue with a configured threshold to check, the operational
          logging component (if enabled) will output the following line:</para>

    <programlisting>
    SCD-1003 : Checking Status of Queue
    </programlisting>

   </section>

  </section>

  <section>
<title>Client Exception<emphasis role='bold'>s</emphasis></title>
   <para>When a Slow Consumer is disconnected, the client receives a 506 error from the broker
        wrapped in a JMSException and the Session and Connection are closed:</para>

<programlisting>
Dispatcher-Channel-1 2010-09-01 16:23:34,206 INFO [qpid.client.AMQSession.Dispatcher]
    Dispatcher-Channel-1 thread terminating for channel 1:org.apache.qpid.client.AMQSession_0_8@1de8aa8
pool-2-thread-3 2010-09-01 16:23:34,238 INFO [apache.qpid.client.AMQConnection] Closing AMQConnection due to
    :org.apache.qpid.AMQChannelClosedException: Error: Consuming to slow. [error code 506: resource error]
javax.jms.JMSException: 506
at org.apache.qpid.client.AMQConnection.exceptionReceived(AMQConnection.java:1396)
at org.apache.qpid.client.protocol.AMQProtocolHandler.exception(AMQProtocolHandler.java:329)
at org.apache.qpid.client.protocol.AMQProtocolHandler.methodBodyReceived(AMQProtocolHandler.java:536)
at org.apache.qpid.client.protocol.AMQProtocolSession.methodFrameReceived(AMQProtocolSession.java:453)
at org.apache.qpid.framing.AMQMethodBodyImpl.handle(AMQMethodBodyImpl.java:93)
at org.apache.qpid.client.protocol.AMQProtocolHandler$1.run(AMQProtocolHandler.java:462)
at org.apache.qpid.pool.Job.processAll(Job.java:110)
at org.apache.qpid.pool.Job.run(Job.java:149)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
Caused by: org.apache.qpid.AMQChannelClosedException: Error: Consuming to slow. [error code 506: resource error]
at org.apache.qpid.client.handler.ChannelCloseMethodHandler.methodReceived(ChannelCloseMethodHandler.java:96)
at org.apache.qpid.client.handler.ClientMethodDispatcherImpl.dispatchChannelClose(ClientMethodDispatcherImpl.java:163)
at org.apache.qpid.framing.amqp_8_0.ChannelCloseBodyImpl.execute(ChannelCloseBodyImpl.java:140)
at org.apache.qpid.client.state.AMQStateManager.methodReceived(AMQStateManager.java:112)
at org.apache.qpid.client.protocol.AMQProtocolHandler.methodBodyReceived(AMQProtocolHandler.java:511)
... 8 more
main 2010-09-01 16:23:34,316 INFO [apache.qpid.client.AMQSession] Closing session:
    org.apache.qpid.client.AMQSession_0_8@ffeef1
</programlisting>

  </section>

  <section>
<title>Disconnection Thresholds</title>
   <section>
<title>Topic Subscriptions</title>
    <para>One key feature of SCD is the disconnection of a consuming client when a specified
          threshold is exceeded. For a pub-sub model using topics, this means that messages will no
          longer be delivered to the private queue which was associated with that consuming client,
          thus reducing any associated backlog in the broker. </para>

   </section>

   <section>
<title>Durable Topic Subscriptions</title>
    <para>For durable subscriptions, simply disconnecting the consuming client will not suffice
          since the associated queue is by definition durable and messages would continue to flow to
          it after disconnection, potentially worsening any backing up of data on the broker. </para>

    <para>The solution is to configure durable subscriptions to delete the underlying queue on
          disconnection. This means that messages will no longer be delivered to the private queue
          associated with the subscription, thus preventing any backlog. </para>

    <para>Full details of how to configure the thresholds are provided below. </para>

   </section>

   <section>
<title>Message Age Threshold</title>
    <para>You can configure SCD to be triggered on a topic or subscription when the oldest message
          in the associated private queue for the consumer ages beyond the specified value, in
          milliseconds. </para>

   </section>

   <section>
<title>Queue Depth Threshold</title>
    <para>You can opt to use the depth of the queue in bytes as a threshold. SCD will be triggered
          by a queue depth greater than the threshold specified i.e. when a broker receives a
          message that takes the queue depth over the threshold. </para>

   </section>

   <section>
<title>Message Count Threshold</title>
    <para>You can use the message count for the consumer's queue as the trigger, where a count
          higher than that specified will trigger disconnection. </para>

   </section>

   <section>
<title><emphasis role='bold'>Delete Policy</emphasis></title>
    <para>You can configure the policy you wish to apply in your broker configuration. There are
          currently 2 policies available: </para>

    <para>
<emphasis role='bold'>Delete Temporary Queues Only</emphasis>
    </para>

    <para>If you do not specify a &lt;topicDelete/&gt; element in your configuration, then only temporary
          queues associated with a topic subscription will be deleted on client disconnect. This is
          the default behaviour. </para>
        <para/>

    <para>
<emphasis role='bold'>Delete Durable Subscription Queues</emphasis>
    </para>

    <para>If you add the &lt;topicDelete/&gt; element with the sub-element
          &lt;delete-persistent/&gt; to your config, then the persistent queue which is associated
          with durable subscriptions to a topic will also be deleted. This is an important
          consideration since without deleting the underlying queue the client's unconsumed data
          will grow indefinitely while they will be unable to reconnect to that queue due to the SCD
          threshold configured, potentially having an adverse effect on the application or broker in
          use.</para>
        <para/>

    <para><emphasis role="bold"> Example Topic Configuration </emphasis></para>

    <para/>

    <para>
The following steps are required to configure SCD:
    </para>

<itemizedlist>
    <listitem>
    <para>Enable SCD checking for your virtual host</para>
    </listitem>
    <listitem>
    <para>Specify frequency for SCD checking</para>
    </listitem>
    <listitem>
    <para>Define thresholds for the topic</para>
    </listitem>
    <listitem>
    <para>Define the policy to apply on trigger </para>
    </listitem>
</itemizedlist>

    <para>The example below shows a simple definition, with all three thresholds specified and a
          simple disconnection, with deletion of any temporary queue, defined. </para>

    <para>For a durable subscription to this topic, no queue deletion would be applied on disconnect
          - which is likely to be undesirable (see section above). </para>

<programlisting>
&lt;topics&gt;
	 &lt;topic&gt;
	 &lt;name&gt;stocks.us.*&lt;/name&gt;
		 &lt;slow-consumer-detection&gt;
			 &lt;!-- The maximum depth before which --&gt;
			 &lt;!-- the policy will be applied--&gt;
			 &lt;depth&gt;4235264&lt;/depth&gt;
			 &lt;!-- The maximum message age before which --&gt;
			 &lt;!-- the policy will be applied--&gt;
			 &lt;messageAge&gt;600000&lt;/messageAge&gt;
			 &lt;!-- The maximum number of message before --&gt;
			 &lt;!-- which the policy will be applied--&gt;
			 &lt;messageCount&gt;50&lt;/messageCount&gt;
			 &lt;!-- Policy Selection --&gt;
			 &lt;policy name="TopicDelete"/&gt;
		 &lt;/slow-consumer-detection&gt;
	 &lt;/topic&gt;
&lt;/topics&gt;
</programlisting>

   </section>

  </section>

  <section>
<title>Important Points To Note</title>
   <para> Client application developers should be educated about how to correctly handle being
        disconnected with a 506 error code, to avoid them getting into a thrashing state where they
        continually attempt to connect, fail to consume fast enough and are disconnected again. </para>

   <para>Clients affected by slow consumer disconnect configuration should always use transactions
        where duplicate processing of an incoming message would have adverse affects, since they may
        receive a message more than once if disconnected before acknowledging a message in flight. </para>

  </section>

 </section>