QPID-4144 HA broker deadlocks on broker::QueueRegistry lock and ha::Primary lock

Running tests repeatedly, the broker deadlocked with the attached stack trace.

The problem call sequences are:
1. QueueRegistry::destroy takes QueuerRegistry lock > ConfigurationObserver::queueDestroy > ha::Primary::queueDestroy takes Primary lock.
2. ConnectionObserver::opened cals Primary::opened lock> RemoteBackup>getQueues().eachQueue

This patch breaks the deadlock at both ends: QueueRegistry no longer holds the lock across the observer call and Primary does not hold the lock across eachQueue.

git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@1362584 13f79535-47bb-0310-9956-ffa450edef68

1 files changed, 10 insertions, 11 deletions

diff --git a/cpp/src/qpid/broker/QueueRegistry.cpp b/cpp/src/qpid/broker/QueueRegistry.cpp
index 2916d7bb93..1401356444 100644
--- a/cpp/src/qpid/broker/QueueRegistry.cpp
+++ b/cpp/src/qpid/broker/QueueRegistry.cpp
@@ -79,18 +79,17 @@ QueueRegistry::declare(const string& declareName, bool durable,
     return result;
 }
 
-void QueueRegistry::destroyLH (const string& name) {
-    QueueMap::iterator i = queues.find(name);
-    if (i != queues.end()) {
-        Queue::shared_ptr q = i->second;
-        queues.erase(i);
-        if (broker) broker->getConfigurationObservers().queueDestroy(q);
+void QueueRegistry::destroy(const string& name) {
+    Queue::shared_ptr q;
+    {
+        qpid::sys::RWlock::ScopedWlock locker(lock);
+        QueueMap::iterator i = queues.find(name);
+        if (i != queues.end()) {
+            Queue::shared_ptr q = i->second;
+            queues.erase(i);
+        }
     }
-}
-
-void QueueRegistry::destroy (const string& name){
-    RWlock::ScopedWlock locker(lock);
-    destroyLH (name);
+    if (broker && q) broker->getConfigurationObservers().queueDestroy(q);
 }
 
 Queue::shared_ptr QueueRegistry::find(const string& name){