QPID-4360: Non-ready HA broker can be incorrectly promoted to primary

A joining broker now attempts to contact all known members of the cluster and
check their status. If any brokers are in a state other than "joining" the
broker will refuse to promote. This will allow rgmanager to continue to try
addresses till it finds a ready brokers.

Note this reqiures ha-brokers-url to be a list of all known brokers, not a
virtual IP.  ha-public-url can still be a VIP.

git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk/qpid@1394706 13f79535-47bb-0310-9956-ffa450edef68

1 files changed, 14 insertions, 0 deletions

diff --git a/cpp/src/tests/ha_tests.py b/cpp/src/tests/ha_tests.py
index f1620cf55d..a8d16a77c9 100755
--- a/cpp/src/tests/ha_tests.py
+++ b/cpp/src/tests/ha_tests.py
@@ -866,6 +866,20 @@ class RecoveryTests(BrokerTest):
         s.sync(timeout=1)      # And released after the timeout.
         self.assertEqual(cluster[2].ha_status(), "active")
 
+    def test_join_ready_cluster(self):
+        """If we join a cluster where the primary is dead, the new primary is
+        not yet promoted and there are ready backups then we should refuse
+        promotion so that one of the ready backups can be chosen."""
+        # FIXME aconway 2012-10-05: smaller timeout
+        cluster = HaCluster(self, 2, args=["--link-heartbeat-interval", 1])
+        cluster[0].wait_status("active")
+        cluster[1].wait_status("ready")
+        cluster.bounce(0, promote_next=False)
+        self.assertRaises(Exception, cluster[0].promote)
+        os.kill(cluster[1].pid, signal.SIGSTOP) # Test for timeout if unresponsive.
+        cluster.bounce(0, promote_next=False)
+        cluster[0].promote()
+
 if __name__ == "__main__":
     shutil.rmtree("brokertest.tmp", True)
     qpid_ha = os.getenv("QPID_HA_EXEC")