git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk@1740289 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk@1706480 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk@1642758 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk@1622057 13f79535-47bb-0310-9956-ffa450edef68

This commit adds a --cluster-manager flag to qpid-ha tool.

Without this flag
- the 'promote' command is not listed in the tool help.
- using the promote command raises an error saying that it is only for cluster manager use
  and mentioning the --cluster-manager flag.

With the flag: promote functions as before.

The qpid-ha help text for promote is also more clear now that it is for cluster
manager only.

Originally the idea was to split qpid-ha into two tools but I have kept one tool
with the flag and warning messages because it:

- avoids packaging changes that might trip things up.

- helps people who are already using qpid-ha promote: their scripts will
  break but the error message explains how to fix it.

I think the special role of promote is sufficiently clear now even if it is
part of the same tool.

This commit also updates the following to take account of the new flag:
- rgmanager qpidd-primary script.
- qpidd tests.
- qpid book HA chapter.

NOTE: THIS WILL BREAK TEST HARNESSES that do promotion outside of rgmanager.
You'll need to add the --cluster-manager flag in the relevant places.

git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk@1619877 13f79535-47bb-0310-9956-ffa450edef68

There are two issues here, both related to the fact that rgmanager sees qpidd
and qpidd-primary as two separate services.

1. The service start/stop scripts can be called concurrently. This can lead to
   running a qpidd process who's pid is not in the pidfile. rgmanager cannot
   detect or kill this qpidd and cannot start another qpidd because of the lock
   on the qpidd data directory.

2. rgmanager sees a primary failure as two failures: qpidd and qpidd-primary,
   and will then try to stop and start both services. The order of these actions
   is not defined and can lead to rgmanager killing a service it has just
   started.

This patch makes two major changes to the init scripts:

1. Uses flock to lock the sensitive stop/start part of the scripts to ensure
   they are not executed concurrently.

2. On "stop" the scripts check if a running qpidd is primary or not. "qpidd stop"
   is a no-op if the running broker is primary, "qpidd-primary stop" is a no op
   if it is not. This ensures that a broker will be stopped by the same stream
   of service actions that started it.

Minor changes in this patch:
- better logging of broker start-up and shut-down sequence.
- qpid-ha heartbeat use half of timeout option.
- add missing timeouts in qpid-ha.


Notes:

This changes the behavior of 'clusvcadm -d <qpidd-service>' on the primary node.
Previously this would have stopped the qpidd service on that node, killed the
qpidd process and relocated the primary service. Now this will stop the qpidd
service (as far as rgmanager is concerned) but will not kill qpidd or relocate
the primary service. When the primary is relocated the qpidd service wil not be
able to re-start on that node until it is re-enabled with 'clusvcadm -e'.

git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk@1614895 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk@1611747 13f79535-47bb-0310-9956-ffa450edef68

git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk@1590118 13f79535-47bb-0310-9956-ffa450edef68

- Remove some dead code.
- Removed "set" command - not ready for production. All settings in qpidd.conf.
  - Removed related tests in ha_tests
- Improved help on promote command.
- Made option group for common broker connection options.

git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk@1589834 13f79535-47bb-0310-9956-ffa450edef68

- Added timeout to qpid-ha.
- qpidd init script pings broker to verify it is not hung.
- updated documentation in qpid/doc/book/src/cpp-broker/Active-Passive-Cluster.xml.

The new results for the cases mentioned in the bug:

a] stopped ALL brokers: rgmanager restarts the entire cluster but data is lost.
   Equivalent to killing all the  brokers at once. This does not affect quorum because
   only qpidd services are affected, not other services managed by cman.

b] stopped the primary: rgmanager restarts the primary after a timeout and promotes one of the backups.

c] stopped a backup: rgmanager restarts the backups after a timeout.
   Clients that are actively sending messages may see a delay while backup is restarted.

Note you need to set link-heartbeat-interval in qpidd.conf. The default is very
high (120 seconds), it should be set lower to see recovery from sigstop in a
reasonable time.
See the updated documentation in qpid/doc/book/src/cpp-broker/Active-Passive-Cluster.xml.

git-svn-id: https://svn.apache.org/repos/asf/qpid/trunk@1589807 13f79535-47bb-0310-9956-ffa450edef68