diff options
Diffstat (limited to 'qpid/cpp/design_docs/new-cluster-plan.txt')
| -rw-r--r-- | qpid/cpp/design_docs/new-cluster-plan.txt | 86 |
1 files changed, 53 insertions, 33 deletions
diff --git a/qpid/cpp/design_docs/new-cluster-plan.txt b/qpid/cpp/design_docs/new-cluster-plan.txt index 6fb9d3fd9f..32e3f710e7 100644 --- a/qpid/cpp/design_docs/new-cluster-plan.txt +++ b/qpid/cpp/design_docs/new-cluster-plan.txt @@ -25,8 +25,8 @@ Meaning of priorities: [#C] Can be addressed in a later release. The existig prototype is bare bones to do performance benchmarks: -- Implement publish and consumer locking protocol. -- Defered delivery and asynchronous completion of message till self-delivered. +- Implements publish and consumer locking protocol. +- Defered delivery and asynchronous completion of message. - Optimize the case all consumers are on the same node. - No new member updates, no failover updates, no transactions, no persistence etc. @@ -79,7 +79,6 @@ Multicast using fixed-size (64k) buffers, allow fragmetation of messages across * Open questions - ** TODO [#A] Queue sequence numbers vs. independant message IDs. SCHEDULED: <2011-10-07 Fri> @@ -94,18 +93,18 @@ However they imply additional map lookups that might hurt performance. - [ ] Prototype independent message IDs, check performance. * Outstanding Tasks -** TODO [#A] Defer and async complete wiring commands. +** TODO [#A] Defer and async completion of wiring commands. Testing requirement: Many tests assume wiring changes are visible across the cluster once the commad completes. -Name clashes: avoid race if same name queue/exchange declared on 2 -brokers simultaneously +Name clashes: need to avoid race if same name queue/exchange declared +on 2 brokers simultaneously -** TODO [#B] Management support +** TODO [#A] Passing all existing cluster tests. -- Replicate management methods that modify queues - e.g. move, purge. -- Report connections - local only or cluster-wide? +The new cluster should be a drop-in replacement for the old, so it +should be able to pass all the existing tests. ** TODO [#A] Update to new members joining. @@ -140,29 +139,28 @@ Exchange updatee: Updater remains active throughout. Updatee stalls clients until the update completes. -** TODO [#B] TX transaction support. -Extend broker::Cluster interface to capture transaction context and completion. -Running brokers exchange TX information. -New broker update includes TX information. +Updating queue/exchange/binding objects is via the same encode/decode +that is used by the store. Updatee to use recovery interfaces to +recover? -** TODO [#B] DTX transaction support. -Extend broker::Cluster interface to capture transaction context and completion. -Running brokers exchange DTX information. -New broker update includes DTX information. -** TODO [#B] Async completion of accept. -When this is fixed in the standalone broker, it should be fixed for cluster. +** TODO [#A] Failover updates to client. +Implement the amq.failover exchange to notify clients of membership. -** TODO [#B] Persistence support. -InitialStatus protoocl etc. to support persistent start-up (existing code) +** TODO [#B] Initial status protocol. +Handshake to give status of each broker member to new members joining. +Status includes +- persistent store state (clean, dirty) +- cluster protocol version. + +** TODO [#B] Persistent cluster support. +Initial status protoocl to support persistent start-up (see existing code) Only one broker recovers from store, update to others. Assign cluster IDs to messages recovered from store, don't replicate. See Queue::recover. -** TODO [#B] Handle other ways that messages can leave a queue. - -Ways other than a consumer that messages are taken off a queue. - +** TODO [#B] Management support +Replicate management methods that modify queues - e.g. move, purge. Target broker may not have all messages on other brokers for purge/destroy. - Queue::move() - need to wait for lock? Replicate? - Queue::get() - ??? @@ -171,6 +169,26 @@ Target broker may not have all messages on other brokers for purge/destroy. Need to add callpoints & mcast messages to replicate these? +** TODO [#B] TX transaction support. +Extend broker::Cluster interface to capture transaction context and completion. +Running brokers exchange TX information. +New broker update includes TX information. + + // FIXME aconway 2010-10-18: As things stand the cluster is not + // compatible with transactions + // - enqueues occur after routing is complete + // - no call to Cluster::enqueue, should be in Queue::process? + // - no transaction context associated with messages in the Cluster interface. + // - no call to Cluster::accept in Queue::dequeueCommitted + +** TODO [#B] DTX transaction support. +Extend broker::Cluster interface to capture transaction context and completion. +Running brokers exchange DTX information. +New broker update includes DTX information. + +** TODO [#B] Async completion of accept. +When this is fixed in the standalone broker, it should be fixed for cluster. + ** TODO [#B] Network partitions and quorum. Re-use existing implementation. @@ -209,25 +227,27 @@ The old cluster has workarounds in the broker code that can be removed. - [ ] drop security workarounds: cluster code now operates after message decoding. - [ ] drop connection tracking in cluster code. - [ ] simper inconsistent-error handling code, no need to stall. -** TODO [#C] Support for live updates. +** TODO [#C] Support for live upgrades. + Allow brokers in a running cluster to be replaced one-by-one with a new version. The old cluster protocol was unstable because any changes in broker state caused changes to the cluster protocol.The new design should be much more stable. -TODO: think about strategies for allowing live updates while extending -the cluster protocol - - - - +Points to implement: +- Brokers should ignore unknown controls (with a warning) rather than an error. +- Limit logging frequency for unknown control warnings. +- Add a version number at front of every CPG message. Determines how the + rest of the message is decoded. (allows for entirely new encodings e.g. AMQP 1.0) +- Protocol version XML element in cluster.xml, on each control. +- Initial status protocol to include protocol version number. ** TODO [#C] Support for AMQP 1.0. * Testing ** TODO [#A] Pass all existing cluster tests. -Requires [[Defer and async complete wiring commands.]] +Requires [[Defer and async completion of wiring commands.]] ** TODO [#A] New cluster tests. Stress tests & performance benchmarks focused on changes in new cluster: - concurrency by queues rather than connections. |