[rabbitmq-discuss] mirrored queue cluster

[Michael Klishin]

On oct 4, 2013, at 6:51 p.m., "PATAR, SAGAR" <sp345s at att.com> wrote:

> So let's say we have 6 nodes in a cluster  ... I can user 3 nodes for master/slave mirror architecture for few queues and how will the other 3 node help us ??

Put a proxy in front of the cluster. Make your clients connect to it. This will
make sure queues (masters) are roughly evenly distributed between nodes.

Use replication factor = 2 or 3 for mirrored queues. Try modeling your apps
to use more queues, e.g. every consumer could use one or more queues.
As load increases, add one or two more cluster nodes, more consumers
and more producers (as needed). If your cluster will use more queues in total,
you should see increased overall throughput, while maintaining redundancy with
mirrored queues.

It may become an issue that the proxy itself becomes a bottleneck (e.g. its network
link is completely saturated). You can have multiple of those and make some clients
connect to proxy1, some to proxy2, etc, as network capacity permits.

There are other factors impacting cluster throughput and having a proxy is not
an absolutely best way to ensure data locality (avoiding messages being routed
between cluster nodes), but I hope these guidelines give you some idea.

If you have more specific questions or can explain what your topology and workflow
look like, we may be able to provide more specific recommendations.

MK



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20131004/018bb87b/attachment.pgp>