[rabbitmq-discuss] High Scalability and High Availability...How to?!!

[Paul Holyoake]

I've recently had a similar HA and scalability requirement. I couldn't find 
a perfect answer. I achieved HA through a network loadbalancer across a 
pair of clustered RabbitMQ nodes, with mirrored queues. If the NLB fails to 
connect on port 5672 of the primary it failsover to the secondary node. All 
queue data is available on this node due to the queue mirroring.

In order to achieve scalability my assumption is to logically partition the 
data messages and have over clusters to meet that requirement.

Would the NLB in round robin mode be viable and have the nodes effectively 
active/active?



On Thursday, June 12, 2014 5:47:30 PM UTC+1, Shadi Mari wrote:

> Hello,
>
> I am new to Rabbitmq, however i have built so many messaging solution on 
> top of other stacks..
>
> I am trying to leverage the capabilities of RabbitMQ in one of my 
> solutions where HA and high scalability  is a must; so far the better 
> option the came out of my research 
> is to use active/active cluster and mirrored queues in order to provide 
> HA, otherwise using active/standby where the queue lives on the node it was 
> created on, will not achieve HA requirement.
>
> On the other hand, this should be mixed a load balancer, where consumers 
> and publishers are connected to the cluster through in order to direct 
> consumers/publishers to a different node each time the connection takes 
> place.
>
> However, my understanding says that RabbiqMQ in the case of mirrored queue 
> will connect consumers and publishers to the master node which will not 
> help in distributing load across all nodes. I am not sure but it came to my 
> knowledge that SoundCloud uses the same approach!
>
> Is my understanding above correct? if Yes, then how do companies that uses 
> RabbitMQ and require HA and high scalability work it out.
>
> Thank you
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140613/ef261575/attachment.html>