[rabbitmq-discuss] Clustering and scaling

[Stuart Munro]

Sorry Eugene, my understanding may be wrong, I'll try again.

1. Ideally I would like a few queue consumers servers running with the
ability to add more if/when needed on-the-fly (this doesn't affect
RabbitMQ, but I'll get to that in a second)

2. I would like multiple RabbitMQ servers that are redundant, if one of
the RabbitMQ servers dies the cluster should continue working as normal, I
also want the ability to add additional RabbitMQ servers on-the-fly
without having to tell the queue consumers that a new RabbitMQ server now
exists. This is the bit I don't quite understand how to implement.


In order for section 2 to work (from my understanding) I would need to put
a load balancer in front of the RabbitMQ servers, therefore the queue
consumers don't care about what server they pick-up the job from - it also
allows me to add RabbitMQ servers on-the-fly without reconfiguring as the
queue consumer would simply hit the load balancer.

How do people currently implement multiple RabbitMQ servers?

Thanks


-- 
Stuart Munro

e: stuart at state.it
w: www.state.it
m: +44 (0) 7738 755574




On 07/09/2011 19:04, "Eugene Kirpichov" <ekirpichov at gmail.com> wrote:

>Hi Stuart,
>
>I haven't fully understood whether you want scaling for redundancy or
>scaling for load balancing.
>In the first case I don't see why you're interested in the number of
>consumers.
>In the second case I don't see why you're interested in replication.
>
>On Wed, Sep 7, 2011 at 7:49 PM, Stuart Munro <stuart at state.it> wrote:
>> Hi,
>> I was wondering someone could give me a few pointers about using
>>RabbitMQ in
>> cluster mode.
>> So far we have 2 RabbitMQ's in cluster mode (rb-1 and rb-2), rb-2 is in
>>RAM
>> mode.
>> How do other people setup redundancy in terms of queue consumers
>>hitting the
>> cluster?
>> So, for example could I have a load balancer sitting in front of the 2
>>rb's
>> and in a round-robin approach on the load balancer distribute load
>>between
>> them ­ this raises the question ­ how quick is the replication between
>>the
>> nodes?
>> Alternatively, do I setup a failover implementation using something like
>> HAProxy? But this raises the question how to I scale horizontally?
>> Thank you very much for your time, much appreciated.
>> Stuart
>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>
>>
>
>
>
>-- 
>Eugene Kirpichov
>Principal Engineer, Mirantis Inc. http://www.mirantis.com/
>Editor, http://fprog.ru/