[rabbitmq-discuss] Doubt regarding rabbit servers in clustered environment

Mon Nov 22 06:23:35 GMT 2010

Hi,

Please help us, someone ?

Regards,
Kshitiz Garg

On Sat, Nov 20, 2010 at 10:31 AM, Kshitiz Garg <stephanion2002 at gmail.com>wrote:

> This is helpful Mark!
>
> Since our 8 queues are going to handle lot of data, we can cluster 8
> RabbitMQ nodes, each to serve a single queue and its contents. But there are
> two concerns:
>
> 1. In the case a node (say, containing queue a) is down, how will the queue
> a's consumers will get their messages?
>
> 2. A load balancer like HA Proxy in front of these nodes will not be of any
> use since nodes are not similar.
>
> I am afraid that since we are neither getting "Load balancing" nor "High
> availability",  we will have to consider another broker.
>
> Please let me know your thoughts about it.
>
> Regards,
> Kshitiz Garg
>
>
>
>
> On Fri, Nov 19, 2010 at 10:34 PM, Marek Majkowski <majek04 at gmail.com>wrote:
>
>> On Fri, Nov 19, 2010 at 16:46, Kshitiz Garg <stephanion2002 at gmail.com>
>> wrote:
>> > Thanks for the reply Mark,
>> >
>> > In our case, we
>> >
>> > 1. Create two exchanges A and B
>> > 2. Create 8 queues a,b,c,d,e,f,g,h
>> > 3. Bind A,B to these queues.
>> > 4. Consumer is listening to a,b,c,d,e,f,g,h ......
>> > 5. For the above operations, we are using two Rabbit MQ nodes in
>> clustered
>> > mode.
>> >
>> > Please let us know the behaviour of clustered RabbitMQs ....
>> >
>> > Also, I could not understand the statement "all the traffic to a queue
>> gets
>> > forwarded to one particular node."  (we send the messages to an exchange
>> and
>> > not a queue, isn't it ?)
>>
>> Right. I said it's complex :)
>>
>> All the metadata lives in a replicated Erlang database (mnesia). That
>> includes:
>> exchanges, bindings, credentials and so on.
>>
>> So, if you update anything of that - it will be immediately visible
>> in the whole cluster. In other words: when you create an exchange
>> A on node 1, that will automatically create exchange A on node 2.
>>
>> Now, with queues that's more complex. When you connect to node 1
>> and do queue_declare(xx), you are actually creating a queue on
>> that particular node. This information (queue xx is created on node 1)
>> is shared with other nodes. All the nodes know that a location of
>> queue xx is node1.
>>
>> Whenever you publish stuff, to whatever node, the message
>> is pushed to a local exchange (as mentioned before, exchanges
>> are 'copied' to every node). Next step after the exchange
>> is the binding. The bindings are also 'copied' to all the nodes.
>> At that point the node knows: this message should go to a queue xx.
>>
>> The node looks up where the queue lives and pushes a message to
>> a particular node that holds queue xx.
>>
>> So, once again: exchanges and bindings are replicated, they
>> are the same for every node in the cluster.
>> Queue on the other hand lives on a particular node (actually
>> on the node that 'queue_declare' was executed).
>>
>> Clustering != HA
>> (the messages (queue contents) live on a single node)
>>
>> (I'm sure my friends from the team will clarify if I got the details
>> wrong)
>>
>> Is this description any better?
>>
>> Cheers,
>>   Marek
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20101122/ac72689a/attachment-0001.htm>