This is helpful Mark!<br><br>Since our 8 queues are going to handle lot of data, we can cluster 8 RabbitMQ nodes, each to serve a single queue and its contents. But there are two concerns:<br><br>1. In the case a node (say, containing queue a) is down, how will the queue a's consumers will get their messages?<br>
<br>2. A load balancer like HA Proxy in front of these nodes will not be of any use since nodes are not similar.<br><br>I am afraid that since we are neither getting "Load balancing" nor "High availability", we will have to consider another broker.<br>
<br>Please let me know your thoughts about it.<br><br>Regards,<br>Kshitiz Garg<br><br><br><br><div class="gmail_quote">On Fri, Nov 19, 2010 at 10:34 PM, Marek Majkowski <span dir="ltr"><<a href="mailto:majek04@gmail.com">majek04@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;"><div class="im">On Fri, Nov 19, 2010 at 16:46, Kshitiz Garg <<a href="mailto:stephanion2002@gmail.com">stephanion2002@gmail.com</a>> wrote:<br>
> Thanks for the reply Mark,<br>
><br>
> In our case, we<br>
><br>
> 1. Create two exchanges A and B<br>
> 2. Create 8 queues a,b,c,d,e,f,g,h<br>
> 3. Bind A,B to these queues.<br>
> 4. Consumer is listening to a,b,c,d,e,f,g,h ......<br>
> 5. For the above operations, we are using two Rabbit MQ nodes in clustered<br>
> mode.<br>
><br>
> Please let us know the behaviour of clustered RabbitMQs ....<br>
><br>
> Also, I could not understand the statement "all the traffic to a queue gets<br>
> forwarded to one particular node." (we send the messages to an exchange and<br>
> not a queue, isn't it ?)<br>
<br>
</div>Right. I said it's complex :)<br>
<br>
All the metadata lives in a replicated Erlang database (mnesia). That includes:<br>
exchanges, bindings, credentials and so on.<br>
<br>
So, if you update anything of that - it will be immediately visible<br>
in the whole cluster. In other words: when you create an exchange<br>
A on node 1, that will automatically create exchange A on node 2.<br>
<br>
Now, with queues that's more complex. When you connect to node 1<br>
and do queue_declare(xx), you are actually creating a queue on<br>
that particular node. This information (queue xx is created on node 1)<br>
is shared with other nodes. All the nodes know that a location of<br>
queue xx is node1.<br>
<br>
Whenever you publish stuff, to whatever node, the message<br>
is pushed to a local exchange (as mentioned before, exchanges<br>
are 'copied' to every node). Next step after the exchange<br>
is the binding. The bindings are also 'copied' to all the nodes.<br>
At that point the node knows: this message should go to a queue xx.<br>
<br>
The node looks up where the queue lives and pushes a message to<br>
a particular node that holds queue xx.<br>
<br>
So, once again: exchanges and bindings are replicated, they<br>
are the same for every node in the cluster.<br>
Queue on the other hand lives on a particular node (actually<br>
on the node that 'queue_declare' was executed).<br>
<br>
Clustering != HA<br>
(the messages (queue contents) live on a single node)<br>
<br>
(I'm sure my friends from the team will clarify if I got the details wrong)<br>
<br>
Is this description any better?<br>
<br>
Cheers,<br>
<font color="#888888"> Marek<br>
</font></blockquote></div><br>