[rabbitmq-discuss] Doubt regarding rabbit servers in clustered environment

[Kshitiz Garg]

This is helpful Mark!

Since our 8 queues are going to handle lot of data, we can cluster 8
RabbitMQ nodes, each to serve a single queue and its contents. But there are
two concerns:

1. In the case a node (say, containing queue a) is down, how will the queue
a's consumers will get their messages?

2. A load balancer like HA Proxy in front of these nodes will not be of any
use since nodes are not similar.

I am afraid that since we are neither getting "Load balancing" nor "High
availability",  we will have to consider another broker.

Please let me know your thoughts about it.

Regards,
Kshitiz Garg



On Fri, Nov 19, 2010 at 10:34 PM, Marek Majkowski <majek04 at gmail.com> wrote:

> On Fri, Nov 19, 2010 at 16:46, Kshitiz Garg <stephanion2002 at gmail.com>
> wrote:
> > Thanks for the reply Mark,
> >
> > In our case, we
> >
> > 1. Create two exchanges A and B
> > 2. Create 8 queues a,b,c,d,e,f,g,h
> > 3. Bind A,B to these queues.
> > 4. Consumer is listening to a,b,c,d,e,f,g,h ......
> > 5. For the above operations, we are using two Rabbit MQ nodes in
> clustered
> > mode.
> >
> > Please let us know the behaviour of clustered RabbitMQs ....
> >
> > Also, I could not understand the statement "all the traffic to a queue
> gets
> > forwarded to one particular node."  (we send the messages to an exchange
> and
> > not a queue, isn't it ?)
>
> Right. I said it's complex :)
>
> All the metadata lives in a replicated Erlang database (mnesia). That
> includes:
> exchanges, bindings, credentials and so on.
>
> So, if you update anything of that - it will be immediately visible
> in the whole cluster. In other words: when you create an exchange
> A on node 1, that will automatically create exchange A on node 2.
>
> Now, with queues that's more complex. When you connect to node 1
> and do queue_declare(xx), you are actually creating a queue on
> that particular node. This information (queue xx is created on node 1)
> is shared with other nodes. All the nodes know that a location of
> queue xx is node1.
>
> Whenever you publish stuff, to whatever node, the message
> is pushed to a local exchange (as mentioned before, exchanges
> are 'copied' to every node). Next step after the exchange
> is the binding. The bindings are also 'copied' to all the nodes.
> At that point the node knows: this message should go to a queue xx.
>
> The node looks up where the queue lives and pushes a message to
> a particular node that holds queue xx.
>
> So, once again: exchanges and bindings are replicated, they
> are the same for every node in the cluster.
> Queue on the other hand lives on a particular node (actually
> on the node that 'queue_declare' was executed).
>
> Clustering != HA
> (the messages (queue contents) live on a single node)
>
> (I'm sure my friends from the team will clarify if I got the details wrong)
>
> Is this description any better?
>
> Cheers,
>   Marek
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20101120/2aeba6ba/attachment-0001.htm>