[rabbitmq-discuss] Scaling Throughput with Highly Available Queues

[Matthias Radestock]

On 19/03/13 22:12, Richard Raseley wrote:
> There are four servers named rmq01, rmq02, rmq03, and rmq04. Servers
> rmq01 and rmq02 are clustered; servers rmq03 and rmq04 are clustered (so
> we now have 2x two node clusters).
>
> On both clusters we create a vhost named "default", a direct exchange
> named "default", a queue named "default", a user "default" (which has
> read, write and config access to the aforementioned objects) with a
> password of "default" and apply the ha-all mode to the "default" vhost.
>
> We place these two identically configured clusters inside of a load
> balanced pool with a "least queue depth" configuration. We then
> configure the consumers and producers associated with this app to
> connect to the VIP (which maps to the load balanced pool) to send
> messages to the "default" exchange and consume messages from the
> "default" queue.
>
> Assuming we have more producers and consumers than we do RabbitMQ nodes
> (and they get distributed properly), we've now increased our throughput
> - as we now have two master queue processes to work with on this
> "single" queue.

I thought that's where you were heading :)

You really have two queues in the above. Message order is no longer 
preserved between messages ending up in different (real) queues. And 
consumers on one (real) queue will never see the messages published to 
the other. If that is ok in your use case then fine.

You could get the same effect by just having two queues in the same 
cluster, though obviously that requires them to be named differently and 
hence is not transparent to clients.

Matthias.