[rabbitmq-discuss] Scaling Throughput with Highly Available Queues

Richard Raseley richard at raseley.com
Tue Mar 19 22:35:51 GMT 2013


Matthias,

I am glad that you were able to anticipate where I was heading - that means
I am not completely insane (at least as it relates to this). =]

Yes, I understand that they are actually two separate queues (I am treating
them as one logical one for purposes of discussion). In this case the lack
of order preservation and visibility from one cluster to the other isn't an
issue for us. We decided not to go with two queues for the transparency
reasons you mentioned.

Thanks so much for your time today.

Regards,

Richard


On Tue, Mar 19, 2013 at 3:27 PM, Matthias Radestock
<matthias at rabbitmq.com>wrote:

> On 19/03/13 22:12, Richard Raseley wrote:
>
>> There are four servers named rmq01, rmq02, rmq03, and rmq04. Servers
>> rmq01 and rmq02 are clustered; servers rmq03 and rmq04 are clustered (so
>> we now have 2x two node clusters).
>>
>> On both clusters we create a vhost named "default", a direct exchange
>> named "default", a queue named "default", a user "default" (which has
>> read, write and config access to the aforementioned objects) with a
>> password of "default" and apply the ha-all mode to the "default" vhost.
>>
>> We place these two identically configured clusters inside of a load
>> balanced pool with a "least queue depth" configuration. We then
>> configure the consumers and producers associated with this app to
>> connect to the VIP (which maps to the load balanced pool) to send
>> messages to the "default" exchange and consume messages from the
>> "default" queue.
>>
>> Assuming we have more producers and consumers than we do RabbitMQ nodes
>> (and they get distributed properly), we've now increased our throughput
>> - as we now have two master queue processes to work with on this
>> "single" queue.
>>
>
> I thought that's where you were heading :)
>
> You really have two queues in the above. Message order is no longer
> preserved between messages ending up in different (real) queues. And
> consumers on one (real) queue will never see the messages published to the
> other. If that is ok in your use case then fine.
>
> You could get the same effect by just having two queues in the same
> cluster, though obviously that requires them to be named differently and
> hence is not transparent to clients.
>
> Matthias.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130319/83571012/attachment.htm>


More information about the rabbitmq-discuss mailing list