[rabbitmq-discuss] How to handle extremely large queues

Sun Feb 16 18:32:48 GMT 2014

We have some scenarios where queues in RabbitMQ can grow to become
extremely large... e.g. we currently have multiple queues in one cluster
with more than 1 million messages each (currently 7.5 million messages
total on this cluster).

In our current configuration, we have a 3-node cluster with 2 disc and 1
ram node with HA mirroring to all nodes in the cluster. In periods of high
utilization of the cluster, we are noticing frequent partitioning. We have
narrowed it down to this particular use case as none of our other clusters
(running on the same physical hardware with the same cluster configuration)
experience this kind of partitioning.

Is there some better way that we can configure RabbitMQ to handle this kind
of load pattern? I understand this is perhaps not the best way to use
RabbitMQ, but it is unavoidable for the time being. Any suggestions would
be appreciated.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140216/eedff680/attachment.html>