[rabbitmq-discuss] How to handle extremely large queues

Sun Feb 16 19:28:27 GMT 2014

They are many small messages. Each node in the cluster has 8 gigs but is
only using maybe 2. Is memory really the problem?

On Sunday, February 16, 2014, Michael Klishin <michael.s.klishin at gmail.com>
wrote:

>
> 2014-02-16 22:32 GMT+04:00 Greg Poirier <greg.poirier at opower.com<javascript:_e(%7B%7D,'cvml','greg.poirier at opower.com');>
> >:
>
>> In our current configuration, we have a 3-node cluster with 2 disc and 1
>> ram node with HA mirroring to all nodes in the cluster. In periods of high
>> utilization of the cluster, we are noticing frequent partitioning. We have
>> narrowed it down to this particular use case as none of our other clusters
>> (running on the same physical hardware with the same cluster configuration)
>> experience this kind of partitioning.
>>
>> Is there some better way that we can configure RabbitMQ to handle this
>> kind of load pattern? I understand this is perhaps not the best way to use
>> RabbitMQ, but it is unavoidable for the time being. Any suggestions would
>> be appreciated.
>>
>
> Short answer is: give it more RAM.
>
> Relevant blog posts:
>
>
> http://blog.travis-ci.com/2013-08-08-solving-the-puzzle-of-scalable-log-processing/
>
> http://www.rabbitmq.com/blog/2014/01/23/preventing-unbounded-buffers-with-rabbitmq/
> --
> MK
>
> http://github.com/michaelklishin
> http://twitter.com/michaelklishin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140216/c8727fa7/attachment.html>