[rabbitmq-discuss] How to handle extremely large queues

[Alvaro Videla]

Hi,

Keep in mind that there's small memory footprint per message, even if
the message has been paged to disk.

Regards,

Alvaro

On Sun, Feb 16, 2014 at 8:28 PM, Greg Poirier <greg.poirier at opower.com> wrote:
> They are many small messages. Each node in the cluster has 8 gigs but is
> only using maybe 2. Is memory really the problem?
>
>
> On Sunday, February 16, 2014, Michael Klishin <michael.s.klishin at gmail.com>
> wrote:
>>
>>
>> 2014-02-16 22:32 GMT+04:00 Greg Poirier <greg.poirier at opower.com>:
>>>
>>> In our current configuration, we have a 3-node cluster with 2 disc and 1
>>> ram node with HA mirroring to all nodes in the cluster. In periods of high
>>> utilization of the cluster, we are noticing frequent partitioning. We have
>>> narrowed it down to this particular use case as none of our other clusters
>>> (running on the same physical hardware with the same cluster configuration)
>>> experience this kind of partitioning.
>>>
>>> Is there some better way that we can configure RabbitMQ to handle this
>>> kind of load pattern? I understand this is perhaps not the best way to use
>>> RabbitMQ, but it is unavoidable for the time being. Any suggestions would be
>>> appreciated.
>>
>>
>> Short answer is: give it more RAM.
>>
>> Relevant blog posts:
>>
>>
>> http://blog.travis-ci.com/2013-08-08-solving-the-puzzle-of-scalable-log-processing/
>>
>> http://www.rabbitmq.com/blog/2014/01/23/preventing-unbounded-buffers-with-rabbitmq/
>> --
>> MK
>>
>> http://github.com/michaelklishin
>> http://twitter.com/michaelklishin
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>