[rabbitmq-discuss] queue_procs question

[Sean Allen]

Let me ask in a different way:

Why would a node use more memory in binary and queue_procs while publishing
to an exchange and then have that memory drop dramatically without taking
any messages off of the queue? And by dramatically I mean 5-6x less memory
within a few minutes of publishing ending.




On Wed, May 14, 2014 at 4:05 AM, Simon MacMullen <simon at rabbitmq.com> wrote:

> I'm not quite sure what the real question is here. The queue_procs and
> binary memory slices will describe memory used by queues (with message
> bodies in 'binary' and everything else in 'queue_procs'). This is a fairly
> raw count according to what the Erlang VM tells us, so for example this
> includes memory used by unreachable objects that have not yet been GCed.
>
> So "related to publishing and to moving data to the mirror on the other
> node" - well, yes, those are things queues do, so they can use memory doing
> so. But they can do other things too, and thus might use memory for other
> purposes :-)
>
> Cheers, Simon
>
>
> On 13/05/2014 17:31, Sean Allen wrote:
>
>> I'm trying to understand what I'm seeing with memory usage in our cluster.
>>
>> We have a mirrored queue that we sometimes have large amounts of
>> publishes come in during a relatively short period of time. (Say 2
>> million messages during 5-10 minutes)
>>
>> While we are publishing, memory usage on both the primary and mirrored
>> nodes rises greatly. Almost all the memory used is queue_procs and
>> binary. Once publishing ceases, the primary node returns to an amount of
>> memory usage in line with the message size (whereas during publishing
>> its 5 times or so what you would expect just based on message size).
>>
>> The mirrored queue trails behind in terms of memory usage dropping but
>> eventually levels out to similar usage some time later (couple hours
>> later).
>>
>> If I turn off the consumer of the queue, the memory still stabilizes
>> eventually.
>>
>> This has led us to believe that the queue_procs and binary memory usage
>> is related to publishing and to moving data to the mirror on the other
>> node. Is that a reasonable assumption?
>>
>
>


-- 

Ce n'est pas une signature
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140514/f1f8f32b/attachment.html>