[rabbitmq-discuss] queue_procs question

[Simon MacMullen]

I'm not quite sure what the real question is here. The queue_procs and 
binary memory slices will describe memory used by queues (with message 
bodies in 'binary' and everything else in 'queue_procs'). This is a 
fairly raw count according to what the Erlang VM tells us, so for 
example this includes memory used by unreachable objects that have not 
yet been GCed.

So "related to publishing and to moving data to the mirror on the other 
node" - well, yes, those are things queues do, so they can use memory 
doing so. But they can do other things too, and thus might use memory 
for other purposes :-)

Cheers, Simon

On 13/05/2014 17:31, Sean Allen wrote:
> I'm trying to understand what I'm seeing with memory usage in our cluster.
>
> We have a mirrored queue that we sometimes have large amounts of
> publishes come in during a relatively short period of time. (Say 2
> million messages during 5-10 minutes)
>
> While we are publishing, memory usage on both the primary and mirrored
> nodes rises greatly. Almost all the memory used is queue_procs and
> binary. Once publishing ceases, the primary node returns to an amount of
> memory usage in line with the message size (whereas during publishing
> its 5 times or so what you would expect just based on message size).
>
> The mirrored queue trails behind in terms of memory usage dropping but
> eventually levels out to similar usage some time later (couple hours later).
>
> If I turn off the consumer of the queue, the memory still stabilizes
> eventually.
>
> This has led us to believe that the queue_procs and binary memory usage
> is related to publishing and to moving data to the mirror on the other
> node. Is that a reasonable assumption?