[rabbitmq-discuss] 700k msgs using 6.3 GiB?

Thu May 16 05:01:24 BST 2013

Matthias,

Le 2013-05-15 à 22:44, Matthias Radestock a écrit :

> François,
> 
> On 16/05/13 01:52, François Beausoleil wrote:
>> Attached are a report and screenshot showing my RabbitMQ instance
>> using 6.3 GiB of RAM for a mere 700,000 3 KiB messages.
>> 
>> What did I do wrong?
> 
> There are a number of empty queues with fairly high memory usage. Judging from the sequence numbers - several hundred million - and uptime (~3 weeks) they are probably moving at a reasonable pace under normal conditions, though obviously not much is moving at the time of the report since the producers were blocked.

Yup, I move millions of messages per day, per queue. My normal rate is 5000 messages/s publish and acknowledge. The application is a processing pipeline: tweets on one end, analytics on the other.

> Such active queues take up substantially more memory than idle queues. Much of that will be garbage that hasn't been collected yet. That garbage in turn may reference message payloads. That's how megs of queue memory can turn into gigs of binary memory since non-tiny message payloads are not counted towards queue memory but binary memory.

Why would the garbage not have been collected yet? Any ideas? I don't want to schedule weekly server reboots to clean out garbage.

> So I don't think there is anything unusual going on here. Having said that, it would really help our future investigation if you
> 
> 1) ran the most recent rabbit version (3.1.0)
> 2) ran the most recent Erlang version (R16B)
> 3) disabled non-essential plug-ins

I'm waiting for the dust to settle on RabbitMQ 3.1.0 - 3.1.1 really (let other brave souls use the latest and greatest). I use the packaged versions of Erlang on Ubuntu 12.04. I only enable the plugins I need, namely the consistent hash exchange and management console.

Bye,
François

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4142 bytes
Desc: not available
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130516/b8941e7a/attachment.bin>