[rabbitmq-discuss] Growing Memory Use and Queues

Tue Apr 16 17:53:06 BST 2013

We're having some strange memory problems in our rabbit that looks like 
it's attributed to our queues. We're running Rabbit 3.0.1 in production, 
here's the management plugin overview: http://cl.ly/image/0z25241k320Q

As you can see, we have almost no messages queued, but the server is using 
5.7GB of active memory. Here's the memory breakdown of the same node: 
http://cl.ly/image/03381l2M3v2p

The broker claims to be using 4.8 GB to store the 4454 queues. That's 
strange because most of the queues claim to be using 14kb - 23kb, which 
AFAIK, is the in the correct range for an idle queue. An average of 20kb 
per queue * 4500 queues = 90 MB, not 4800 MB (53 times bigger). 

Here's the JSON description of a queue from the /queues call:

{"memory":14624,"idle_since":"2013-04-15 
22:45:20","policy":"","exclusive_consumer_tag":"","messages_ready":0,"messages_unacknowledged":0,"messages":0,"consumers":4,"active_consumers":4,"backing_queue_status":{"q1":0,"q2":0,"delta":["delta","undefined",0,"undefined"],"q3":0,"q4":0,"len":0,"pending_acks":0,"target_ram_count":0,"ram_msg_count":0,"ram_ack_count":0,"next_seq_id":0,"persistent_count":0,"avg_ingress_rate":0.0,"avg_egress_rate":0.0,"avg_ack_ingress_rate":0.0,"avg_ack_egress_rate":0.0},"messages_details":{"rate":0,"interval":90165823981,"last_event":1366065920939},"messages_ready_details":{"rate":0,"interval":90165823981,"last_event":1366065920939},"messages_unacknowledged_details":{"rate":0,"interval":90165823981,"last_event":1366065920939},"name":"Ingestion.1899vw88el","vhost":"/segment","durable":true,"auto_delete":false,"arguments":{},"node":"rabbit at rabbit2"}

A few things to note: we have 4 servers subscribing to every queue, and 
because of the RMQ client (node-amqp) we're using, it creates a channel per 
subscription. I realize that's not ideal and we'll be fixing the client to 
make multiple subscriptions per channel, but either way, the broker reports 
only 350 MB of the memory use coming from the connections vs. 4.8 GB for 
queues. 

The broker has also been up for 76 days without downtime, so maybe there's 
some kind of memory leak? We're seeing the memory go up over time. 

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130416/99e2cc54/attachment.htm>