[rabbitmq-discuss] rabbitmqctl list_queues explanation needed

[Sunil Arora]

Thanks Emile,

This is indeed helpful.

I investigated more and found out the reason behind those increasing queues.

We are using celeryd (http://celeryproject.com) and we were suppose to turn off a particular settings to disable this growing queues. celery was dumping new queue for every background job that was being posted to RabbitMQ and we didn't need that.

Now I want to delete these acknowledged messages. what is the best way to do it ?  I do not want to flush to unacknowledged messages.

Thanks
Sunil



On 08-Mar-2011, at 3:26 PM, Emile Joubert wrote:

> Hi Sunil,
> 
> On 08/03/11 07:02, Sunil Arora wrote:
>> Hi folks,
>> 
>> We are trying to determine what is going under the hood in our
>> rabbitmq server because It because unresponsive last night and we
>> have to restart it. When I say unresponsive, it means it was no
>> longer accepting AMQP connections from our webservers.
> 
> That means the broker probably came under memory pressure. The logfile should confirm that. Rabbit will start accepting connections as soon as enough memory is freed. The broker will continue to deliver messages to consumers while under memory pressure.
> 
>> When we run following command:
>> 
>> /usr/sbin/rabbitmqctl list_queues name memory messages
>> messages_unacknowledged consumers
>> 
>> We see a stream of output given as follows which has around 10K
>> entries.
>> 
>> b9e5e34bcb6c46229baf5bfdfc0deb81	21688	1	0	0
>> bf52747edf2a4b1dbe919a0439bc03fa	21688	1	0	0
>> 39254dd407374dba9dd9f67fc7157613	21688	1	0	0
>> 7bc47977255e4414a975bbdfc9fbc7d3	21688	1	0	0
>> 
>> 
>> Now, I do not understand why memory is being consumed if  messages
>> are acknowledged already. Our expectation is once the message has
>> been delivered and consumed, ack'd, it should not consume any memory
>> in the system.
> 
> The broker forgets about messages as soon as they are acknowledged, so acknowledged messages can't be responsible for memory usage. The memory reported by "rabbitmqctl list_queues" is for all call stack, heap and internal structures associated with the internal Erlang queue process. In your case the total memory used by queues will amount to about 200Mb if each queue contains 1 message.
> 
> If your application doesn't need 10000 simultaneously active queues then you should drained and delete them. Otherwise you will need to add more RAM to your broker.
> 
> Rabbit has the ability to hibernate inactive queue processes, which will save some memory. Frequently checking the queue status using "rabbitmqctl list_queues" or use of the management & monitoring plugin could prevent that from happening.
> 
> 
> Regards
> 
> Emile
>