[rabbitmq-discuss] Problem auto-deleting large number of queues?

[Michael Chiang]

Hello,
I just started playing around with rabbitmq and ran into some issue with auto-deleting large number of queues.  

I setup a rabbitMQ server (2.5.1) on a box with 4 cores, 8GB, and running Linux.  I also create 10 java consumers on a separate box, each one with one connection/channel to a rabbit MQ server.  Each consumer creates 20k queues (none-durable, auto-delete, exclusive), for a total of 200k queues on the server.  With 200k queues on the server, the memory footprint is about 2.8GB, and the server is very stable.  I can send/receive messages without any issues.  If I shutdown each consumer gracefully (ie, by calling channel.close() and then connection.close()), the rabbitMQ server takes about 300 seconds to auto-delete all the queues without much issues.

The problem comes if I don't shutdown the consumers gracefully.  If I kill all 10 consumers simultaneously without doing channel.close/connection.close, the RabbitMQ server suddenly have trouble auto-deleting those queues.  The memory effectively double to 5.6GB, and one of the cpu core is constantly staying at 100% (with 3 other cores doing nothing).  Meanwhile, the server become very unresponsive.  It takes hours before the server calms down.  When the server finally calms down, all queues are deleted and everything goes back to normal.  Does anyone knows what's going on?  Why does rabbitmq have trouble auto-deleting here?  Is there some config setting that I can tweak to fix this issue?

Thanks,
Michael