[rabbitmq-discuss] RabbitMQ server performance issue (erl.exe)

[Yogesh Ketkar]

This is what I played around with.
Had around 5000 messages (each with 20 bytes of payload) pumped on a
queue say MainQueue.

MainQueue has a single consumer which reads messages from the queue
and based on payload does
queueDeclare and routes the messages to newly/already declared queue.
There would at most be 100 queues with name
Queue-1 to Queue-100 which would get generated based on the payload.

There is a threadpool (with 10 threads) which is under control of a
ExecutorService.
Each thread is provided a queue name to work on. Thread reads all the
messages from that queue (using basicGet) and in the end
deletes the queue. Each thread creates a new channel and closes it in
the end.
So overall at any point of time at the most 11 channels are in use (1
by MainQueue consumer and 10 by threads), which I
verified by running rabbitmqadmin.py list channels.

This worked for 5000 messages, but moment I pumped in another 10000
messages on MainQueue, things fell apart.
It started taking 5-10 seconds just for consumer to consume a message
from MainQueue and deliver it to appropriate queue.
When this starts happening erl.exe process is still taking 100% CPU no
matter what you do (killing all consumer Java programs, deleting all
the queues, with rabbitmqadmin.py), CPU usage by erl.exe is still 100%
and now unless you restart the service
RabbitMQ is almost unusable.

I even checked rabbitmqadmin.py list (channels, queues, connections),
everything showed 0. But still CPU consumption was 100%.

Environment: Windowns 7, RAM 4GB

Any clues how to debug this?

regards, Yogesh