[rabbitmq-discuss] latency in starting/stopping a rabbitmq node

Fri Sep 9 03:38:14 BST 2011

Hi,

I'm a rabbitmq newbie and am trying to run some experiments to figure out if
rabbitmq would serve my use case.

I would like to create queues in the order of 100,000s. (one for each of my
customers).

I ran various tests,

I'm using the latest 2.6.0 server and 2.6.0 client, and the following tests
in durable queues mode and in non-durable queues mode.

Tests,
1) to create 1000 queues , produce, consume
2) to create 10000 queues , produce, consume
3) to create 50000 queues, produce and consume.

It works like a charm and the memory usage even with 50,000 queues seem very
reasonable. (the order of 1-1.7G)

However I realized when i wanted to shutdown the broker before starting a
new test, the stop command (rabbitmqctl stop) took a long time
to complete.

I made a small chart of how long the stop command on the broker takes to
execute after the test creates 'N' queues listed below.
Also, in the case of durable queues, i found some weird numbers for the time
taken to restart the queues after a clean/unclean(aborting broker) stop

*NON_DURABLE_QUEUES TEST*
*No of Queues Stop Time*
1000 10.7 seconds
10000 2 minutes
50000 11 minutes

*
*
*DURABLE_QUEUES TEST*
*No of Queues Start Time Stop Time*
1000 2 seconds 10 seconds
10000 24 seconds 2 minutes
10000 after crash it recovers in 20 seconds (on improper shutdown).
50000 even at 6 minutes the queues doesn't start on a improper shutdown

Query 1)
I am curious as to what causes the latency to stop the broker when issued a
rabbitmqctl stop command. It seems to be something to do with the number of
queues created as the stop time increase proportionally as the number of
queues increases.

Query 2)
In the case of durable queues, I measured the time taken to restart the
broker after stopping it (a clean and unclean stop).
I found that even after a clean/unclean stop the time to restart the broker
was just about 20 seconds on an average.
However, in the case where i created 50000 durable queues and did an unclean
stop(just aborted the broker) and tried to restart the broker it didn't
start for over to 6 minutes (when I gave up)...
It was hung in the step of "starting exchange,queue and binding recovery.."
It will be great if someone could explain why this could be caused.

It will be great if someone could answer the above queries or provide me
with some pointers about the same.

Thank you for your help,
-- 
-Praveen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110908/a8b8d9af/attachment.htm>