[rabbitmq-discuss] server crashes with very fast consumers

[alex chen]

John,



> When I run your test, many of the publishers and consumers die because they 
>can't contact the RabbitMQ broker, or can't connect to it in time. This is not 
>surprising, since a small VM with over 1,000 active processes will run very 
>slowly. The failing processes die with messages like "Cannot connect to 
>localhost:5672" and "Opening socket: Connection timed out" and so on.

Did you use the latest amqp_consumer.c that i sent in my previous email?  it 
re-connects instead of exits in case of connection failure.
http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2011-March/012099.html
However, you still need run it on a more powerful machine.  the broker crash 
happens when 1000 consumers start consuming messages simultaneously.  To to 
that, you need to have all publishers running at the same time also.  Otherwise, 
if messages in 1000 queues are published sequentially, some consumers will 
finish consuming 3000 messages before others start.  the machine i used for 
testing has 16 GB memory and 8 cpu.  i did not see any publisher got timeout.

> Although these processes were temporarily unable to connect to the RabbitMQ 
>broker, the broker itself seems to behave properly; it processes messages from 
>and to the subset of publishers and consumers that were able to get through the 
>initial connection storm. It had nothing unusual in its log.

as mentioned above, this is because you did not get all consumers to start 
simultaneously.  


> Could you confirm that this is NOT the failure mode you saw? And how did you 
>know that the RabbitMQ broker had crashed? Did the process go away? Or was it 
>impossible to contact it later on? And how did you try to contact it?

When it crashed, "ps auxww | grep beam" did not show the broker process 
running.  "rabbmitmqctl list_queues" failed to connect.
"telnet localhost 5672" failed.  

-alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110408/2bb6ea2b/attachment.htm>