[rabbitmq-discuss] Limitations with RabbitMQ Java Client

[Michael Klishin]

On 22 Oct 2013, at 20:02, "k.madnani84" <k.madnani84 at gmail.com> wrote:

> I am using the RabbitMQ Java client 3.1.5 to test the performance using
> MultiCastMain.java.But my RabbitMQ crashes when executing the following
> scenario:
> 

…

> My Memory High Watermark is 4.7 GB.
> 
> Is there any benchmark on how many consumers and producers maximum i can use
> with the above watermark.
> 
> i am facing this error at client end:
> 
> DefaultExceptionHandler: Consumer
> com.rabbitmq.client.QueueingConsumer at e02ddd
> (amq.ctag-Hk684-l36FBBCV79J4p7aw) method handleDelivery for channel
> AMQChannel(amqp://rbtadmin@135.213.26.33:5672/,1) threw an exception for
> channel AMQChannel(amqp://rbtadmin@135.213.26.33:5672/,1):
> java.lang.OutOfMemoryError: unable to create new native thread
> 	at java.lang.Thread.start0(Native Method)
> 	at java.lang.Thread.start(Thread.java:640)
> 	at
> java.util.concurrent.ThreadPoolExecutor.addIfUnderCorePoolSize(ThreadPoolExecutor.java:703)

No, RabbitMQ does not crash.

MulticastMain fails with java.lang.OutOfMemoryError
because it cannot create a new thread. Your OS is running out of kernel threads.

Every RabbitMQ Java client connection uses 6 threads (5 consumer work pool threads + I/O thread).
Multiplied by 2000 in your example, that means you need 12,000 kernel threads.

Use fewer publishers and consumers. Having thousands on a single machine won't give
you any better throughput than a couple of hundreds, in fact, it may yield worse results.

MK