[rabbitmq-discuss] RabbitMQ cluster with potentially millions of connections

[Alvaro Videla]

You can access the thread history here:
http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2013-January/025176.html


On Tue, Jan 29, 2013 at 9:12 PM, Sandeep Aggarwal <
aggarwalsandeep at optonline.net> wrote:

> To RabbitMQ gurus - Could you please post the replies here instead of just
> replying to the original person (Roman in this case). This is something of
> interest to me as well and I would like to follow this thread. Thx!
>
>
> On Saturday, January 26, 2013 6:58:36 AM UTC-5, Romanas S. wrote:
>>
>> Hi guys,
>>  I'm trying to come up with an architecture for a RabbitMQ cluster that
>> could potentially handle millions of (persistent) connections. I've set up
>> a cluster of Rabbits with 3 nodes (16GB RAM, 4x cpu cores each) and a Load
>> Balancer in front of them, increased all of the system limits on sockets
>> and file descriptors, but can't seem to scale to more then 30k connections
>> on the entire cluster.
>>
>> Each connections has it's own exchange and a temporary direct queue, RAM
>> usage is about 2-3GB on each node and CPU usage peaks at about 200% (which
>> on 4 cores isn't all that bad). At first all is quite well and the cluster
>> manages upwards of 1k msgs/second (I haven't tried more, pretty sure it
>> would handle it). However, after scaling to that many connections, the
>> cluster seems to grind to a halt and eventually nodes become unresponsive.
>> The management API takes upwards of 5 (FIVE!) minutes to retrieve a server
>> summary report. Am I missing something here? Or is this just a bad idea
>> altogether? I was expecting that if each node was able to handle 30k+
>> connections, I could have a cluster of 50 or so (with multiple balancers,
>> of course!) and live a happy life  ;-)
>>
>> Thanks,
>> Roman
>>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130129/83e832d7/attachment.htm>