[rabbitmq-discuss] RabbitMQ cluster with potentially millions of connections

[Romanas S.]

Hi guys,
 I'm trying to come up with an architecture for a RabbitMQ cluster that 
could potentially handle millions of (persistent) connections. I've set up 
a cluster of Rabbits with 3 nodes (16GB RAM, 4x cpu cores each) and a Load 
Balancer in front of them, increased all of the system limits on sockets 
and file descriptors, but can't seem to scale to more then 30k connections 
on the entire cluster. 

Each connections has it's own exchange and a temporary direct queue, RAM 
usage is about 2-3GB on each node and CPU usage peaks at about 200% (which 
on 4 cores isn't all that bad). At first all is quite well and the cluster 
manages upwards of 1k msgs/second (I haven't tried more, pretty sure it 
would handle it). However, after scaling to that many connections, the 
cluster seems to grind to a halt and eventually nodes become unresponsive. 
The management API takes upwards of 5 (FIVE!) minutes to retrieve a server 
summary report. Am I missing something here? Or is this just a bad idea 
altogether? I was expecting that if each node was able to handle 30k+ 
connections, I could have a cluster of 50 or so (with multiple balancers, 
of course!) and live a happy life  ;-)

Thanks,
Roman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130126/f278a7d7/attachment.htm>