[rabbitmq-discuss] RabbitMQ cluster with potentially millions of connections
aggarwalsandeep at optonline.net
Tue Jan 29 20:12:38 GMT 2013
To RabbitMQ gurus - Could you please post the replies here instead of just
replying to the original person (Roman in this case). This is something of
interest to me as well and I would like to follow this thread. Thx!
On Saturday, January 26, 2013 6:58:36 AM UTC-5, Romanas S. wrote:
> Hi guys,
> I'm trying to come up with an architecture for a RabbitMQ cluster that
> could potentially handle millions of (persistent) connections. I've set up
> a cluster of Rabbits with 3 nodes (16GB RAM, 4x cpu cores each) and a Load
> Balancer in front of them, increased all of the system limits on sockets
> and file descriptors, but can't seem to scale to more then 30k connections
> on the entire cluster.
> Each connections has it's own exchange and a temporary direct queue, RAM
> usage is about 2-3GB on each node and CPU usage peaks at about 200% (which
> on 4 cores isn't all that bad). At first all is quite well and the cluster
> manages upwards of 1k msgs/second (I haven't tried more, pretty sure it
> would handle it). However, after scaling to that many connections, the
> cluster seems to grind to a halt and eventually nodes become unresponsive.
> The management API takes upwards of 5 (FIVE!) minutes to retrieve a server
> summary report. Am I missing something here? Or is this just a bad idea
> altogether? I was expecting that if each node was able to handle 30k+
> connections, I could have a cluster of 50 or so (with multiple balancers,
> of course!) and live a happy life ;-)
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rabbitmq-discuss