[rabbitmq-discuss] RabbitMQ cluster with potentially millions of connections

[Romanas]

Hi Jason,

mc717990 wrote
> Umm, I'm reasonably sure millions of persistent connections isn't possible
> through a single entry point.  First, with TCP/IP communications an
> interface only has 65k ports (roughly).  Of those, quite a few are
> typically in use for things like SSH requests or similar.  I'm actually
> surprised you made it to 30k simultaneous connections.  Now, you should be
> able to handle that many channels when distributed across a cluster - at
> least I think so though I've not really tested that situation.  The kicker
> is even if a server can handle say 30k connections and so you have your 50
> servers, the load balancer itself would become your bottle neck as it
> couldn't keep open a million persistent connections (again, 65k limit,
> which is really a lot lower).  Most load balancers act as a proxy, not as
> a DNS lookup in concept.

Yes I'm aware of this limitation - there are ways around it, e.g. having
multiple LBs and perform DNS load balancing on them (like the Amazon ELB
does) or having multiple outbound network interfaces on the LB. However
that's not the point. My problems is that even if I spread the load myself
without going through the LB, the cluster seems to grind to a halt after a
while. 

As Matthias suggested, I've switched to using a single exchange for all
connections (each connection with it's own routing key), however I'm still
struggling to go above 10-15k connections per node. I did disable the
management plugin on all but one node (just to keep track of what's going
on), but that didn't seem to help much. I'm pretty sure this isn't such an
obscure use case. Is there anything I could do to try troubleshooting this
(the log files aren't very useful!)? 

Regards,
Roman



--
View this message in context: http://rabbitmq.1065348.n5.nabble.com/RabbitMQ-cluster-with-potentially-millions-of-connections-tp24698p24720.html
Sent from the RabbitMQ mailing list archive at Nabble.com.