[rabbitmq-discuss] RabbitMQ cluster with potentially millions of connections

[Simon MacMullen]

On 28/01/13 17:11, Romanas wrote:
> As Matthias suggested, I've switched to using a single exchange for all
> connections (each connection with it's own routing key), however I'm still
> struggling to go above 10-15k connections per node. I did disable the
> management plugin on all but one node (just to keep track of what's going
> on), but that didn't seem to help much. I'm pretty sure this isn't such an
> obscure use case. Is there anything I could do to try troubleshooting this
> (the log files aren't very useful!)?

I think Matthias meant to disable management altogether. Elsewhere you 
mentioned that the management overview page was very slow; this is a 
known issue with large numbers of queues / connections (which we are 
working on) so don't take that as evidence of the cluster grinding to a 
halt. And management does have some per-object overheads of its own; it 
would be good to eliminate them.

So what goes wrong when you start this many connections? (and queues - I 
assume you still have one queue per connection?) Do you run out of 
memory, does CPU reach 100%? "rabbitmqctl status" can tell you more 
about where memory is going.

Cheers, Simon

-- 
Simon MacMullen
RabbitMQ, VMware