[rabbitmq-discuss] RabbitMQ with many queues

Mon Apr 15 22:31:18 BST 2013

I know there has been discussion in the past about limits for queues in a
rabbit cluster, and there is no real hard limit except for what you impose
on yourself via file descriptors and memory (to a degree). I am curious
about those of you who have run Rabbit clusters with many thousands of
queues, how its been performance and management wise.

Our application structure is based on an 'account' and so everything that
needs to be queued needs to be segregated by account because of database
connections. We have a stream of generic change events that flow into the
cluster to a single master exchange and each interested functional area is
an exchange that is bound to the master exchange on a set of event types
via topic routing. Each functional area exchange has a queue for each
account.

Right now this is not a big problem because we have relatively few
functional areas and accounts, but as we grow bigger the number of queues
will grow very quickly. This is why Im curious about others' experiences
with complicated routing in combination with many many queues. Is there an
alternative that I should be considering? We are currently not in
production with this setup, but are very close.

Thanks guys!

- Eric Berg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130415/5cb670c2/attachment.htm>