[rabbitmq-discuss] Running into some issues with high queue counts(again)
j.14159 at gmail.com
Thu Apr 5 19:02:49 BST 2012
I seem to be hitting some limits with RabbitMQ queue volumes again, here's
the rough breakdown of the situation for the purposes of my current testing:
One RabbitMQ node, 8 cores, 32GB RAM - Rabbit version 2.8.1-1, Erlang R13B
on Ubuntu 10.04
2 nodes connecting and trying to create 150k queues each on a topic exchange
10 routing keys
The 2 nodes that connect run code written for Akka 2.0 with the development
AMQP module, Scala 2.9.1, OpenJDK-6
Creating ~200k queues goes fine, only takes ~5min. Past this, I get a lot
of timing out on channel/queue creation(timeout value bumped up to 60
The use case is essentially some realtime-ish event based stuff for web
clients(will be done using Play! 2.0). Under the current plan, any given
user will create at least 2 queues: one for general events that occur
during their session and one for each category of data they're
receiving(e.g. a chat session, updates from various data sources/streams).
The 300k queues I'm attempting to create is a much smaller count than we'd
eventually be running in production.
So the question is, am I approaching this the wrong way? Should I be
treating RabbitMQ as a more general message bus and performing more complex
routing on the non-RabbitMQ web server/application nodes? Any and all
feedback would be most welcome.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rabbitmq-discuss