[rabbitmq-discuss] Scaling Rabbit

Wed Mar 23 20:29:04 GMT 2011

Hi folks,

I've been noodling a scale problem for a while, and wanted some input.

Here's my scenario:

I have a relatively small number of publishers (1000), who I expect will
want to publish a very large number of messages (let's say 10 billion
messages a day) to a about 10 exchanges.

My naive first stab at trying to get this to scale using rabbit would be

   - Create 10 direct exchanges
   - Create 100 queues to each of the exchanges (randomly distributed onto
   several nodes in a rabbit cluster).
   - Bind each of the queues to a topic (random number from 1-100)
   - Attach consumers to each of these queues (insert load balancer here)
   - Have publishers randomly pick a number from 1-100 and use that as the
   routing key

Does this approach make sense for getting scale?

Cheers

Mark Steele
Bering Media Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110323/cb0d6cb9/attachment.htm>