[rabbitmq-discuss] Scaling Rabbit

[Jim Irrer]

To better understand - The 10 billion messages each need to be processed
with the same
sort of processing?  The reason for having 10 direct exchanges would be that
there are 10 different kinds of
messages, is that true, or was your intent to have the message producers
have a
role in choosing a consumer?

Thanks,

- Jim

Jim Irrer     irrer at umich.edu       (734) 647-4409
University of Michigan Hospital Radiation Oncology
519 W. William St.             Ann Arbor, MI 48103


On Wed, Mar 23, 2011 at 4:29 PM, Mark Steele <msteele at beringmedia.com>wrote:

> Hi folks,
>
> I've been noodling a scale problem for a while, and wanted some input.
>
> Here's my scenario:
>
> I have a relatively small number of publishers (1000), who I expect will
> want to publish a very large number of messages (let's say 10 billion
> messages a day) to a about 10 exchanges.
>
> My naive first stab at trying to get this to scale using rabbit would be
>
>    - Create 10 direct exchanges
>    - Create 100 queues to each of the exchanges (randomly distributed onto
>    several nodes in a rabbit cluster).
>    - Bind each of the queues to a topic (random number from 1-100)
>    - Attach consumers to each of these queues (insert load balancer here)
>    - Have publishers randomly pick a number from 1-100 and use that as the
>    routing key
>
> Does this approach make sense for getting scale?
>
> Cheers
>
> Mark Steele
> Bering Media Inc.
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110323/f4fd6a9b/attachment-0001.htm>