[rabbitmq-discuss] questions about distributed queue

[Brian Mitchell]

On Mon, Aug 17, 2009 at 14:32, Alexis
Richardson<alexis.richardson at gmail.com> wrote:
> On Mon, Aug 17, 2009 at 5:22 PM, Paul Dix<paul at pauldix.net> wrote:
>> So what exactly does option 1 look like?
>>
>> It sounds like it's possible to have a queue with the same id on two
>> different nodes bound to the same exchange.
>
> Not quite.  Same routing - two queues, two ids.  Actually now that I
> think about it that won't give you exactly what you need.  More below.

Right. I figured the producers would round robin their production in
some fashion for that method. Talking to Paul on IRC made it clear
that it would be ideal to limit the amount of logic and load balancing
that the producer and/or consumer is responsible for.

Another idea I've played with is splitting the vhosts or clusters up
and optionally only replicating internally on each one to a reasonable
limit for availability control. In this case, sharding methods like
bucket selection could be used to assign producers to a single cluster
or vhost combo taking the need to cycle out. Consumers could either
connect to a single cluster or vhost or many as appropriate for the
flow characteristics expected.

The only plusses of this are the fact that you can scale complex
exchange/queue setups w/o having to generate and track a lot of unique
routing keys that need to be coordinated with the consumer side.
Instead you just have a list of hosts that can be used. I've also
thought of a few ways one could put a load balancer in front of this
quite easily on either side if there is a concern with how smart a
client must be.

Brian.