[rabbitmq-discuss] Millions of Queues

[Martin Sustrik]

Alexis,

>> This is a classic example of application that requires pub/sub
>> federation. I don't believe it's implemented in RabbitMQ yet, but it
>> should be doable.
> 
> Martin, you could achieve this using the relay queue in Ben's shovel,
> rather than a true federation, right?  This is because the model you
> describe is actually  a cascade, with messages propagating in one
> direction through a ramified 'branching tree' structure, with view
> consistency at every point.  If the branches recombined, then we would
> want a true federation.  But they don't afaict.
> 
> Although, I am not convinced a federated solution is needed at all.
> The cluster should suffice, no?

It depends on the use case, of course. If the message load is moderate 
you can pass each message down the tree to *all* terminal brokers. These 
would do the matching and distribute the message to appropriate queues.

If the message load is high (no idea how much traffic can 80M users 
produce) you can easily overload the network by passing every message to 
every terminal broker. In this case some kind of up-the-tree binding 
propagation (whether we'll call it federation or not) would be required.

Martin