[rabbitmq-discuss] 50k bindings to two queues?

[Ben Hood]

David,

On Thu, Sep 3, 2009 at 12:23 AM, David King<dking at ketralnis.com> wrote:
> Is it practical to have an exchange with two queues and 50k bindings
> to those two queues?

Bindings to direct exchanges are indexed such that the runtime should
be roughly logarithmic. Topic and header exchanges incur a linear
scan. Fanout exchanges can be quick because they route
unconditionally.

> That is, I have all of these categories, and I load-balance their
> processing over two nodes. So I want message.sports and message.bacon
> to go to the same node, and message.somethingelse to go to the other
> node. But I have 50k of those categories, and growing. While I may add
> more nodes, I'd do so *much* more slowly than new categories would be
> created

Load balancing in Rabbit pertains to queues and not exchanges, since
exchanges do not run as a separate process. I don't know whether this
affects your layout.

> Is the binding-matching that efficient? Or am I better off having some
> intermediate process that does its own balancing?

ATM the best you can do is to consume and republish with an embedded
consumer if you can implement a match algorithm that is tuned to your
actual scenario. Going forwards, it would be nice to be able to plug
this in (under the assumption that you have a good matcher).

BTW have you done any load testing with your current setup yet?

HTH,

Ben