[rabbitmq-discuss] Keeping track of exchanges -- Any Advice

[MarcusR]

Okay, I feel like a real idiot. For some reason I was thinking that
direct exchanges required a 1:1 relationship to exchanges / queues.
Pretty silly of me considering I implemented the client/protocol for
our AMQP client.

That vastly simplifies the problem. Direct exchanges will work
marvellously. My question is, if there are going to be 50,000 queues,
is there any performance advantage to dividing these queues across
multiple exchanges on the same box? I do not know what kind of
algorithmns rabbit uses for exchange lookup and queue matching for the
direct exchange. So, is it better for 1 exchange to have 50,000 queues
on the same machine or 100 exchanges to have 500 queues on the same
machine?

On Jan 30, 6:42 am, Simon MacMullen <si... at rabbitmq.com> wrote:
> On 30/01/12 06:31, MarcusR wrote:
>
> > Hello Simon, thanks for the response.
>
> > I am not really sure to be honest. I am not very familiar with message
> > brokers and their performance characteristics. I suppose I could have
> > a topicexchangeand then bind queues to various topics... Such as
> > have a queue for user1.queu1, user2.queue1, user3.queue1, etc
>
> Unless you need wildcards you may not even need a topicexchange. *If*
> routing turns out to be an expensive thing for you then a directexchangecan give a notable performance boost.
>
> > My question is how expensive is routing?
>
> Depends :) If you're sending lots of small non-persistent messages fast
> then it can become a somewhat significant factor, just because RabbitMQ
> isn't doing mush *else* per message.
>
> Usually making routing simpler will make it faster.
>
> > Also, regarding the routing,
> > is it more performant to have one master topicexchangefor all the
> > users or to distribute users across multiple topic exchanges or does
> > it not matter?
>
> It doesn't matter.
>
> > I assume rabbitmq first looks up theexchangeand then goes through
> > each queue to see if there is a match for routing. If this is the
> > case, I imagine it would be more performant to create a thousand such
> > topic exchanges and divide all the users across them as opposed to one
> > topicexchangefor all users.
>
> No, it's not quite like that - topic routing is O(log(n)). But I think
> the n there is the total number of topic bindings, not perexchange.
>
> > In the end, I plan to run several rabbitmq nodes with a bucket of
> > users assigned to each rabbit node. Assigning each user to a node
> > based on a hash. Is this best practice for scaling rabbitmq for this
> > type of scenerio? Currently rabbitmq is the bottle neck in my
> > application benchmarks so this was my plan to scale it. Please correct
> > me if there is a better practice.
>
> This sort of sharding is probably the most scalable solution if it's
> practical. If it's not practical then clustering or federation would be
> alternative routes...
>
> Cheers, Simon
>
> --
> Simon MacMullen
> RabbitMQ, VMware
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss