[rabbitmq-discuss] Keeping track of exchanges -- Any Advice

Sat Feb 4 01:02:09 GMT 2012

Hi, Marcus:

The actual work of doing the routing through an exchange is actually
done by the Rabbit Erlang process associated with the *channel* you're
publishing on... in other words there's no "exchange process."

The work that your channel process does varies depending on the type of
exchange.  In the case of topic exchanges there's a nice and well tuned
trie-based matching algorithm that Matthew has blogged about on the
Rabbit website in the past.  Fan out and direct exchanges work about as
you'd expect.

Unlike exchanges, queues *do* have an Erlang process associated with them,
that receives messages that have been routed through the appropriate
exchange, by the channel process, which passes the AMQP message to the
queue process using Erlang messaging.

As always, it pays to perform an experiment that lines up with the scenario
you're expecting to face in your production system.  But the above
observations should get you started on how to set your expectations...

Best regards,
Jerry

----- Original Message -----
From: "MarcusR" <marcus_rubeus at hotmail.com>
To: rabbitmq-discuss at lists.rabbitmq.com
Sent: Friday, February 3, 2012 4:49:34 PM
Subject: Re: [rabbitmq-discuss] Keeping track of exchanges -- Any Advice

Okay, I feel like a real idiot. For some reason I was thinking that
direct exchanges required a 1:1 relationship to exchanges / queues.
Pretty silly of me considering I implemented the client/protocol for
our AMQP client.

That vastly simplifies the problem. Direct exchanges will work
marvellously. My question is, if there are going to be 50,000 queues,
is there any performance advantage to dividing these queues across
multiple exchanges on the same box? I do not know what kind of
algorithmns rabbit uses for exchange lookup and queue matching for the
direct exchange. So, is it better for 1 exchange to have 50,000 queues
on the same machine or 100 exchanges to have 500 queues on the same
machine?

On Jan 30, 6:42 am, Simon MacMullen <si... at rabbitmq.com> wrote:
> On 30/01/12 06:31, MarcusR wrote:
>
> > Hello Simon, thanks for the response.
>
> > I am not really sure to be honest. I am not very familiar with message
> > brokers and their performance characteristics. I suppose I could have
> > a topicexchangeand then bind queues to various topics... Such as
> > have a queue for user1.queu1, user2.queue1, user3.queue1, etc
>
> Unless you need wildcards you may not even need a topicexchange. *If*
> routing turns out to be an expensive thing for you then a directexchangecan give a notable performance boost.
>
> > My question is how expensive is routing?
>
> Depends :) If you're sending lots of small non-persistent messages fast
> then it can become a somewhat significant factor, just because RabbitMQ
> isn't doing mush *else* per message.
>
> Usually making routing simpler will make it faster.
>
> > Also, regarding the routing,
> > is it more performant to have one master topicexchangefor all the
> > users or to distribute users across multiple topic exchanges or does
> > it not matter?
>
> It doesn't matter.
>
> > I assume rabbitmq first looks up theexchangeand then goes through
> > each queue to see if there is a match for routing. If this is the
> > case, I imagine it would be more performant to create a thousand such
> > topic exchanges and divide all the users across them as opposed to one
> > topicexchangefor all users.
>
> No, it's not quite like that - topic routing is O(log(n)). But I think
> the n there is the total number of topic bindings, not perexchange.
>
> > In the end, I plan to run several rabbitmq nodes with a bucket of
> > users assigned to each rabbit node. Assigning each user to a node
> > based on a hash. Is this best practice for scaling rabbitmq for this
> > type of scenerio? Currently rabbitmq is the bottle neck in my
> > application benchmarks so this was my plan to scale it. Please correct
> > me if there is a better practice.
>
> This sort of sharding is probably the most scalable solution if it's
> practical. If it's not practical then clustering or federation would be
> alternative routes...
>
> Cheers, Simon
>
> --
> Simon MacMullen
> RabbitMQ, VMware
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss at lists.rabbitmq.com
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss