[rabbitmq-discuss] RabbitMQ/AMQP understanding

Fri Aug 22 21:05:03 BST 2008

On Fri, Aug 22, 2008 at 4:25 PM, Ben Hood <0x6e6562 at gmail.com> wrote:

> Torben,
>
> On Fri, Aug 22, 2008 at 2:31 PM, Torben Hoffmann
> <torben.lehoff at googlemail.com> wrote:
>
> > Not sure what a MOM system is, but I am trying to map the functionality
> of a
> > legacy system to AMQP since I think it could save a lot of effort and
> money.
>
> MOM = Message-oriented middleware
>
> > For my basic problem I would like to have a queue for the group G that I
> can
> > send to and then all members of the queue (bar the sender) will receive
> that
> > message.
>
> As indicated in my previous mail, one way this can be achieved is
> using a fanout exchange.
>
> [Taken from the spec] The fanout exchange type works as follows:
>
> 1. A message queue binds to the exchange with no arguments.
> 2. A publisher sends the exchange a message.
> 3. The message is passed to the message queue unconditionally.
>
> Here is a simple example using the Ruby client for AMQP:
>
> http://github.com/tmm1/amqp/tree/master/examples/mq/clock.rb
>
> Although this is a Ruby example, the basic principle of a creating a
> queue, binding it to an exchange and subscribing a consumer to it hold
> true of any client language. I just used this as an example because it
> is a quick link.

Good example - that part is now under control for me.

>
>
> > It does not matter if I have one or more logical instances, but from a
> > architectural perspective I think that it would be best to have a single
> > logical instance so that I could utilise the distribution mechanisms of
> > RabbitMQ without knowing about the topology at the application level.
> >
> > If I have more than one server how would the messages be distributed?
>
> This is achieved using OTP distribution mechanisms with some
> AMQP/Rabbit-specific optimizations between nodes.
>
> In *very* loose terms, a client opens a channel on a particular node
> and this broker-side channel creates a communication link between
> itself and the underlying queue *instance*, wherever that may
> physically reside. If it declares a queue that doesn't already exist,
> the queue will be created locally, if it already exists it *may*
> already reside on another node, but OTP message passing keeps this
> transparent.

Good - this is what I gather from the documentation. It fits my needs rather
well.

>
>
> > If I have a AMQP server at each of my machines and user A sends a message
> to
> > the group G how would that message be routed to  B and C? How should the
> > servers be connected/configured to allow that routing?
>
> G would be the name an exchange and B & C create queues and bind them
> to the exchange G.
>
> As indicated above this could be a simple fanout, but a direct
> exchange with multiple bindings would work as well.

Okay, I did not think of this type of solution, but I think this would work
just as fine. So either I get a lot of queues or a lot of exchanges
depending on how I choose to represent the groups. I will let that be up to
a practical test on a couple of machines.

>
> Note that you do not necessarily need an AMQP instance on machine that
> clients run on. Multiple remote clients can connect to a single broker
> instance.

Yes, but for the sake of performance I think that the more I can distribute
the work load the better.

>
>
> > Note that I could have hundreds of machines but I will send the message
> to
> > the few of them that has a member of the group registered at the machine.
>
> When you say you have hundreds of machines, do you mean machines that
> you will use to run consumer processes on?

Indeed! The machines are already there and cannot be removed, but with AMQP
I think that I can let them do more of the work in my system and remove all
the machines that connect the machines where the consumer processes run.

>
> The size of your broker cluster will depend (amongst other factors) on
> the amount of messages you need the system to be able to handle.

A system today consisting of 100 "consumer" machines plus some inner
machines handles in the neighbourhood of 3,000 message per second.
So when I aim to remove the inner machines that does the most of the
intelligence regarding routing of the messages I expect to get rid of the
inner bottleneck and thus get a higher performance... or at least one that
is closer to the limits of the "consumer" machines which I think is a better
constraint.

Sorry if I sound a bit vague at times but I have only just read about AMQP
and are now trying to get the concepts for a new AMQP based architecture in
place.

Cheers,
Torben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20080822/20ca0184/attachment.htm 