<div dir="ltr"><br><br><div class="gmail_quote">On Fri, Aug 22, 2008 at 4:25 PM, Ben Hood <span dir="ltr"><<a href="mailto:0x6e6562@gmail.com">0x6e6562@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Torben,<br>
<br>
On Fri, Aug 22, 2008 at 2:31 PM, Torben Hoffmann<br>
<div class="Ih2E3d"><<a href="mailto:torben.lehoff@googlemail.com">torben.lehoff@googlemail.com</a>> wrote:<br>
<br>
</div><div class="Ih2E3d">> Not sure what a MOM system is, but I am trying to map the functionality of a<br>
> legacy system to AMQP since I think it could save a lot of effort and money.<br>
<br>
</div>MOM = Message-oriented middleware<br>
<div class="Ih2E3d"><br>
> For my basic problem I would like to have a queue for the group G that I can<br>
> send to and then all members of the queue (bar the sender) will receive that<br>
> message.<br>
<br>
</div>As indicated in my previous mail, one way this can be achieved is<br>
using a fanout exchange.<br>
<br>
[Taken from the spec] The fanout exchange type works as follows:<br>
<br>
1. A message queue binds to the exchange with no arguments.<br>
2. A publisher sends the exchange a message.<br>
3. The message is passed to the message queue unconditionally.<br>
<br>
Here is a simple example using the Ruby client for AMQP:<br>
<br>
<a href="http://github.com/tmm1/amqp/tree/master/examples/mq/clock.rb" target="_blank">http://github.com/tmm1/amqp/tree/master/examples/mq/clock.rb</a><br>
<br>
Although this is a Ruby example, the basic principle of a creating a<br>
queue, binding it to an exchange and subscribing a consumer to it hold<br>
true of any client language. I just used this as an example because it<br>
is a quick link.</blockquote><div><br>Good example - that part is now under control for me.<br> <br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
<div class="Ih2E3d"><br>
> It does not matter if I have one or more logical instances, but from a<br>
> architectural perspective I think that it would be best to have a single<br>
> logical instance so that I could utilise the distribution mechanisms of<br>
> RabbitMQ without knowing about the topology at the application level.<br>
><br>
> If I have more than one server how would the messages be distributed?<br>
<br>
</div>This is achieved using OTP distribution mechanisms with some<br>
AMQP/Rabbit-specific optimizations between nodes.<br>
<br>
In *very* loose terms, a client opens a channel on a particular node<br>
and this broker-side channel creates a communication link between<br>
itself and the underlying queue *instance*, wherever that may<br>
physically reside. If it declares a queue that doesn't already exist,<br>
the queue will be created locally, if it already exists it *may*<br>
already reside on another node, but OTP message passing keeps this<br>
transparent.</blockquote><div><br>Good - this is what I gather from the documentation. It fits my needs rather well.<br> <br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
<div class="Ih2E3d"><br>
> If I have a AMQP server at each of my machines and user A sends a message to<br>
> the group G how would that message be routed to B and C? How should the<br>
> servers be connected/configured to allow that routing?<br>
<br>
</div>G would be the name an exchange and B & C create queues and bind them<br>
to the exchange G.<br>
<br>
As indicated above this could be a simple fanout, but a direct<br>
exchange with multiple bindings would work as well.</blockquote><div><br>Okay, I did not think of this type of solution, but I think this would work just as fine. So either I get a lot of queues or a lot of exchanges depending on how I choose to represent the groups. I will let that be up to a practical test on a couple of machines. <br>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><br>
Note that you do not necessarily need an AMQP instance on machine that<br>
clients run on. Multiple remote clients can connect to a single broker<br>
instance.</blockquote><div><br>Yes, but for the sake of performance I think that the more I can distribute the work load the better.<br> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
<div class="Ih2E3d"><br>
> Note that I could have hundreds of machines but I will send the message to<br>
> the few of them that has a member of the group registered at the machine.<br>
<br>
</div>When you say you have hundreds of machines, do you mean machines that<br>
you will use to run consumer processes on?</blockquote><div><br>Indeed! The machines are already there and cannot be removed, but with AMQP I think that I can let them do more of the work in my system and remove all the machines that connect the machines where the consumer processes run.<br>
<br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><br>
The size of your broker cluster will depend (amongst other factors) on<br>
the amount of messages you need the system to be able to handle.</blockquote><div><br></div></div>A system today consisting of 100 "consumer" machines plus some inner machines handles in the neighbourhood of 3,000 message per second.<br>
So when I aim to remove the inner machines that does the most of the intelligence regarding routing of the messages I expect to get rid of the inner bottleneck and thus get a higher performance... or at least one that is closer to the limits of the "consumer" machines which I think is a better constraint.<br>
<br>Sorry if I sound a bit vague at times but I have only just read about AMQP and are now trying to get the concepts for a new AMQP based architecture in place.<br><br>Cheers,<br>Torben<br><br><br></div>