[rabbitmq-discuss] Driver connecting to multiple instances in a cluster

[Matthias Radestock]

Srdan,

On 10/10/12 19:40, Srdan Kvrgic wrote:
> Let me put it to you this way. Assume we have a cluster (MQ1, MQ2)
> with two queues (fragments1, fragments2) under same exchange. The
> client connects to MQ1.
>
> If the client wants to send a message to fragments 1, the message
> will be sent by client and received and routed by MQ1.
>
> On the other hand, if the client wants to post a message on
> fragments2, MQ1 will have to forward the message to MQ2 which means
> the message will be sent from client, received by MQ1, sent by MQ1
> and received and routed by MQ2.

...and if the consumer(s) for MQ2 are connected to MQ1 then there is a 
second hop to traverse.

And yes, this is fairly expensive.

> This is a neat solution but when running at high speed it becomes a
> bottleneck. So, we created autobahn which is a jruby transport layer
> wrapper that automatically connects to all MQ instances and sends the
> message to the server that hosts the queue. That gave us a massive
> boost in throughput. Yey us.
>
> So why am I on the figurative soap-box? The problem's solved, right?
> Why am I not on a beach under an umbrella wearing a stupid-ass hat
> and sipping on a Bahama-mama? Well.. The throughput improvement was
> so substantial that I couldn't help feeling this should be the
> driver's default behavior.

Unfortunately that is impossible in the general case:

1) In AMQP a single published message can get routed to multiple queues. 
Indeed that is typical for pub-sub type applications. If those queues 
reside on different nodes then there is no 'right' node to publish to.

2) AMQP is a connection-oriented protocol, and the notion of sessions, 
manifested in AMQP's concept of channels, is important. For example, 
ordering guarantees, transactions/confirms, qos prefetching and error 
handling are all scoped to channels. So performing operations across 
multiple connections has different behaviour than when doing so across a 
single connection. This may not matter to some applications, but in 
general it does.

3) There is no way for a client to find out which node a queue resides 
on, other than going via the management interface, which is not 
something you'd want to happen in a plain AMQP client.

4) A mirrored queue's master node of queue can change during its 
lifetime due to failover events.


Regards,

Matthias.