[rabbitmq-discuss] RabbitMQ and a two-site deployment connected via WAN.

[Simon MacMullen]

Hi Daniel.

On 04/05/10 11:29, Daniel Pittman <daniel at rimspace.net> said:
> G'day.
>
> I am looking at deploying RabbitMQ to provide messaging services inside, and
> between, our two sites using both the AMQP and STOMP interfaces.
>
> As far as I can tell we have two choices for deploying this:
>
> 1. Use an Erlang cluster over the WAN, as a single instance of RabbitMQ
>
> 2. Use the 'shovel' extension to push messages between two distinct RabbitMQ
>     clusters, one at each site.
>
> The only real discussion I can find about this is from 35 weeks ago, where the
> comment was that using shovel might better handle a situation where the WAN
> was down, but that it was not production ready.

As an aside, Matthew now tells me that the shovel is "production ready", 
although at the moment it still requires compiling the broker, Erlang 
client and shovel from source.

> My impression is that, generally speaking, we are likely to be best served by
> using an Erlang cluster over the network: the WAN is fairly reliable, and will
> soon have two redundant paths, further reducing the risk of a split cluster.
>
>
> Am I correct in my understanding that using RabbitMQ in this way means:
>
> Clients will connect to their local RabbitMQ node, and when they create a
> queue it will exist only on that specific node.

Yes.

> Erlang will route messages to the correct node, either inside the site or over
> the WAN as required.

Yes.

> Erlang and/or RabbitMQ will ensure that only one copy of a message traverses
> the WAN, even if there are multiple recipients at the remote site.

Not in 1.7.2. Versions of this optimisation existed in previous versions 
of RabbitMQ, but have since been disabled since they broke some ordering 
guarantees.

There's a new, correct, version of this optimisation on branch bug19844 
in Mercurial (so, err, requiring you to compile from source again). It's 
in a fairly reasonable state and hopefully should get merged into the 
default branch soon and thus find its way into the next release. You are 
very welcome, and indeed encouraged, to test it out before that happens.

> Erlang and RabbitMQ will recover from a brief interruption of clustering, such
> as a reboot of the server at one site, without manual intervention.

I believe so.

> Further, is there anything useful that I can read about Erlang and RabbitMQ
> WAN clustering?  My research, to date, has not given me much to learn from,
> which makes me wonder what hidden complexity I might be risking...

I assume you've seen the clustering guide:

http://www.rabbitmq.com/clustering.html

(but that's pretty basic, I admit), and the Distributed Erlang manpage:

http://www.erlang.org/doc/reference_manual/distributed.html

(or at least the section on security).

Yes, we could probably do with more documentation...

Cheers, Simon
-- 
Simon MacMullen
Staff Engineer, RabbitMQ
SpringSource, a division of VMware