[rabbitmq-discuss] Clustering over a WAN

[John DeTreville]

Yes, but...

Some users want geographic distribution, and while that's pretty much impossible in Mnesia (which even so is already a distributed database!), it doesn't become a whole lot easier with a different distributed database. 

One possible approach to building a geographically distributed messaging system (possibly as an eventual successor or supplement to RabbitMQ) would be to provide eventual consistency instead of the stronger consistency currently provided, perhaps building it on top of an appropriate eventual-consistency database system. This would unfortunately require pervasive changes in the queuing API. 

In my spare time, I've been thinking a little about the design of such an API, and I'll post some more details shortly. 

Cheers,
John

On Feb 28, 2011, at 8:37, "Jon Brisbin" <jon at jbrisbin.com> wrote:

> Would any of these issues be resolved by using a distributed K/V store as the backend for both messages and metadata? I know the internals of using mnesia to store all that metadata can't easily be abstracted, but if you didn't have that particular hurdle to jump, would that make clustering any easier?
> 
> Jon Brisbin
> 
> http://jbrisbin.com
> Twitter: @j_brisbin
> 
> 
> On Feb 28, 2011, at 9:51 AM, Matthew Sackman wrote:
> 
>> On Mon, Feb 28, 2011 at 03:27:42PM +0000, Chris Hampson wrote:
>>> We're currently attempting to maintain a RabbitMQ cluster over a WAN between some of our sites, 2 in the US, one in the UK and another in India.
>> 
>> That's a bad idea.
>> 
>>> For the most part this seems to be working fine, but it seems a little fragile and we can't seem to get it to recover from failures very well.
>> 
>> Correct. Mnesia, which Rabbit uses to store all sorts of data (though
>> not messages), copes poorly with network partitions.
>> 
>>> If anyone can provide any advice to aid our situation we'd be most
>>> grateful (even if it is "don't do that you loony, separate them out
>>> and shovel messages between sites when necessary")
>> 
>> Yeah, the latter's a fair summary of my advice on this ;) We do state
>> quite prominantly that clustering is not for HA, though I admit we state
>> it on http://www.rabbitmq.com/pacemaker.html rather than
>> http://www.rabbitmq.com/clustering.html which is not really as helpful
>> as it could be.
>> 
>> The problem is that mnesia, and by extension Rabbit, goes for the
>> Consistency and Availability part of the CAP triangle, and so just does
>> not cope well with partitions ocurring.
>> http://en.wikipedia.org/wiki/CAP_theorem
>> 
>> We are working on solving some of these problems in a number of
>> different ways. If the shovel is a viable solution then I'd recommend
>> you use that, otherwise please let us know some more details of your use
>> case and we should be able to advise further.
>> 
>> Best wishes,
>> 
>> Matthew
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> 
> 
> 
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss