[rabbitmq-discuss] Fwd: Fwd: active/active

[John Apps]

Jason suggested I post my response to his note rather than a private reply
to him. Well, it just so happens that that is what I intended, but managed
to get the email addresses all mixed - again.
It is my belief that the world does need truly durable/persistent/reliable
messaging. It is also the case that there are many applications out there
which use it even when they had no need originally. DECmessageQ (Oracle
Message Queue these days) is an example in point.

---------- Forwarded message ----------
From: Jason J. W. Williams <jasonjwwilliams at gmail.com>
Date: Tue, May 11, 2010 at 20:46
Subject: Re: [rabbitmq-discuss] Fwd: active/active
To: John Apps <johndapps at gmail.com>


Hi John,

It'd be great if you could share that with the list. Regarding
duplication, I think that's easily addressed. If you're in a situation
where the sender is going to re-send if they don't get confirmation it
was accepted/processed on the other side, then you don't need durable
messages. You're providing the durability.

So the replay concept would only apply to durable messages and would
allow all involved to make the assumption the message will eventually
arrive and trust RabbitMQ to get it there...something that can't be
assumed at the moment with clustering.

-J

On Tue, May 11, 2010 at 1:25 AM, John Apps <johndapps at gmail.com> wrote:
> Option 1 below is neat and something that a product called Reliable
> Transaction Router (RTR) has offered for many years with considerable
> success. So, what, you might ask, has that to do with RabbitMQ? Well,
quite
> simply, only to show that the concept is a viable one and has been used in
> practice for many years.
> One of the caveats with it is when the queues are feeding into two
> databases, one a copy of the other; no, not mirrored in the classic sense,
> rather, the messages are sent to both. The thing to take care of here is
> that when messages are being replayed, i.e., one server has been down and
> comes back up, whereupon the other server then plays the missed messages
to
> it, is that the DB on the server processing the messages MUST not have
been
> changed whilst the server was down.
> What would be 'nice to have' in this scenario is a flag stating that the
> messages are being re-played. This helps to avoid the problem of possible
> duplicate messages by making one aware of the fact that a re-play is in
> progress.
> With the purchase of GemStone, perhaps the folks at RabbitMQ can use some
of
> the distributed messaging technology from that product suite?
>
> On Tue, May 11, 2010 at 00:43, Jason J. W. Williams
> <jasonjwwilliams at gmail.com> wrote:
>>
>> It was suggested to me I relay a convo I had with Alexis earlier this
>> year making the case for better active/active fail-over in RabbitMQ.
>> Particularly, when working with durable messaging.
>>
>>
>> ---------- Forwarded message ----------
>> From: Jason J. W. Williams <jasonjwwilliams at gmail.com>
>> Date: Mon, Jan 18, 2010 at 7:07 PM
>> Subject: Re: active/active
>> To: Alexis Richardson <alexis at rabbitmq.com>
>>
>>
>> Hey Alexis,
>>
>> Currently, we have a single RMQ 1.5 pair that are setup
>> active/passive. That is to say, our apps publish to node A and if it
>> is unavailable publish to node B. Nodes A and B are standalone in that
>> they are not in a cluster together. This is because the messages being
>> published are durable, and in the event node A crashes any pending
>> durable messages would not be auto-resubmitted if node A were
>> clustered with node B (node B's queue essentially hides the pendings
>> that were in the queue on node A before the crash).
>>
>> This set up works OK for us, as the load is fairly light and
>> durability is more important for these tasks than availability.
>>
>> That being said we are designing a rework of our statistics logging
>> infrastructure around Rabbit. Our stats is very high volume and
>> availability is more important than durability (in fact none of the
>> messages will be durable). So our intention is to create a new Rabbit
>> cluster with three nodes and a load-balanced IP in front (since
>> auto-redirect based on load is no longer supported....nasty memo to
>> follow... ;) ).
>>
>> Our desire due to the fact that we're moving from a colo'd environment
>> to a cloud environment, is to collapse both Rabbit "clusters"
>> into a single cluster to maximize efficiency. It would also allow our
>> existing app to benefit from the performance of a load-sharing
>> environment. For that to happen however, Rabbit would have to do one
>> of two things when clustered regarding durable messages:
>>
>> 1.) When a crashed node is restarted, replay the pending/durable
>> messages into the queue on the node now responsible for the queue
>> after the crash.
>> 2.) Replicate all durable queues to more than one node, so that
>> durable queue contents continue after a node crash.
>>
>> Frankly, option 1 is just fine with us and would solve our issue.
>>
>> Does this help? Thank you for being concerned.
>>
>> -J
>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
>
> --
> ---
> John Apps
> (49) 171 869 1813
>



-- 
---
John Apps
(49) 171 869 1813
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20100511/551af758/attachment.htm