[rabbitmq-discuss] Best way to live migrate RabbitMQ consumer load from one cluster to another

[Arun Rao]

I have not used HA-proxy but I will give you my experience on this
scenario: I have 3 clusters setup, so even during peak traffic, I can
safely remove one cluster out and upgrade or do any maintenance.

Key understanding: Your clients catch Shutdownsignals and reconnect.

I have F5 loadbalancer and 3 clusters behind the VIP. During maintenance, I
take one cluster completely out of rotation and wait for the traffic to
switch over to other clusters. After an hour or so, there are still few
connections left behind because the active connections dont switch over so
I do a restart of the cluster I took out of rotation. When you do the
restart of the cluster, there should be no pending messages in the cluster
so ideally you would want to find out the clients connected to this cluster
and restart them so they connect to the cluster you are wanting to connect.

This is purely based on my experience, please use with caution :)


On Fri, May 2, 2014 at 9:46 AM, Matt Pietrek <mpietrek at skytap.com> wrote:

> Pinging on this thread - No responses as of yet, and given other traffic
> topics here, I'd think somebody would have some thoughts or opinions.
>
> Thanks,
>
> Matt
>
>
> On Wed, Apr 30, 2014 at 10:49 AM, Matt Pietrek <mpietrek at skytap.com>wrote:
>
>> Apologies if this has been answered somewhere already, but my searching
>> isn't turning up anything.
>>
>> I'm looking for advice on how to best move client traffic from one
>> RabbitMQ cluster to another without service interruption. I suspect it can
>> be done with HA-proxy, but I can't figure out the steps. I'm open to other
>> ideas as well.
>>
>> Scenario: We have a 2-node cluster with all queues mirrored. Let's say
>> it's running RabbitMQ 3.2.2. Clients connect to this cluster through a VIP
>> maintained by keepalived on both RabbitMQ hosts.
>>
>> We now want to move to a different identical cluster, but running 3.3.1.
>> I'm thinking I can bring up this new cluster "side-by-side" with the old
>> cluster, such that they're running concurrently. All new connections would
>> go to the new cluster, while already established connections to the 3.2.2
>> cluster will remain alive and working, until all clients have eventually
>> connected to the new cluster. At that point the 3.2.2 cluster can be shut
>> down.
>>
>> I have a hunch I can put HA-proxy in front of both clusters, and though
>> live configuration changes have it migrate the traffic to the new cluster
>> while leaving existing cluster connections alone. What I don't know is the
>> commands/configuration options/steps to implement it.
>>
>> Thoughts and guidance on how to best do this?
>>
>> Thanks,
>>
>> Matt
>>
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140502/b15cd6f3/attachment.html>