[rabbitmq-discuss] Best way to live migrate RabbitMQ consumer load from one cluster to another

[Matt Pietrek]

Thanks for the reply Arun,

Your scenario is similar to mine, and using multiple clusters is hopefully
in our future. The difference is that I'm trying to achieve it with
haproxy, rather than a dedicated load balancer like F5. My haproxy
experience is limited so I'm hoping somebody has done something like that
and can point me in a general direction.

Matt


On Fri, May 2, 2014 at 1:11 PM, Arun Rao <arunrao.seattle at gmail.com> wrote:

> I have not used HA-proxy but I will give you my experience on this
> scenario: I have 3 clusters setup, so even during peak traffic, I can
> safely remove one cluster out and upgrade or do any maintenance.
>
> Key understanding: Your clients catch Shutdownsignals and reconnect.
>
> I have F5 loadbalancer and 3 clusters behind the VIP. During maintenance,
> I take one cluster completely out of rotation and wait for the traffic to
> switch over to other clusters. After an hour or so, there are still few
> connections left behind because the active connections dont switch over so
> I do a restart of the cluster I took out of rotation. When you do the
> restart of the cluster, there should be no pending messages in the cluster
> so ideally you would want to find out the clients connected to this cluster
> and restart them so they connect to the cluster you are wanting to connect.
>
> This is purely based on my experience, please use with caution :)
>
>
> On Fri, May 2, 2014 at 9:46 AM, Matt Pietrek <mpietrek at skytap.com> wrote:
>
>> Pinging on this thread - No responses as of yet, and given other traffic
>> topics here, I'd think somebody would have some thoughts or opinions.
>>
>> Thanks,
>>
>> Matt
>>
>>
>> On Wed, Apr 30, 2014 at 10:49 AM, Matt Pietrek <mpietrek at skytap.com>wrote:
>>
>>> Apologies if this has been answered somewhere already, but my searching
>>> isn't turning up anything.
>>>
>>> I'm looking for advice on how to best move client traffic from one
>>> RabbitMQ cluster to another without service interruption. I suspect it can
>>> be done with HA-proxy, but I can't figure out the steps. I'm open to other
>>> ideas as well.
>>>
>>> Scenario: We have a 2-node cluster with all queues mirrored. Let's say
>>> it's running RabbitMQ 3.2.2. Clients connect to this cluster through a VIP
>>> maintained by keepalived on both RabbitMQ hosts.
>>>
>>> We now want to move to a different identical cluster, but running 3.3.1.
>>> I'm thinking I can bring up this new cluster "side-by-side" with the old
>>> cluster, such that they're running concurrently. All new connections would
>>> go to the new cluster, while already established connections to the 3.2.2
>>> cluster will remain alive and working, until all clients have eventually
>>> connected to the new cluster. At that point the 3.2.2 cluster can be shut
>>> down.
>>>
>>> I have a hunch I can put HA-proxy in front of both clusters, and though
>>> live configuration changes have it migrate the traffic to the new cluster
>>> while leaving existing cluster connections alone. What I don't know is the
>>> commands/configuration options/steps to implement it.
>>>
>>> Thoughts and guidance on how to best do this?
>>>
>>> Thanks,
>>>
>>> Matt
>>>
>>
>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>
>>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140505/b39ffd42/attachment.html>