[rabbitmq-discuss] How to handle failover without losing messages if one of RabbitMQ node crashes in cluster

[Matthias Radestock]

Jaymin,

apologies for the late reply...

On 08/09/11 12:06, Jaymin Shah wrote:
> I have RabbitMQ cluster (2 instances of v2.6) on EC2. With this setup I
> am able to process around 51K/s messages.
>
> I have one exchange and 3 queues bound to each instance totaling to 6
> consumers and 3 producer each sending messages to each
> instance totaling to 6 producers. Note that queue is not durable.
>
> I am expected to validate if one of instance in cluster goes down then
> would I be losing messages? If yes then what could be done to avoid
> that. We tried durable queues but it does not give desired performance
> due to very high IO and that too on virtual environment like EC2.

In order to avoid message loss, some redundancy is required. That cannot 
come for free, so there is necessarily some impact on performance. I 
suggest you give the new active-active HA a try: 
http://www.rabbitmq.com/ha.html. This is reasonably well tuned and 
scales well, e.g. you can set up a cluster of N nodes, with each queue 
being mirrored to just two of the nodes.


Regards,

Matthias.