[rabbitmq-discuss] How to achieve HA consumers?

Mon Sep 3 10:42:39 BST 2012

On 03/09/12 10:29, Francesco Mazzoli wrote:
> That said, whoever wants to consume from the queue needs to consume from the
> token queue first.  In the case of the OP, the main consumer will start first,
> and the second right after it.  This way, the second will be blocked until the
> first consumer dies/quits and the token message will be requeued.

In pseudo code, this is what each consumer looks like:

token_consumer = channel.basic_consume(token_queue)
wait_for_consumer_message(token_consumer)
main_consumer = channel.basic_consume(main_queue)
while(true) {
   msg = wait_for_consumer_msg(main_consumer)
   process_message(msg)
   channel.basic_ack(msg)
}

There is one snag with this: When a consumer dies, the order in which 
the token message and any pending "proper" messages are requeued is 
undefined. That means the consumer that is taking over might start 
consuming from the main queue before the unack'ed messages have been 
requeued. The messages will get requeued eventually, but they will be 
out of order.

In practice, introducing a slight delay before consuming from the main 
queue should prevent this.

Matthias.