[rabbitmq-discuss] Load balancing with multiple consumers on a single queue

Thu Jul 31 09:20:36 BST 2008

David.Corcoran at edftrading.com wrote:
> 
> rabbitmq-discuss-bounces at lists.rabbitmq.com wrote on 30/07/2008 18:02:30:
> 
> Hey Matthias,
>> Wouldn't a simple solution be to have all consumers consuming off a
>> single queue using basic.get? I.e. consumer gets job with basic.get,
>> does work, sends ack, gets next job with basic.get, etc.
> 
> Not really because a lot of our jobs are only a few ms so the rtt would
> play too much part if we did it this way. The idea of having a local queue
> full of messages works great for us, as long as there's a way to reject
> messages if someone else's queue is empty.

Could you use basic.recover(requeue=true) in that case? You could ack 
the first n messages on your local queue that you want to deal with, 
then issue the recover to have any subsequently delievered messages 
requeued.

Of course when requeued those messages will be allocated to available 
consumers, so some may come back to the same subscriber (that would also 
be the case if you ack and sent them back to the same queue yourself).

To avoid that (without having some form of flow control or prefetch 
limit) you would I think need to temporarily cancel your subscription 
and re-subscribe when you have processed the n messages (or a portion 
thereof) from your local queue.