[rabbitmq-discuss] Load balancing with multiple consumers on a single queue
gsim at redhat.com
Thu Jul 31 09:20:36 BST 2008
David.Corcoran at edftrading.com wrote:
> rabbitmq-discuss-bounces at lists.rabbitmq.com wrote on 30/07/2008 18:02:30:
> Hey Matthias,
>> Wouldn't a simple solution be to have all consumers consuming off a
>> single queue using basic.get? I.e. consumer gets job with basic.get,
>> does work, sends ack, gets next job with basic.get, etc.
> Not really because a lot of our jobs are only a few ms so the rtt would
> play too much part if we did it this way. The idea of having a local queue
> full of messages works great for us, as long as there's a way to reject
> messages if someone else's queue is empty.
Could you use basic.recover(requeue=true) in that case? You could ack
the first n messages on your local queue that you want to deal with,
then issue the recover to have any subsequently delievered messages
Of course when requeued those messages will be allocated to available
consumers, so some may come back to the same subscriber (that would also
be the case if you ack and sent them back to the same queue yourself).
To avoid that (without having some form of flow control or prefetch
limit) you would I think need to temporarily cancel your subscription
and re-subscribe when you have processed the n messages (or a portion
thereof) from your local queue.
More information about the rabbitmq-discuss