[rabbitmq-discuss] Load balancing with multiple consumers on a single queue

[Matthias Radestock]

David.Corcoran at edftrading.com wrote:
> I'm not sure how message delivery credits work but having read through a
> little it looks like it might be size oriented.

basic.qos can do count based windows too. So with a window size of one 
you can guarantee that at most one message is "in flight" or being 
processed by the consumer.

This has come up before, and basic.qos is on our todo list. As with 
basic.reject, and perhaps more so, there are quite a few challenges in 
implementing it, which is why it hasn't been done yet. One difficulty is 
that basic.qos operates on an entire channel (or, optionally, even an 
entire connection). Since a single channel can consume messages from 
multiple queues, and these queues can have consumers on other channels 
(with consumption possibly limited by basic.qos) we basically have a 
nice little optimisation & consensus problem at our hands: figuring out 
which queues should send which messages to which channels such that the 
maximum number of messages gets delivered while staying inside the 
configured qos limits. And all that in a distributed setting. And the 
limits change all the time outside the brokers control - clients ack 
messages whenever they please, the topology may change, etc, etc. Great 
fun :)


Matthias.