[rabbitmq-discuss] load distribution among queue consumers and consumer_tag in non-exclusive mode

[Tim Watson]

On 7 Jul 2013, at 04:15, Ashok Anand <ashok.anand at gmail.com> wrote:

> Another question I had, regarding reading multiple messages. Consider a distributed WAN deployment situation, where consumer and rabbitmq broker are 60 ms apart (RTT of 120 ms). If consumer can get only one message from queue in one read, then it can only get 7-8 messages/second.

Are you assuming the use of a synchronous method such as basic.get? Otherwise the consumer can avoid the round trip by using streaming delivery with basic.consume instead.

> We can use  multi-threading, but if there is support of reading multiple messages in one read, that could improve the read rate even in WAN deployment and with single thread - I was wondering if there is a support for such read or if there is a plan.. 
> 
> 

The broker won't batch messages, though if you're using basic.consume with a prefetch count > 1 your clients can ack or reject multiple messages easily.

Your application can batch messages itself of course, if that offers particular advantages for your specific deployment topology. We've had several people ask for this feature in the past. One of the problems with it is that confirms and transactions would be difficult in the face of batches, and since it's very easy to do it at the application layer, it doesn't seem worth the cost in complexity. Besides, batching tends to be useful in synchronous protocols, whereas AMQP publishing and consuming is asynchronous by default. And of course you can have packets coalesced at the tcp layer by turning on nagle's algorithm, though we tend to disable that for producers.

Cheers,
Tim