[rabbitmq-discuss] RabbitMQ big queue load balance issue

[Michael Klishin]

On 4 July 2014 at 12:35:54, DeepNightTwo (deepnighttwo at gmail.com) wrote:
> > My question is, is there a better way to handle the big queue requirement?  
> If not, is there an official plugin/exchange for "random distribute"?  
> The
> plugin is now working with latest rabbitmq 3.3.x. So I have to  
> use 3.1.x
> now.

See consistent-hash exchange. It's less efficient than the most commonly used
built-in types (except the headers one) but should be good enough for a lot of cases.

Also, take a look at http://blog.travis-ci.com/2013-08-08-solving-the-puzzle-of-scalable-log-processing/
— if you don't need message ordering, you don't really need a single queue and can get away
with N queues and N consumers having its own channel with the same basic.qos setting. 
--  
MK  

Staff Software Engineer, Pivotal/RabbitMQ