[rabbitmq-discuss] Consistent Hash Exchange Type

[Matthew Sackman]

Hi,

I've just written a consistent-hash exchange type plugin. See
http://hg.rabbitmq.com/rabbitmq-consistent-hash-exchange/file/default/README.md
for the full README.

Currently, I've not done a .ez release so you'll need to compile
yourselves, but I'll get around to the .ez early next week, I hope.

Essentially, this plugin allows you to use an exchange to load-balance
messages between queues in such a way that the loss or addition of
queues does not cause substantial changes to the bucket selected by a
hash.

Normally we strongly suggest that you should have multiple consumers
hanging off a single queue and rely on the queue to do round-robin
between the consumers. However, in some cases, the performance of a
single queue is not sufficient and, as a queue is single threaded,
routing messages to multiple queues, especially on different machines
within the same cluster, can achieve much higher overall performance.
Without something like a consistent-hash exchange type, engineering such
a distribution of messages is difficult. There are other applications
too for such an exchange type.

As ever, feedback welcome. Note that this has received not a great deal
of testing so far - it's rather hot off the press - and whilst it works
for me, that's not to say there aren't bugs in it!

http://hg.rabbitmq.com/rabbitmq-consistent-hash-exchange


Matthew