[rabbitmq-discuss] RabbitMQ scalability design question

Wed Jul 24 10:31:50 BST 2013

Flavio Pompermaier:

> In my current implementation I use a single queue and a single multi-threaded consumer (via Spring-AMQP) that allow me to achieve this goal.
> But, I'm not sure what's the best way of scaling..is there some RabbitMQ mechanism that I can exploit?
> Should I change my design?
> Maybe I should create a dedicated exchange per source and add more queues as the load grows..but here I still got problems when I receive this end message..

Indeed having a "stop message" implies message ordering. But it is also the case with
multiple consumers per queue.

So the issue really is how to coordinate consumers, regardless of how many queues are
involved.

Depending on the delay between one consumer receiving the stop message and all consumers
stopping you can tolerate, you can either notify other consumers via a separate group of
queues (lets call them "control queues") or using a key/value store like Redis, or a full blown
coordination system such as ZooKeeper.
--
MK

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 495 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130724/33d9cfc6/attachment.pgp>