[rabbitmq-discuss] RabbitMQ scalability design question

[Flavio Pompermaier]

Yes, I think you got the point..Unfortunately I'm not so expert of RabbitMQ
to fully understand how to integrate Redis or Zookeeper in RabbitMQ :(
I think my use case is an already encountered one, isn't it? is there some
reference to an already existing solution?
If not, could you please give me some more reference or hint about
implementing the solution you proposed?


On Wed, Jul 24, 2013 at 11:31 AM, Michael Klishin <mklishin at gopivotal.com>wrote:

> Flavio Pompermaier:
>
> > In my current implementation I use a single queue and a single
> multi-threaded consumer (via Spring-AMQP) that allow me to achieve this
> goal.
> > But, I'm not sure what's the best way of scaling..is there some RabbitMQ
> mechanism that I can exploit?
> > Should I change my design?
> > Maybe I should create a dedicated exchange per source and add more
> queues as the load grows..but here I still got problems when I receive this
> end message..
>
> Indeed having a "stop message" implies message ordering. But it is also
> the case with
> multiple consumers per queue.
>
> So the issue really is how to coordinate consumers, regardless of how many
> queues are
> involved.
>
> Depending on the delay between one consumer receiving the stop message and
> all consumers
> stopping you can tolerate, you can either notify other consumers via a
> separate group of
> queues (lets call them "control queues") or using a key/value store like
> Redis, or a full blown
> coordination system such as ZooKeeper.
> --
> MK
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130724/a9c3a87f/attachment.htm>