[rabbitmq-discuss] consume messages from multiple queues based on wildcard?

[Jason McIntosh]

We've used the x-consistent-hash plugin to good effect on some of this,
particularly on multi-site locations where the pipe is big but latency is
high.  i.e. we publish to a single exchange (or multiple exchanges) as a
direct queue, which binds to an x-consistent-hash exchange using whatever
routing key.  Then that x-consistent-hash exchange binds to N queues of
name "queue.N".  We then shovel each of those queues to a remote site,
where they all land on the same exchange and get merged back into a single
queue.

If you really need your consumer to consume from multiple queues and you
don't know the queue names, you can query the management UI to get a list
of queues.  We separate our stuff out into virtual host per applications to
allow some of this functionality.

Jason


On Wed, Jul 17, 2013 at 3:56 PM, Michael Klishin <mklishin at gopivotal.com>wrote:

> David van Geest:
>
> > We would like to scale our worker processes on the other end of the
> message queue(s) up and down as needed, without having the message
> producers know anything about it.  Any ideas on how to achieve this goal?
>
> You can use multiple queues and have multiple consumers for each one.
>
> Every app instance can run multiple consumers. Choosing what queues to
> consume
> from is probably application-specific. Techniques such as consistent
> hashing
> can help ensure every app instance consumes from about the same # of
> queues.
> --
> MK
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>


-- 
Jason McIntosh
http://mcintosh.poetshome.com/blog/
573-424-7612
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130723/8b7ebaec/attachment.htm>