[rabbitmq-discuss] consume messages from multiple queues based on wildcard?
mcintoshj at gmail.com
Tue Jul 23 06:17:12 BST 2013
We've used the x-consistent-hash plugin to good effect on some of this,
particularly on multi-site locations where the pipe is big but latency is
high. i.e. we publish to a single exchange (or multiple exchanges) as a
direct queue, which binds to an x-consistent-hash exchange using whatever
routing key. Then that x-consistent-hash exchange binds to N queues of
name "queue.N". We then shovel each of those queues to a remote site,
where they all land on the same exchange and get merged back into a single
If you really need your consumer to consume from multiple queues and you
don't know the queue names, you can query the management UI to get a list
of queues. We separate our stuff out into virtual host per applications to
allow some of this functionality.
On Wed, Jul 17, 2013 at 3:56 PM, Michael Klishin <mklishin at gopivotal.com>wrote:
> David van Geest:
> > We would like to scale our worker processes on the other end of the
> message queue(s) up and down as needed, without having the message
> producers know anything about it. Any ideas on how to achieve this goal?
> You can use multiple queues and have multiple consumers for each one.
> Every app instance can run multiple consumers. Choosing what queues to
> from is probably application-specific. Techniques such as consistent
> can help ensure every app instance consumes from about the same # of
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rabbitmq-discuss