<div dir="ltr">We've used the x-consistent-hash plugin to good effect on some of this, particularly on multi-site locations where the pipe is big but latency is high. i.e. we publish to a single exchange (or multiple exchanges) as a direct queue, which binds to an x-consistent-hash exchange using whatever routing key. Then that x-consistent-hash exchange binds to N queues of name "queue.N". We then shovel each of those queues to a remote site, where they all land on the same exchange and get merged back into a single queue. <div>
<br></div><div style>If you really need your consumer to consume from multiple queues and you don't know the queue names, you can query the management UI to get a list of queues. We separate our stuff out into virtual host per applications to allow some of this functionality.</div>
<div><br></div><div style>Jason</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Jul 17, 2013 at 3:56 PM, Michael Klishin <span dir="ltr"><<a href="mailto:mklishin@gopivotal.com" target="_blank">mklishin@gopivotal.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">David van Geest:<br>
<div class="im"><br>
> We would like to scale our worker processes on the other end of the message queue(s) up and down as needed, without having the message producers know anything about it. Any ideas on how to achieve this goal?<br>
<br>
</div>You can use multiple queues and have multiple consumers for each one.<br>
<br>
Every app instance can run multiple consumers. Choosing what queues to consume<br>
from is probably application-specific. Techniques such as consistent hashing<br>
can help ensure every app instance consumes from about the same # of queues.<br>
--<br>
MK<br>
<br>
<br>_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br>Jason McIntosh<br><a href="http://mcintosh.poetshome.com/blog/">http://mcintosh.poetshome.com/blog/</a><br>573-424-7612
</div>