FWIW, my "solution" (pretty basic) to the same problem is to have one exchange/queue combo per type of message (or job). IOW I use the same queue for all subscribers to any exchange. <br><br>Each consumer ("agent" in me-speak) subscribes to the queues it knows how to handle, assuming I want it to...I can set up workers anywhere, but some tasks are best handled in certain places, due to network or hardware issues.<br>
<br>RabbitMQ (at least with v1.7.2...I dunno about others.) does a really neat round-robin distribution of messages when multiple consumers subscribe to the same queue. Workers "ack" each message only when they're finished doing the work (yes, that can create network timeout issues), so there's near-perfect load-balancing built in. Any time I need to pass more info than I want to try to push through RabbitMQ, I store it in S3 and pass the filename.<br>
<br>HTH, but actually I hope someone else has a better idea...<br><br>-David<br><br><div class="gmail_quote">On Fri, Jun 10, 2011 at 5:15 AM, PADIOU Pierre-Marie (MORPHO) <span dir="ltr"><<a href="mailto:pierre-marie.padiou@morpho.com">pierre-marie.padiou@morpho.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div link="blue" vlink="purple" lang="FR">
<div>
<p class="MsoNormal">Hello,</p>
<p class="MsoNormal"> </p>
<p class="MsoNormal"><span lang="EN-US">I am currently struggling with the
following use case:</span></p>
<p class="MsoNormal"><span lang="EN-US">- I am using a publish/subscribe model</span></p>
<p class="MsoNormal"><span lang="EN-US">- I’ve got many subscribers and I
only want one of them to process a publisher’s message</span></p>
<p class="MsoNormal"><span lang="EN-US">- requests can be of many types (say A to Z),
and each consumer knows how to deal with only a subset of them</span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">For example :</span></p>
<p class="MsoNormal"><span lang="EN-US">- consumer 1 can process requests A to M</span></p>
<p class="MsoNormal"><span lang="EN-US">- consumer 2 can process requests N to Z</span></p>
<p class="MsoNormal"><span lang="EN-US">- consumer 3 can process requests F to Q</span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">There is actually far more types than that
(say a thousand). For now, the only solution I see would be to create as many
queues as requests types (queueA, queueB, etc), and make consumer listen to the
appropriate queues.</span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">Thing is: </span></p>
<p class="MsoNormal"><span lang="EN-US">- this implies that consumers create as
many threads as queues there are listening from (more than a hundred in my
case), which I would like to avoid.</span></p>
<p class="MsoNormal"><span lang="EN-US">- or I could make a single thread of a consumer
randomly and sequentially dequeue messages from all queues assigned to it?</span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">Is there a better way to handle this?</span></p>
<p class="MsoNormal"><span lang="EN-US"><br>
Thanks in advance</span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US">Pierre-Marie</span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
<p class="MsoNormal"><span lang="EN-US"> </span></p>
</div>
<pre style="white-space: normal;">#<br>" Ce courriel et les documents qui lui sont joints peuvent contenir des informations confidentielles ou ayant un caractère privé. S'ils ne vous sont pas destinés, nous vous signalons qu'il est strictement interdit de les divulguer, de les reproduire ou d'en utiliser de quelque manière que ce soit le contenu. Si ce message vous a été transmis par erreur, merci d'en informer l'expéditeur et de supprimer immédiatement de votre système informatique ce courriel ainsi que tous les documents qui y sont attachés."<br>
******<br>" This e-mail and any attached documents may contain confidential or proprietary information. If you are not the intended recipient, you are notified that any dissemination, copying of this e-mail and any attachments thereto or use of their contents by any means whatsoever is strictly prohibited. If you have received this e-mail in error, please advise the sender immediately and delete this e-mail and all attached documents from your computer system."<br>
#<br></pre></div>
<br>_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
<br></blockquote></div><br>