If I understand this correctly, either some messages might end up in more than one queue if groups compete for the header types, or there would be no way to divide consumers such that consumer A handles group 1 messages, B handles group 2, but C handles some of group 1 and some of group 2.<br>
<br>'Course I don't always understand correctly {8'><br><br><div class="gmail_quote">On Fri, Jun 10, 2011 at 5:53 AM, Chris Chew <span dir="ltr"><<a href="mailto:chrisch@ecollege.com">chrisch@ecollege.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">> From: <a href="mailto:rabbitmq-discuss-bounces@lists.rabbitmq.com">rabbitmq-discuss-bounces@lists.rabbitmq.com</a> [<a href="mailto:rabbitmq-discuss-bounces@lists.rabbitmq.com">rabbitmq-discuss-bounces@lists.rabbitmq.com</a>] on behalf > of PADIOU Pierre-Marie (MORPHO) [<a href="mailto:pierre-marie.padiou@morpho.com">pierre-marie.padiou@morpho.com</a>]<br>
><br>
> <snip><br>
<div class="im">><br>
> There is actually far more types than that (say a thousand). For now, the only solution I see would be to create as many queues as requests types (queueA, queueB, etc), and make consumer listen to the appropriate queues.<br>
><br>
> Thing is:<br>
> - this implies that consumers create as many threads as queues there are listening from (more than a hundred in my case), which I would like to avoid.<br>
> - or I could make a single thread of a consumer randomly and sequentially dequeue messages from all queues assigned to it?<br>
><br>
> Is there a better way to handle this?<br>
<br>
</div>Here's an alternative solution that might work for you:<br>
<br>
* Producers produce messages into a headers exchange, specifying the header "Type" = "A,B,C...Z"<br>
* Consumers are grouped into "type groups" (e.g. group 1: A-M, group 2: N-Q, group 3: R-Z)<br>
* The consumer idempotently declares a queue for its group and binds the queue for each of the message types handled by the group<br>
<br>
...which should offer these features:<br>
<br>
* Messages will be routed only to consumers that know how to handle that message type<br>
* Consumers only have to consume from one queue<br>
* Consumers can be load-balanced by group without having to process duplicates<br>
* A message of a certain type could potentially be sent to multiple groups if applicable<br>
<br>
Keep in mind that the Headers exchange is among the slowest options. But, realistically, this is rarely a problem since it is rather like comparing Formula 1 cars.<br>
<br>
Cheers,<br>
<br>
Chris<br>
_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
</blockquote></div><br>