[rabbitmq-discuss] publish/suscribe model with typed consumers

Fri Jun 10 11:53:52 BST 2011

> From: rabbitmq-discuss-bounces at lists.rabbitmq.com [rabbitmq-discuss-bounces at lists.rabbitmq.com] on behalf > of PADIOU Pierre-Marie (MORPHO) [pierre-marie.padiou at morpho.com]
>
> <snip>
>  
> There is actually far more types than that (say a thousand). For now, the only solution I see would be to create as many queues as requests types (queueA, queueB, etc), and make consumer listen to the appropriate queues.
>  
> Thing is: 
> - this implies that consumers create as many threads as queues there are listening from (more than a hundred in my case), which I would like to avoid.
> - or I could make a single thread of a consumer randomly and sequentially dequeue messages from all queues assigned to it?
>  
> Is there a better way to handle this?

Here's an alternative solution that might work for you:

* Producers produce messages into a headers exchange, specifying the header "Type" = "A,B,C...Z"
* Consumers are grouped into "type groups" (e.g. group 1: A-M, group 2: N-Q, group 3: R-Z)
* The consumer idempotently declares a queue for its group and binds the queue for each of the message types handled by the group

...which should offer these features:

* Messages will be routed only to consumers that know how to handle that message type
* Consumers only have to consume from one queue
* Consumers can be load-balanced by group without having to process duplicates
* A message of a certain type could potentially be sent to multiple groups if applicable

Keep in mind that the Headers exchange is among the slowest options.  But, realistically, this is rarely a problem since it is rather like comparing Formula 1 cars.

Cheers,

Chris