[rabbitmq-discuss] Problem with multiple subscribers on the same queue

Mon Aug 24 15:49:52 BST 2009

I'll have to check our code but I think we handle this by not ack'ing  
the message. The ineffecient part is it may be reassigned to another  
consumer who also can't handle it.

-J

Sent via iPhone

On Aug 24, 2009, at 8:00, Ben Hood <0x6e6562 at gmail.com> wrote:

> Frank,
>
> Sorry about the late response of your mail.
>
> Boisson, Franck wrote:
>> I have an exchange, let’s say “marketdata.test.command” and a  
>> queue name
>> “marketdata.test#RPC”.
>>
>> Queue “marketdata.test#RPC” is binded to “marketdata.test.comma 
>> nd”
>> exchange through “subscribe” key.
>>
>>
>>
>> I have severals subscribers (which, in my case, are
>> data-producers)listening to “marketdata.test#RPC” queue and each  
>> time a
>> message is sent on this queue, one of the producer take it as a
>> subscriber request.
>>
>>
>>
>> My problem appears when a data-producer can handle only a given  
>> type of
>> product. In this case, if it is the first joined to subscribe to a A
>> product, and if it does not support this product, it should be able  
>> to
>> requeue the message in order to join another data-producer (which  
>> could
>> be able to handle the request).
>>
>> The simple way I was seeing was to reject the message when it  
>> arrive on
>> A (basic.Reject) but I see that RabbitMQ do not support it yet.
>>
>>
>>
>> Do you have a solution for my case?
>
> There are couple of ways around this:
>
> - When a consumer decides that it can't process a message, it could
> publish it back to an alternate exchange that has no binding to a  
> queue
> that this consumer would read from. After re-publishing the message,
> this consumer would acknowledge the message so that it is gone from  
> the
> original queue. Whilst this would straight forward to implement, it  
> is a
> bit of hack and increases the amount of orchestration you'd have to do
> in total;
>
> - You could set up more judicious bindings so that the type of message
> exposed in the routing key or a message header and then bind queues
> based on this predicate. If you were to use a topic exchange, you  
> could
> publish with routing keys that indicate what type of message it is,  
> e.g.
> "product.A", "product.B", etc. Consumers would then be as exact as  
> then
> can when setting up bindings to their queues. A consumer that could  
> only
> process type B messages would bind with "product.B" and an omnivorous
> consumer would bind with "product.*". The same can be achieved with a
> headers exchange. You could put the message type into a header field  
> and
> then set up a routing rule that only binds on a particular message  
> type.
> Both approaches are similar.
>
> That would get you out of the woods, but there may be some things you
> might want to consider, such as:
>
> - Be aware that both topic exchanges and headers exchanges incur a
> linear scan of all the bindings. If this is an issue for you, you  
> might
> have to put some effort into more sophisticated direct exchange  
> bindings;
>
> - At any stage of the game, for whatever reason (e.g. a bug in your
> setup), you will *invariably* receive a message you cannot process and
> you will have to think about some kind of catch all error handling,
> either by reposting to an exception queue or just logging the bad
> message and discarding it;
>
>>
>> Do you have a release date for the basic.reject feature?
>
> No, we don't. This is due to the fact that up until now, very few have
> asked for this feature, so it has been deprioritized over other
> features. Also, it is slightly tricky to implement because it is a
> stateful command. Having said that, I myself encountered a scenario
> recently on a customer project where this would have been very  
> handy :-(
>
>
> BTW would you be OK with with taking this topic onto the public
> discussion forum as other people might be interested in it too?
>
> HTH,
>
> Ben
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss