[rabbitmq-discuss] Problem with multiple subscribers on the same queue

Mon Aug 24 15:00:08 BST 2009

Frank,

Sorry about the late response of your mail.

Boisson, Franck wrote:
> I have an exchange, let’s say “marketdata.test.command” and a queue name
> “marketdata.test#RPC”.
>
> Queue “marketdata.test#RPC” is binded to “marketdata.test.command”
> exchange through “subscribe” key.
>
>
>
> I have severals subscribers (which, in my case, are
> data-producers)listening to “marketdata.test#RPC” queue and each time a
> message is sent on this queue, one of the producer take it as a
> subscriber request.
>
>
>
> My problem appears when a data-producer can handle only a given type of
> product. In this case, if it is the first joined to subscribe to a A
> product, and if it does not support this product, it should be able to
> requeue the message in order to join another data-producer (which could
> be able to handle the request).
>
> The simple way I was seeing was to reject the message when it arrive on
> A (basic.Reject) but I see that RabbitMQ do not support it yet.
>
>
>
> Do you have a solution for my case?

There are couple of ways around this:

- When a consumer decides that it can't process a message, it could
publish it back to an alternate exchange that has no binding to a queue
that this consumer would read from. After re-publishing the message,
this consumer would acknowledge the message so that it is gone from the
original queue. Whilst this would straight forward to implement, it is a
bit of hack and increases the amount of orchestration you'd have to do
in total;

- You could set up more judicious bindings so that the type of message
exposed in the routing key or a message header and then bind queues
based on this predicate. If you were to use a topic exchange, you could
publish with routing keys that indicate what type of message it is, e.g.
"product.A", "product.B", etc. Consumers would then be as exact as then
can when setting up bindings to their queues. A consumer that could only
process type B messages would bind with "product.B" and an omnivorous
consumer would bind with "product.*". The same can be achieved with a
headers exchange. You could put the message type into a header field and
then set up a routing rule that only binds on a particular message type.
Both approaches are similar.

That would get you out of the woods, but there may be some things you
might want to consider, such as:

- Be aware that both topic exchanges and headers exchanges incur a
linear scan of all the bindings. If this is an issue for you, you might
have to put some effort into more sophisticated direct exchange bindings;

- At any stage of the game, for whatever reason (e.g. a bug in your
setup), you will *invariably* receive a message you cannot process and
you will have to think about some kind of catch all error handling,
either by reposting to an exception queue or just logging the bad
message and discarding it;

>
> Do you have a release date for the basic.reject feature?

No, we don't. This is due to the fact that up until now, very few have
asked for this feature, so it has been deprioritized over other
features. Also, it is slightly tricky to implement because it is a
stateful command. Having said that, I myself encountered a scenario
recently on a customer project where this would have been very handy :-(

BTW would you be OK with with taking this topic onto the public
discussion forum as other people might be interested in it too?

HTH,

Ben