[rabbitmq-discuss] producer/consumer and every n-th message
matthias at rabbitmq.com
Fri Jul 8 09:11:40 BST 2011
On 08/07/11 00:03, Torsten Curdt wrote:
> Let's say I have n producers sending messages to an exchange and m
> consumers consuming from a single queue bound to that exchange. When a
> consumer does not acknowledges a message (and dies), the message
> should go back into the queue for another consumer to pick it up.
> That should be easy to implement I assume?
As Michael said, yes, that's easy.
> I also assume for this scenario a direct exchange would do?
> How bad would be the contention n and m are fairly large?
> ...or do I just need to have enough broker?
If it's just the *numbers* of producers/consumers that are large, rather
than the message rates, then you can scale beyond a single broker by
adding nodes. That does work for coping with increased message rates too
to a degree, but the single queue will become a bottle neck at some
point and clustering won't help to overcome that.
> I would also like to be able to "tap into" the stream of messages
> flowing through the exchange. I assume I could use a fanout exchange
> instead. Have the normal queue with the m consumers and a second queue
> with a single consumer "tapping in".
Yep, that would work.
> Is there a away to tell the fanout consumer to deliver every message
> to Q1 but only every 10th message to Q2?
> How else could one do this?
Are you just concerned about the 2nd queue filling up? Then perhaps set
a message ttl on the queue
(http://www.rabbitmq.com/extensions.html#queue-ttl). Or use the
'immediate' flag on publish - though I'd discourage that.
More information about the rabbitmq-discuss