[rabbitmq-discuss] Exchange Feature request: Drop Duplicates

[Laing, Michael]

In our scenarios, messages are ultimately delivered to a 'retail' rabbitmq
instance to be delivered to a client. The pipelines that process and
deliver messages are purposefully redundant, hence there may be multiple
replicas of each message 'racing' to the endpoint.

Usually, the replicas are resolved before getting to the retail rabbit.
When components fail, however, duplicates can leak through during a small
window of time. We eliminate those duplicates at the retail layer by
looking at each message_id. Ultimately, our client contract allows
duplicates as well in case one slips by.

It seems to me that this is a generic issue.

What would be useful in our case, and hopefully for many others, would be a
'Duplicate Message ID Window' in milliseconds, as an exchange attribute.

If non-zero, the exchange would drop any message with a duplicate
message_id that appeared within the specified window of time, possibly
routing it to the alternate exchange, if set.

In our case, a window of a few seconds, perhaps up to a minute would
suffice.

Thanks,

Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20131111/3b1b6434/attachment.htm>