[rabbitmq-discuss] Unique Messages in Queue

Fri Jul 31 19:08:03 BST 2009

Vidit,

Vidit Drolia wrote:
> The primary problem is that since the action being triggered by the
> message is an email, I can't revert the action. So I am trying to
> ensure that the application sending emails gets a message only once.
> Is there another approach I can take to this problem?

If you replace "process(msg)" in my last email with "send_email(msg)", 
you will see that what you are asking for is impossible. The best one 
can do (in any system, involving rabbit or not) is to make it *very 
unlikely* that an email is sent more than once. As long as we can agree 
on that, let's proceed ...

If your main concern is removing the duplicates the senders can produce, 
then I suggest inserting a filtering proxy, i.e. a process that consumes 
messages from one queue, de-dups them and publishes the non-dups to 
another exchange.

This process does need to keep some state, so, as you say, if it crashes 
and the state is lost then you may get some dups. The process is very 
simple though, so the likelihood of it crashing should be low. Given 
that we have established that there can be no 100% no-dup guarantee, is 
it really worth worrying about that? If the answer is yes, then 
persisting that state, or replicating it between several redundant nodes 
are possible options.

Regards,

Matthias.