[rabbitmq-discuss] Unique Messages in Queue
matthias at lshift.net
Fri Jul 31 19:08:03 BST 2009
Vidit Drolia wrote:
> The primary problem is that since the action being triggered by the
> message is an email, I can't revert the action. So I am trying to
> ensure that the application sending emails gets a message only once.
> Is there another approach I can take to this problem?
If you replace "process(msg)" in my last email with "send_email(msg)",
you will see that what you are asking for is impossible. The best one
can do (in any system, involving rabbit or not) is to make it *very
unlikely* that an email is sent more than once. As long as we can agree
on that, let's proceed ...
If your main concern is removing the duplicates the senders can produce,
then I suggest inserting a filtering proxy, i.e. a process that consumes
messages from one queue, de-dups them and publishes the non-dups to
This process does need to keep some state, so, as you say, if it crashes
and the state is lost then you may get some dups. The process is very
simple though, so the likelihood of it crashing should be low. Given
that we have established that there can be no 100% no-dup guarantee, is
it really worth worrying about that? If the answer is yes, then
persisting that state, or replicating it between several redundant nodes
are possible options.
More information about the rabbitmq-discuss