What I've been doing is keeping a log in memcached of what tasks were in process, and then persisting those to some kind of non-volatile storage, along with their completion status. If I get a message for the same task, I check to see if another process was already handling it before starting it on one of the consumers.<br>
<br>- Alex<br><br><div class="gmail_quote">On Thu, May 17, 2012 at 5:06 AM, Matthew Sackman <span dir="ltr"><<a href="mailto:matthew@rabbitmq.com" target="_blank">matthew@rabbitmq.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<br>
<div class="im"><br>
On Thu, May 17, 2012 at 09:27:29AM +0800, 黄大鹏 wrote:<br>
> As the article <<High Availability in RabbitMQ: solving part of the<br>
</div>> puzzle<<a href="http://www.rabbitmq.com/blog/2011/10/25/high-availability-in-rabbitmq-solving-part-of-the-puzzle/" target="_blank">http://www.rabbitmq.com/blog/2011/10/25/high-availability-in-rabbitmq-solving-part-of-the-puzzle/</a>>>><br>
<div class="im">> mentioned, when master breakdown, the last ack maybe not sync to all<br>
> slave,so when a slave promote to be a new master, the msgs which actualy be<br>
> consumed maybe redelivered to the consumers.<br>
> Is there any way to identify these duplicated messages? our scenario is :<br>
> multi producers, multi queue (every consumer has a private queue ), a<br>
> direct exchange and a fanout exchange.<br>
<br>
</div>Actually, msg duplication can happen via a number of means not involving<br>
HA at all. HA just adds a few more places where such duplication can<br>
occur.<br>
<br>
There is no automatic way: a msg that's has the "isDelivered" flag set<br>
on it *may* have already been delivered to another client, which is to<br>
say that if that flag isn't set, then you shouldn't have to test for<br>
duplication. But really, you're going to have to have some sort of<br>
client side cache of hashes of recently seen messages, or some such.<br>
<br>
I'm afraid I'm not aware of any off-the-shelf solution for this, but<br>
this problem crops up regularly so there should be a fair amount of<br>
material out there on this stuff.<br>
<br>
Matthew<br>
_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
</blockquote></div><br>