Hi Iain,<div><br></div><div>Acks are indeed asynchronous.<br><br><div class="gmail_quote">On Thu, Jan 19, 2012 at 1:59 AM, Iain Hull <span dir="ltr"><<a href="mailto:iain.hull@workday.com">iain.hull@workday.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple"><div><p class="MsoNormal"><span lang="EN-IE">Hi,<u></u><u></u></span></p><p class="MsoNormal">
<span lang="EN-IE"><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-IE">We are using RabbitMQ 2.5.1 with the java client. Our application requires at quite reliable at most once delivery QoS Semantics. We achieve this by using persistent messages and calling basic_ack just before processing each message (using the channel that delivered the message). If the ack fails we discard the message and try the next message from the consumers internal in memory queue. If a channel or connection is closed, a management thread re-establishes the channel or connection and reconnects the consumer asynchronously, so all unacked messages get redelivered to the consumer. The original version of these messages might still be in the consumers internal queue, but since we try to ack them before processing they will be silently ignored.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-IE"><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-IE">The problem is some messages are getting processed twice. This means that a message that is acked successfully by the client is getting redelivered when the connection drops at the same time as the ack. We currently have some network conditions that we haven’t figured out yet that cause the connection to drop every 15 minutes. When this happens the message sometimes gets redelivered to consumer on a different system.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-IE"><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-IE">I have attached a network diagram to explain our topology. We have two rabbitmq boxed in a cluster, but they are not involved in any HA or automatic failover. Failover is handled by the clients on the producer and consumer systems. The queues and exchanges are duplicated across each server (with a server identifier added). We have multiple producer systems each connecting to one rabbitmq server (chosen at random). There are two ESB servers forming a cluster that consume the messages from the queues on both rabbitmq servers. The consumers on each ESB merge the queues from each rabbitmq server and present it as one queue to the application layer. The application layer does not get the message until after it is acked. <u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-IE"><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-IE">Obviously we have to fix our network issues. However we would also like to understand how we can process messages twice when a connection drops. Is the ack asynchronous? Or is there a chance of a race condition when a connection is dropped during an ack where the client considers the message acked but the server does not? Or does this look like a bug?<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-IE"><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-IE">I haven’t yet made a small application to reproduce this but should be able to if required.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-IE"><u></u> <u></u></span></p><p class="MsoNormal"><span lang="EN-IE">Regards,<u></u><u></u></span></p><p class="MsoNormal"><span lang="EN-IE">Iain.<u></u><u></u></span></p></div></div>
<br>_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br>Eugene Kirpichov<br>Principal Engineer, Mirantis Inc. <a href="http://www.mirantis.com/" target="_blank">http://www.mirantis.com/</a><br>Editor, <a href="http://fprog.ru/" target="_blank">http://fprog.ru/</a><br>
</div>