[rabbitmq-discuss] Acked Message getting redelivered

[Eugene Kirpichov]

Hi Iain,

Acks are indeed asynchronous.

On Thu, Jan 19, 2012 at 1:59 AM, Iain Hull <iain.hull at workday.com> wrote:

> Hi,****
>
> ** **
>
> We are using RabbitMQ 2.5.1 with the java client.  Our application
> requires at quite reliable at most once delivery QoS Semantics. We achieve
> this by using persistent messages and calling basic_ack just before
> processing each message (using the channel that delivered the message).  If
> the ack fails we discard the message and try the next message from the
> consumers internal in memory queue.  If a channel or connection is closed,
> a management thread re-establishes the channel or connection and reconnects
> the consumer asynchronously, so all unacked messages get redelivered to the
> consumer.  The original version of these messages might still be in the
> consumers internal queue, but since we try to ack them before processing
> they will be silently ignored.****
>
> ** **
>
> The problem is some messages are getting processed twice.  This means that
> a message that is acked successfully by the client is getting redelivered
> when the connection drops at the same time as the ack.  We currently have
> some network conditions that we haven’t figured out yet that cause the
> connection to drop every 15 minutes.  When this happens the message
> sometimes gets redelivered to consumer on a different system.****
>
> ** **
>
> I have attached a network diagram to explain our topology.  We have two
> rabbitmq boxed in a cluster, but they are not involved in any HA or
> automatic failover.  Failover is handled by the clients on the producer and
> consumer systems.  The queues and exchanges are duplicated across each
> server (with a server identifier added).  We have multiple producer systems
> each connecting to one rabbitmq server (chosen at random).  There are two
> ESB servers forming a cluster that consume the messages from the queues on
> both rabbitmq servers.  The consumers on each ESB merge the queues from
> each rabbitmq server and present it as one queue to the application layer.
> The application layer does not get the message until after it is acked. **
> **
>
> ** **
>
> Obviously we have to fix our network issues.  However we would also like
> to understand how we can process messages twice when a connection drops.
>  Is the ack asynchronous? Or is there a chance of a race condition when a
> connection is dropped during an ack where the client considers the message
> acked but the server does not?  Or does this look like a bug?****
>
> ** **
>
> I haven’t yet made a small application to reproduce this but should be
> able to if required.****
>
> ** **
>
> Regards,****
>
> Iain.****
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>


-- 
Eugene Kirpichov
Principal Engineer, Mirantis Inc. http://www.mirantis.com/
Editor, http://fprog.ru/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120119/26cf4255/attachment.htm>