[rabbitmq-discuss] Implement guaranteed delivery with ReturnListener and ConfirmListener
mklishin at gopivotal.com
Mon Jul 14 20:15:19 BST 2014
On 14 July 2014 at 22:02:52, ArnavJoshi (arnav.joshi at mathworks.com) wrote:
> > For testing purposes, we are checking for the confirms for messages
> are being written to undeclared/unbound/currently unavailable
> queues. We
> observe that although the ReturnListener method throws a 312
> error code, the ConfirmListener still calls the ‘handleAck’
> method, and not
> the ‘handleNack’ method as we would expect.
I can see that some may expect unroutable messages
to be nack-ed but RabbitMQ cannot know whether that's the case: in some cases
it's fine for messages to not be routed anywhere. So acks verify a bunch of things:
1. Your connection is alive
2. Your channel is still open (no exceptions)
3. Routed message was delivered to all queue mirrors (if applicable)
Using a nack to indicate that the message is unroutable would confuse a lot
of apps into thinking that 1) or 2) were not true.
> If this implementation is by design – could you suggest a way in
> which we
> can implement guaranteed delivery, i.e.:
> 1. Confirm that the message has not only reached the broker, but
> has been
> processed by one or more queues.
> 2. Throw a nack, or generate an exception when the message cannot
> be routed
> and that the message has been persisted to disk
> 3. Making blocking calls to our publishes in the ReturnListener
> implementation when we are trying to write to such queues?
Group messages in batches if you can. Then
publish a batch, use Channel#waitForConfirms. If it returns false, schedule
the batch for re-publishing. If a return arrives, do the same.
You'll also need to have a way to identify messages, but this is generally true
Batching will provide good throughput with easier to use but generally slower API (#waitForConfirms vs. ConfirmListener).
Because basic.return may arrive after #waitForConfirms returns, you may want to schedule
a "batch check" that runs in a few seconds and completely marks the batch as successfully
> Follow up question: If a queue process being killed is the only
> for ConfirmListener.handleNack being called – shouldn’t an
> undeclared/currently unavailable queue result in handleNack
> being called.
> Our suspicion is that the message which cannot be routed to a currently
> unavailable queue is handled like an unroutable message, which
> is confirmed
> immediately after basic.return but is also ‘ACK’ed, which seems
Can you expand on how exactly your experiment was conducted? Did you receive
an ack for a message routed to a queue whose master was unavailable? Was it the only
queue the message was routed to?
Staff Software Engineer, Pivotal/RabbitMQ
More information about the rabbitmq-discuss