[rabbitmq-discuss] Redelivery of unacknowledged messages

[Ben Hood]

Jonatan,

On Fri, Oct 3, 2008 at 6:23 PM, Jonatan Kallus <jonatan.kallus at gmail.com> wrote:
> The system around the queue involves a few web servers using php to enqueue
> messages at almost every page load and a single java application that
> subscribes to the queue and makes one http request to another site for each
> message. (To take load off from the web servers and to make their flow
> asynchronous, in opposite to when they made the http request themselves.)
> The http request usually takes around 50 ms, but sometimes more and
> sometimes they fail or timeout after 60 s. We are using a number of
> concurrent connections for the http client, not to let a single http request
> timing out block the queue. The RabbitMQ subscriber starts threads that
> handles the http requests as long as there are available http connections,
> if there are no available http connections, the handleDelivery method waits
> (while-loop) for a free connection, thus blocking from receiving more
> messages until there is time to handle them.
>
> Currently I acknowledge messages that results in a successful http request,
> but any message that fails due to issues outside of our system (timeouts
> etc.) does not get acknowledged, since we want them to be delivered again
> later.


So if I understand you correctly, the problem that may arise is not an
issue with the message per se, but one of the availability of the
downstream system.

In which case it may be an idea to retry the downstream system,
potentially n times.

After n retries, you might decide that it's pointless to continue, so
you just end the channel (unacked messages get requeued) and tell your
friendly sysad that something is wrong with the other system.

In this case, it wouldn't help to be able to reject the message,
since, if I understand you correctly, the next message may also have a
reasonable chance of failing as well.

In this particular scenario, message rejection as an exception
management strategy is too fine grained, because the actual root cause
of the errors is very coarse grained - the downstream system has
barfed. Hence you might want to reflect this situation by ceasing to
try to send it anything at all.

After the issue with the downstream system is resolved, restart the
consumer(s) and resume processing.

Basic.Reject is a fine grained mechanism - it is sent by a particular
consumer to the broker with a "don't send me this one again"
sentiment.

It is also an academic mechanism in RabbitMQ, because it isn't implemented :-(

HTH,

Ben