[rabbitmq-discuss] Cleaning up unhandled messages

Fri Jan 21 18:03:25 GMT 2011

I've implemented the following before.

* Consumer gets a message
* Consumer throws exception
* Consumer stores md5sum of message in array of "bad messages"
* Consumer gets the message again
* Consumer throws exception
* Consumer checks bad messages array
* Bad message exists already, ack it, and publish the message to an
errors exchange.
* Errors exchange has a queue/consumer that logs to disk, sends alerts, etc.

Scott Brooks

On Fri, Jan 21, 2011 at 10:48 AM, Peter Ledbrook <peter at cacoethes.co.uk> wrote:
> Hi,
>
> What techniques do people use in dealing with messages that remain
> unhandled on a queue, for example if no consumer can process them
> successfully (perhaps because there's something wrong with the
> message)?
>
> The specific situation that this question derives from is this:
>
>    * A message causes an exception in the message handler;
>    * the message is hence not acked;
>    * the message never gets picked up again.
>
> Now, the last item is really a result of a bug in the consumer, but
> there is still a problem of messages that get left on the queue.
>
> One suggestion that springs to mind is that the consumer should catch
> the exception, ack the message and report it as bad in some way (if
> that is in fact the case). I was also thinking of having a scavenger
> consumer that deals with messages that get left like this, but that
> would probably pick up messages that should be processed by the normal
> consumer (and won't cause an exception).
>
> Anyone have any other ideas?
>
> Thanks,
>
> Peter
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>