<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi,<br>
<br>
We have a queue which is used to send messages to an app, which then
processes them, and inserts some data into a database. Perhaps
people will have seen similar setups before :). Sometimes, the app
cannot process the message for some reason, and will use
basic.reject with requeue=false to discard it. We have a dead-letter
queue, so discarded messages end up there.<br>
<br>
Sometimes (actually, always), failures to process are not really due
to errors in the messages, but in the processing code in the
application. For example, we had a case where a zero in a particular
field in a message led to an infinity after some calculations, which
then caused database inserts to blow up. As another, we have an
ongoing epidemic of database deadlocks causing transient failures to
insert.<br>
<br>
Our response to these problem is to fix the bug in the processing
code, and then re-run the messages. We do this by prodding the
upstream message-sending system to repeat all the relevant messages.
This is manual and error-prone (it involves deleting documents from
a MongoDB server by hand, then bouncing an app server >_<),
and also rather inefficient.<br>
<br>
What i would like to do instead is move the discarded messages from
the dead-letter queue back to the main queue. In fact, we may want
to move messages from the dead-letter queue to a holding queue, and
from there back to the main queue, so that an infinite loop is never
possible. What is the best way to do this?<br>
<br>
We could write code of our own that talks to RabbitMQ to move the
messages. I would like to avoid doing this if possible, because
every line of code we write is a line of code we can get wrong.<br>
<br>
We could use the shovel plugin to do this - in some ways this seems
ideal. However, shovels are configured in RabbitMQ's configuration
files as essentially permanent features, which we don't want; we
want to be able to run a shovel-like process just once, rather than
constantly. Also, shovels seem to be geared towards moving messages
between queues in different brokers, whereas in our case, both
queues are in a single broker. Would it still be sensible to use a
shovel?<br>
<br>
If not, are there any other utilities in the RabbitMQ ecosystem
which we could use for this?<br>
<br>
Thanks in advance,<br>
tom<br>
<br>
<div class="moz-signature">-- <br>
<p>Tom Anderson | Developer | +44 20 7826 4312 | <a
href="http://timgroup.com/">timgroup.com</a></p>
<div style="color:DarkGrey; font-size: small;">
<p>STATEMENT OF CONFIDENTIALITY: The information contained in
this electronic message and any attachments to this message
are intended for the exclusive use of the addressee(s) and may
contain confidential or privileged information. If you are not
the intended recipient, please notify Tom Anderson at TIM
Group at <a class="moz-txt-link-abbreviated" href="mailto:tom.anderson@timgroup.com">tom.anderson@timgroup.com</a> and destroy all copies of
this message and any attachments.</p>
<p>TIM Group is the trading name for YouDevise Limited.
YouDevise Limited is registered in England, No. 3331176.
Registered office: 3 Copthall Avenue, London, EC2R 7BH.</p>
</div>
</div>
</body>
</html>