<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">(26/07/13 18:03), Tom Anderson wrote:<br>
</div>
<blockquote cite="mid:51F2ABDA.9020709@timgroup.com" type="cite">
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
<div class="moz-cite-prefix">On 26/07/13 17:07, Ceri Storey wrote:<br>
</div>
<blockquote cite="mid:51F29EA7.20807@lshift.net" type="cite">
<meta http-equiv="Context-Type" content="text/html;
charset=ISO-8859-1">
<div class="moz-cite-prefix">(26/07/13 16:49), Tom Anderson
wrote:<br>
</div>
<blockquote cite="mid:51F29A82.8040302@timgroup.com" type="cite">
Implemented exactly as described there, it yields an infinite
loop for unprocessable messages. You might therefore also want
to keep a count of the number of processing attempts in a
header on the message, and more thoroughly reject messages
which reach some maximum number of attempts. I think you could
do the final rejection by setting a routing key on the message
when you reject it for the last time, and having exchange B be
a direct exchange which routes to either queue B or some final
deadletter queue.<br>
<br>
If you want exponential backoff in the retries, then life gets
more complicated (multiple timeout queues, selected between by
a routing key set by the consumer of A?). We are currently
pussyfooting around this issue at my company. I will report
back here if we ever implement a good solution!<br>
</blockquote>
<br>
I've just written some code to do exactly this; limited retries
with exponential backoff. That said, we're kind of cheating in
that we store the retry state in a secondary datastore and
buffer messages in the client. <br>
<br>
So whenever we receive a message, we:<br>
<ul>
<li>When we post each message, we assign it a unique
message_id<br>
</li>
<li>Lookup message's due time by it's message_id property in
our datastore</li>
<li>Stash the message in a heap queue</li>
<li>When the message becomes due, remove it from the heap
queue and pass it to the client code.</li>
<li>If the client code succeeds, then we finally ack the
message. Otherwise, we reject the message. <br>
</li>
</ul>
</blockquote>
<br>
I'm currently writing almost exactly the same thing! The
difference being that i'm putting the due time in a header on the
message rather than in a lookaside store, and that my component
moves messages from a queue to an exchange, rather than from a
queue to client code directly.<br>
<br>
<blockquote cite="mid:51F29EA7.20807@lshift.net" type="cite">
<p>Whilst you can scale this horizontally, you will need enough
buffer space to hold a reasonable proportion of your queue,
although<i> what</i> proportion depends on how much you care
about timeliness.</p>
</blockquote>
<br>
I'm not sure i understand. Don't you need to have enough space to
hold all the messages that could be delayed at any given time? In
our case, that happens to not be all that large, fortunately.<br>
</blockquote>
I've set the prefetch buffer set to a conservative number of
messages; mostly to avoid accidentally causing a denial of service
of my own application. To be clear, this will mean that we don't see
some messages until after they become due, but that's okay in our
case. <br>
</body>
</html>