[rabbitmq-discuss] Push to back of Queue on NAck
tom.anderson at timgroup.com
Fri Jul 26 18:03:22 BST 2013
On 26/07/13 17:07, Ceri Storey wrote:
> (26/07/13 16:49), Tom Anderson wrote:
>> Implemented exactly as described there, it yields an infinite loop
>> for unprocessable messages. You might therefore also want to keep a
>> count of the number of processing attempts in a header on the
>> message, and more thoroughly reject messages which reach some maximum
>> number of attempts. I think you could do the final rejection by
>> setting a routing key on the message when you reject it for the last
>> time, and having exchange B be a direct exchange which routes to
>> either queue B or some final deadletter queue.
>> If you want exponential backoff in the retries, then life gets more
>> complicated (multiple timeout queues, selected between by a routing
>> key set by the consumer of A?). We are currently pussyfooting around
>> this issue at my company. I will report back here if we ever
>> implement a good solution!
> I've just written some code to do exactly this; limited retries with
> exponential backoff. That said, we're kind of cheating in that we
> store the retry state in a secondary datastore and buffer messages in
> the client.
> So whenever we receive a message, we:
> * When we post each message, we assign it a unique message_id
> * Lookup message's due time by it's message_id property in our datastore
> * Stash the message in a heap queue
> * When the message becomes due, remove it from the heap queue and
> pass it to the client code.
> * If the client code succeeds, then we finally ack the message.
> Otherwise, we reject the message.
I'm currently writing almost exactly the same thing! The difference
being that i'm putting the due time in a header on the message rather
than in a lookaside store, and that my component moves messages from a
queue to an exchange, rather than from a queue to client code directly.
> Whilst you can scale this horizontally, you will need enough buffer
> space to hold a reasonable proportion of your queue, although/what/
> proportion depends on how much you care about timeliness.
I'm not sure i understand. Don't you need to have enough space to hold
all the messages that could be delayed at any given time? In our case,
that happens to not be all that large, fortunately.
> Also, you are effectively implementing a secondary queue in your
> application (retaining rabbit for it's reliability properties), which
> seems less than ideal.
Yes. We did kick around the idea of writing a custom exchange to do
this, but that sounded really scary.
Tom Anderson | Developer | +44 20 7826 4312 | timgroup.com
STATEMENT OF CONFIDENTIALITY: The information contained in this
electronic message and any attachments to this message are intended for
the exclusive use of the addressee(s) and may contain confidential or
privileged information. If you are not the intended recipient, please
notify Tom Anderson at TIM Group at tom.anderson at timgroup.com and
destroy all copies of this message and any attachments.
TIM Group is the trading name for YouDevise Limited. YouDevise Limited
is registered in England, No. 3331176. Registered office: 3 Copthall
Avenue, London, EC2R 7BH.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rabbitmq-discuss