[rabbitmq-discuss] Push to back of Queue on NAck

[Will Hughes]

This is following on from a tweet I made earlier about wishing that 
RabbitMQ had the ability to push messages to the back of the queue when 
NAcked. 
Alvaro Videla replied suggesting I ask for the feature here. 


Scenario: 
I'm processing messages from a queue with confirmation required. 
The number of messages relative to the consumption rate can be quite high 
at times. In excess of hundreds of thousands of messages, with consumption 
rate in the low hundreds per second range. 

Sometimes we have to abandon processing a message for some transient reason 
(eg: external resource is locked/unavailable), and so we send a NAck.  
At the moment, RabbitMQ will immediately redeliver that message to any of 
the consumers. If the transient condition is still in place, we can 
potentially get into a tight loop where we're constantly consuming the same 
message(s) hundreds of times per second. 

For our scenario, we *can* duplicate the message, send it to the 
exchange/queue, and Ack the original. 

Something I thought might be a bit nicer is if we could have an option to 
have RabbitMQ put the message at the back of the queue. Depending on the 
size of the queue and our processing rate, this might give us an easier way 
to try other messages in the queue first. 

Another option I was thinking of to enforce more a delay was to see if I 
could chain together two queues using dead-letter-exchange settings: 

Queue A, no TTL, DLX=Exchange B (with Queue B bound to it). 
Queue B, ttl of (x) seconds, DLX= Exchange A (with Queue A bound to it). 

When we NAck the mesage from Queue A, we set requeue=false. 

I've not seen much discussion so far on whether it's sane to hook up two 
queues/exchanges in a loop like this. 
Closest I've seen is this: 
http://yuserinterface.com/dev/2013/01/08/how-to-schedule-delay-messages-with-rabbitmq-using-a-dead-letter-exchange/ but 
that's just using TTL to DLX to delay processing intially, not the whole 
chain. 

Suggestions or comments? 

Cheers
Will Hughes. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130726/a8189ff1/attachment.htm>