[rabbitmq-discuss] : Delayed processing of data

[Ajinkya Apte]

Thanks for the reply.  So the only problem I see with what I want to do and with whats available in RabbitMQ is I will have to time slice and create multiple queues.
For example:
One queue with a timeout of 5 min, the other 10 min. Then according to the delay time just put incoming data into the proper queue.
The down side I can see is that new data that will be thrown in a queue with say a 10 min timeout, will be theoretically processed anywhere between 1-10 mins. I cannot kinda guarantee that the data will be handled only after 10 mins.

Does this make sense to you?

Do you create a new queue per request coming in, depending on a the delay?

AJ

From: <Laing>, Michael <michael.laing at nytimes.com<mailto:michael.laing at nytimes.com>>
Reply-To: Discussions about RabbitMQ <rabbitmq-discuss at lists.rabbitmq.com<mailto:rabbitmq-discuss at lists.rabbitmq.com>>
Date: Thursday, March 20, 2014 6:36 PM
To: Discussions about RabbitMQ <rabbitmq-discuss at lists.rabbitmq.com<mailto:rabbitmq-discuss at lists.rabbitmq.com>>
Subject: Re: [rabbitmq-discuss] : Delayed processing of data

We currently use TTLs and Dead Letter Exchanges to accomplish this.

An app we call Route looks at messages and either:

  1.  sends them to an exchange called Distribute, or
  2.  declares a queue with a name reflecting the embargo datetime and a DLX of Distribute and sends the message to it with a TTL calculated to release at that datetime.

This works fine. You can dynamically set a policy to delete the queue.

Probably we will move away from this in the next few months simply because we wish to persist as little as possible in rabbitmq. But it is quite simple and functional.

ml


On Thu, Mar 20, 2014 at 2:11 PM, Ajinkya Apte <aapte at zynga.com<mailto:aapte at zynga.com>> wrote:
Hey guys,
I want to delay the processing of some data coming into a queue. The data has delayTime associated with it.

What I want to do?

  1.  Data is coming in with a delayTime associated with it.
  2.  Some data can come with the delayTime=5 mintes from now, some data with delayTime=10 and so on. This delayTime can be anything.
  3.  I want to delay the processing of this data.

Questions?

  1.  From the documentation seems like using message TTL won't be the way to go, right?
  2.  Will I have to use the 'Dead Letter Exchanges' concept along with multiple retry queues having different timeouts?
  3.  Any other suggestions?

Please let me know.
Thanks,

AJ

_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss at lists.rabbitmq.com<mailto:rabbitmq-discuss at lists.rabbitmq.com>
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140321/7e57c1e7/attachment.html>