[rabbitmq-discuss] Slow consumer disconnect availability on RAbbitMQ

[Prashant Yadav]

Hi Jerry,

Thanks for getting back. 

Actually we do have a requirement where I want consumer to stop getting message once I know that its slow( my consumption logic connects to remote server and if server is not responding or is slow for any reason I dont want my queue to get clogged), I would want to stop queue from populating.

The catch here is , I too dont want to purge all the messages or drop it . I too tried to use TTL and queue lease , but am not sure about messages resisding in the queue i.e whether they are requed or silently dropped( in my case I want them to by returned back to exchange and make the queue alive after some duration)

N.B: I guess suspending the queue can be little tricky , let me know yout thoughts

Regds
Prashant


-----Original Message-----
From: Jerry Kuch [mailto:jerryk at vmware.com]
Sent: Wed 1/26/2011 11:45 PM
To: Prashant Yadav; rabbitmq-discuss at lists.rabbitmq.com
Subject: RE: Slow consumer disconnect availability on RAbbitMQ
 
Hi, Prashant...

> Do we have slow consumer detection available in RabbitMQ like we have
> in Apache qpid ( Slow Consumer Disconnect (SCD) is a new feature in
> Qpid that provides a configurable mechanism to prevent a single slow
> consumer from causing a back up of unconsumed messages on the broker )

Possibly, but can I ask a couple of questions about your actual use
case?  First, are you actually seeing cases where your consumers are
so slow that the broker's own self defense mechanisms like paging
messages to disk and memory-based producer flow control either aren't
engaging, or are being overwhelmed in some way?  Second, are you sure
you actually want to disconnect clients that are consuming slowly, but
whose connections are otherwise alive and healthy?

> This is a very nice feature wherein we could identify and take
> corrective actions against the consumer. Please suggest any
> alternative method to achieve this behavior. Also when we identify SCD
> can we defer the queue for some time( timed defer) before it comes
> back to service something like suspend Queue for 5 mins and try
> again.

I presume that one of the requirements of the behavior you desire
would be that you discard messages from a queue whose consumers aren't
keeping up to its load since simply disconnecting a slow consumer
without some ability to replace it by one or more less slow ones isn't
going to help.  In that case, you could try the per-queue message TTL
extension that Rabbit supports.  It's documented here:

http://www.rabbitmq.com/extensions.html#queue-ttl

This lets you declare queues with a limit on how long a message
published to a queue can remain there before it is discarded.  This
would let you escape the back up of unconsumed messages that you're
concerned about and, for many types of message flows, is a sensible
behavior in any event since the relevance of ancient messages that
have been languishing unconsumed in queues probably degrades with time
anyway.

Best regards,
Jerry

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110127/792f2061/attachment-0001.htm>