[rabbitmq-discuss] Slow consumer disconnect availability on RAbbitMQ

Mon Jan 31 15:30:49 GMT 2011

Hi Prashant,

On Thu, Jan 27, 2011 at 05:15:50AM +0530, Prashant Yadav wrote:
> Actually we do have a requirement where I want consumer to stop getting message once I know that its slow( my consumption logic connects to remote server and if server is not responding or is slow for any reason I dont want my queue to get clogged), I would want to stop queue from populating.

Why?

In general, it seems to me that slow consumer detection is solving the
problem the wrong way.

Rabbit and message brokers in general are there to decouple components.
Rabbit is pretty good these days at coping with receiving messages
faster than they're being sent out, buffering messages in RAM and on
disk and generally absorbing such pressure.

It seems odd to me that because you have pressure downstream (slow
responding server) you want to address that upstream by throwing away
the queue and its messages. If your messages are of transient value then
I'd have thought that the per-queue message TTL would be ideal: that
will silently drop messages from the queue once they have resided in the
queue for more than the specified time.

> The catch here is , I too dont want to purge all the messages or drop it . I too tried to use TTL and queue lease , but am not sure about messages resisding in the queue i.e whether they are requed or silently dropped( in my case I want them to by returned back to exchange and make the queue alive after some duration)

What you seem to be asking for is that the broker buffers
(store+forward) messages whenever things are going fine, but that once
the consumer starts going slowly, the back pressure is passed all the
way back up to the producer. This negates one of the key purposes of the
broker - if you want this level of direct connection between the
producer and consumer then why not connect them directly together - why
are you using RabbitMQ at all in the middle?

You may also like to look at the "immediate" flag on basic.publish which
might do what you want.

Best wishes,

Matthew