[rabbitmq-discuss] Determining age of queue contents

Sun Jan 22 14:23:45 GMT 2012

[Please CC as I am not on the list.]

The fine-grained stats plugin provides information about message processing rate and queue lengths, which can be used to infer whether a queue is "lagging". Two problems:

1. The only way to reliably infer the true processing rate of a queue is to implement a stateful watching algorithm, being a function both of the current processing rate as well as the *growth rate* of the queue; a queue that grows faster than messages are processes is a potential bottleneck. While fairly trivial to write, when you merely want to monitor queues from a Nagios plugin, I would wish this was something built into RabbitMQ itself, closest to where the information is stored. A Nagios plugin would have to store state data in the file system.

2. Even when looking at processing rate and growth rate, there is not enough sufficient information to determine a realistic churn rate. What's really needed is to track the age of the newest and last messages in the queue. If you have this age difference, you can define acceptables deadline for message processing in a monitoring system, which is a better heuristic than the processing rate.

Would it be possible to implement a way to extract a per-queue age value (in a performant way)?