[rabbitmq-discuss] a lot of unacked messages "in the flight"

[Jerry Kuch]

On Tue, Jan 21, 2014 at 1:06 PM, Kane Kim <kane.isturm at gmail.com> wrote:

> Is it ok to have many unacked messages in rabbitmq (millions)?
>

The real answer is "it depends" but be mindful and understand what it means
and what it depends on.

 A message in the unacked state has been delivered to a consumer, but the
broker is still maintaining knowledge of it.  That way, if the consumer
crashes, and its channel closes, without having sent an ACK, the broker
knows to re-queue the message for another delivery attempt.  It does this,
because, in the absence of the consumer's ACK, it has no way of knowing the
consumer got any useful work done with the consumed message before it
perished.

Messages languishing on the broker in the 'unacked' state consume
resources.  At the very least memory.  If they were published with
persistence requested, or if the broker has paged them to disk under memory
pressure, they will also consume disk space.  Finally, there's some small
in memory indexing overhead associated with each message as well.

If you let messages build up too long you can get into trouble as your
broker runs low on memory or disk resources.  The broker will try to pause
publishers to avoid being swamped, but you're still best managing your
application in a way that such conditions don't exist for long periods.

If you have *millions* of un-acked messages in queues, I would start to
wonder if perhaps your consumers are getting their work done and failing to
ACK.  Or perhaps they're working properly and obeying proper ACK
discipline, but incoming work is arriving at a rate that outpaces them.  In
that case you'd be wise to add more consumers....

A good general mantra:  "A Happy Rabbit is an Empty" rabbit.  Message
brokers will hold messages to buffer against transient spikes in traffic,
but they're not intended to be long term stores of persistent data, and
shouldn't be treated as such.  If you have a lot of messages piling up in
the broker, look at your producer arrival and consumer exit situations to
understand what's happening.  It's often a sign that either the ambient
conditions of your system have changed (e.g. a flood of new demand due to a
surge in work) or something has crashed or backed up...

Knowing the expected and usual queue sizes and whatnot, learned by
experience and observation of your system, is a good practice in
operationalizing this sort of stuff.

Best regards,
Jerry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140122/0b8f0174/attachment.html>