[rabbitmq-discuss] General approaches for tracking unreclaimed memory

[Garrett Smith]

On Wed, Oct 28, 2009 at 11:01 AM, Matthias Radestock
<matthias at lshift.net> wrote:
> Garrett,
>
> Garrett Smith wrote:
>> I need both non-exclusive/non-auto-delete so that the queue is
>> preserved when either the creating connection or the last consumer
>> goes away.
>
> Why do you want the queue to be preserved when you do not want any of
> its messages preserved?

Let's say I restart a process and terminate a connection on an
exclusive queue. I'm interested in the messages -- I'm gonna come back
online after the restart. I don't want to delete the queue. Trust me
:)

Let's say, on the other hand, that I decommission the queue owner and
(admittedly sloppily) don't delete the queue. Messages pile up and I
run out of memory/processes.

I understand this is entirely application specific. It's like running
out of disk space when you keep writing to a database. I understand
that and will deal with it easily enough, though I do think that
protections against resource exhaustion are important.

>> I know that the expires/TTL is ignored in rabbit, and is perhaps not
>> entirely fleshed out in the 0.8 spec, but it seems an important
>> feature wrt resource resource management.
>
> Discarding messages from a queue based on some expiry property is but
> one of many different conceivable ways in which one may wish to manage
> resources. Yes, it happens to be one that is half-defined by the spec,
> but I doubt it's the most useful one. We will get round to implementing
> it eventually, but we are considering others too, such as limiting
> queues by msg count or size, eviction policies based on message
> properties other than expiry, etc.

In my experience (again, working with qpid), time-to-live/expires is a
flexible and effective way to deal with this problem. Not trying to
spark a debate, but it's not obvious to me how that approach is
inferior to the others you listed. I understand if this is topic for
future consideration though.

Garrett