[rabbitmq-discuss] RabbitMQ memory management

Fri Sep 12 00:59:27 BST 2008

Edwin,

On Thu, Sep 11, 2008 at 11:46 PM, Edwin Fine <emofine at gmail.com> wrote:
> I don't know if I am reading this discussion correctly, but I am using
> RabbitMQ specifically in a store-and-forward context. It should be able to
> store persistent messages until the disk fills up even if no consumer is
> draining the queue. If it is going to die because memory fills up before
> forwarding gets done, that's a show-stopper for me. I don't mean to beat you
> over the head by comparing RabbitMQ to WebSphere MQ, but it doesn't fall
> over in this scenario unless the disk fills up (or the max queue depth is
> reached), and even then, it just refuses to take more messages. IMHO, this
> is how any messaging and queuing system that supports persistent messages
> should work.
>
> Please tell me I am misunderstanding.

No, you've understood it correctly. If you send enough non-persistent
messages, eventually the queues will fill up, the broker will crash
and you will lose all of the messages that you sent to that node. ATM
Rabbit does not actively refuse to accept a message based on any
system resource usage. The most it will do is to not acknowledge the
receipt of a message that it cannot accept the responsibility of
requeueing should a crash occur.

It is perfectly legitimate to compare Rabbit to MQ Series - maybe you
can elaborate on the conditions and constraints under which MQ Series
implements the behaviour you expect. Are you talking about
non-transactional transient messages, transactional transient
messages, non-transactional persistent messages or transactional
persistent messages?

Furthermore, I would recommend due diligence to anybody evaluating
Rabbit - or any piece of software for that matter - make sure you test
the real hard requirements your application has.

Remember, we're not trying to sell you something, so if Rabbit doesn't
meet your needs, ask us, and we will give you an honest opinion of the
limitations of the system, even if it means you using something else.

Then you have the option of leaving it for something else, or
improving it to suit your requirements.

HTH,

Ben