[rabbitmq-discuss] msg_store_transient usage question

Thu Sep 26 18:21:55 BST 2013

Hi Simon,

That pages says this:

It is possible to set vm_memory_high_watermark_paging_ratio to a greater
value than 1.0. In this case queues will not page their contents to disc.

What would happen then? Will RabbitMQ stop receiving messages? Drop
messages?

Cheers,

Alvaro

On Thu, Sep 26, 2013 at 3:39 AM, Simon MacMullen <simon at rabbitmq.com> wrote:

> On 26/09/2013 4:37AM, Peter Wright wrote:
>
>> i was hoping someone could help explain to me under what circumstances
>> messages are written to the msg_store_transient directory.  i have been
>> witnessing this directory grow to 10's of GB - infact it spun out of
>> control at one point consuming almost 100G of space.  i do have a queue
>> with quite a few messages ready for consumption, yet when i monitor
>> memory utilization for this queue it rarely spikes above 20MB.
>>
>
> The memory use reported by the queue does not include message bodies
> (since these are shared across queues). Also it does not include anything
> that's been paged out, so if messages are paging out it can be quite low
> anyway.
>
> The decision as to whether to page messages is made globally based on how
> much memory is in use. All released versions of RabbitMQ (since 2.0.0 when
> the persister was rewritten) will start to page out when 20% of the way to
> the high watermark. By default the high watermark is at 40% of physical
> memory, so paging will start at 8% memory used.
>
> That's probably too cautious, so the next release (3.2.x) will raise the
> paging ratio from 20% to 50% (so paging will start at 20% memory used by
> default) and also make it configurable. See http://next.rabbitmq.com/**
> memory.html#memsup-paging<http://next.rabbitmq.com/memory.html#memsup-paging>
>
> Having said that, if msg_store_transient grows to 100GB of space, I
> suspect you probably have 100GB of messages. If you don't have 100GB of
> RAM, you're going to page out...
>
> Hope this helps.
>
> Cheers, Simon
>
> --
> Simon MacMullen
> RabbitMQ, Pivotal
> ______________________________**_________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.**rabbitmq.com<rabbitmq-discuss at lists.rabbitmq.com>
> https://lists.rabbitmq.com/**cgi-bin/mailman/listinfo/**rabbitmq-discuss<https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130926/d6e4a9ce/attachment.htm>