[rabbitmq-discuss] RabbitMQ memory management

Ben Hood 0x6e6562 at gmail.com
Sun Sep 14 11:53:13 BST 2008


On Sat, Sep 13, 2008 at 11:28 AM, Matthias Radestock
<matthias at lshift.net> wrote:
> Alexis Richardson wrote:
>> * provide a means to tell producers to back off, or alert an operator
> The easiest solution I can think of is to
> 1) configure Erlang's memsup
> (http://www.erlang.org/doc/apps/os_mon/index.html) to trigger alarms
> when memory consumption gets tight.
> This can be done without any code change; in the rabbitmq-server startup
> script simply change the "-os_mon start_memsup false" to "true" and
> adjust the thresholds with additional options of the form "-memsup
> <param> <value>"
> When a threshold is reached, a message like this will appear in the
> rabbit.log:
> =INFO REPORT==== 13-Sep-2008::10:59:37 ===
>     alarm_handler: {set,{process_memory_high_watermark,<0.31.0>}}
> When the memory usage drops below the threshold again a similar message
> is logged.
> One can also set up SNMP monitoring, but that is more complicated.
> 2) get queues to drop messages when memory consumption is above the
> thresholds.
> This does require some coding, but not very much.
> We set up an alarm handler that informs all a node's queues when a "high
> memory" alarm is set/cleared. Queues record that information as part of
> their state.
> When a message is routed to a queue while the alarm is set, and the
> queue cannot immediately route the message to an auto-ack consumer - in
> other words, the message requires queueing - it discards the message. If
> that happens and either the mandatory or immediate flag were set, and
> the message could not be routed to any other queues / consumers, then
> the message is returned to the sender with basic.return.
> We can think of other actions to take instead of discarding messages,
> but the above is simple and neatly exploits the existing
> mandatory/immediate functionality.

I think this is a good idea of how to use the mangement functionality
that comes with OTP. This is one of the reasons why we are using OTP
in the first place.


More information about the rabbitmq-discuss mailing list