[rabbitmq-discuss] Confusing disk free space limit warning
Mark Hingston
mark.hingston at vardr.com
Mon Sep 17 05:31:22 BST 2012
Hi guys,
I'm new to the list so please forgive me if I forgot any important info.
First of all thanks to the rabbitmq devs for releasing a great product :)
I'm running Rabbitmq 2.8.4 and connecting to it with a celery / kombu /
amqplib client. I'm running on Ubuntu 12.04.1 LTS on an Amazon EC2
instance. After running for quite some time without issue, I saw this
warning in the Rabbit log:
=INFO REPORT==== 16-Sep-2012::06:49:17 ===
=INFO REPORT==== 16-Sep-2012::06:49:17 ===
Disk free space limit now below limit. Free bytes:2000396288
Limit:1000000000
=INFO REPORT==== 16-Sep-2012::06:49:17 ===
alarm_handler: {clear,{resource_limit,disk,rabbit at mq1}}
The amount of free space reported in the log is around twice the amount
of space that rabbit requires in this instance, so I can't understand
why this log occurred. Following this rabbit did seem to rate limit all
clients, so I believe that it was acting as though we had breached the
limit. There were no subsequent rabbit logs for the next ~24 hours. Not
knowing much about rate limiting, etc, we actually restarted our amqp
clients and then rabbit itself, after which everything seemed to start
working fine again.
This is the output of rabbitmq status (done on the next day, which was
when i realised there was a throttling issue):
Status of node rabbit at mq1 ...
[{pid,828},
{running_applications,
[{rabbitmq_management,"RabbitMQ Management Console","2.8.4"},
{xmerl,"XML parser","1.2.10"},
{rabbitmq_management_agent,"RabbitMQ Management Agent","2.8.4"},
{rabbitmq_mochiweb,"RabbitMQ Mochiweb Embedding","2.8.4"},
{webmachine,"webmachine","1.7.0-rmq2.8.4-hg"},
{mochiweb,"MochiMedia Web Server","1.3-rmq2.8.4-git"},
{inets,"INETS CXC 138 49","5.7.1"},
{rabbitmq_stomp,"Embedded Rabbit Stomp Adapter","2.8.4"},
{rabbit,"RabbitMQ","2.8.4"},
{mnesia,"MNESIA CXC 138 12","4.5"},
{os_mon,"CPO CXC 138 46","2.2.7"},
{sasl,"SASL CXC 138 11","2.1.10"},
{amqp_client,"RabbitMQ AMQP Client","2.8.4"},
{stdlib,"ERTS CXC 138 10","1.17.5"},
{kernel,"ERTS CXC 138 10","2.14.5"}]},
{os,{unix,linux}},
{erlang_version,
"Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2]
[async-threads:30] [kernel-poll:true]\n"},
{memory,
[{total,35645328},
{processes,12347544},
{processes_used,12333872},
{system,23297784},
{atom,1548841},
{atom_used,1538490},
{binary,301336},
{code,17536634},
{ets,2116424}]},
{vm_memory_high_watermark,0.3999999998873157},
{vm_memory_limit,709948211},
{disk_free_limit,1000000000},
{disk_free,1808924672},
{file_descriptors,
[{total_limit,924},{total_used,10},{sockets_limit,829},{sockets_used,6}]},
{processes,[{limit,1048576},{used,225}]},
{run_queue,0},
{uptime,8035}]
...done.
Any help you guys could give with tracking down the cause of this issue
would be appreciated.
Cheers,
Mark
More information about the rabbitmq-discuss
mailing list