[rabbitmq-discuss] RabbitMQ Crashes with Out Of Memory errors.

[harel at thisisglow.com]

Hi, 

We've been using RabbitMQ Since early 1.x days and it has been a great and 
reliable queueing system. In the last few months we've started experiencing 
severe rabbit crashes, all due to 'out of memory' errors, where rabbit is 
killed by the kernel. We're using rabbit with celery in a python 
application.

We did go through some infrastructure changes when those crashes began - we 
moved to a different data centre and upgraded rabbit from 2.x to latest 3.x 
(as well as celery to the latest 3.x version). The spec of the machine 
running rabbit did not change though - the old and new servers had 16GB of 
ram and both had a few other things running on them - a few celery nodes 
and memcached (limited to 2GB). 
In our old setup, no matter how big the queues become rabbit stayed alive 
and well. In the new setup once memory usage spikes too high, rabbit dies. 

As an intermediate solution we've prepared a 32GB ram machine and will use 
the old 16GB one as a secondary (i.e., run rabbit in a cluster). However it 
is very important to us to understand why we are experiencing those crashes 
- adding hardware only solves the symptom but not the problem. 
Why does rabbit fail when memory is spiking? What causes the kernel to kill 
it? The error message is always a cryptic Out Of Memory and Killed by 
Kernel though I'm convinced there is something trivial we are missing in 
our new setup that causes the kernel to terminate rabbit in those cases. 

Any help is highly appreciated. 

Harel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130815/04db12d5/attachment.htm>