[rabbitmq-discuss] rabbitmq-server crashes hard while consuming 31GB of RAM

Wed Nov 23 10:30:11 GMT 2011

Hello there!

I am running a seismic calculation using python-celery (rev. 2.4.2, on
an ubuntu 11.04 server) and the number of task queues peaks at 6700.

nevertheless rabbitmq-server (rev. 2.7.0) crashes approx. 2.5 hours into
the calculation having 3912 queues and consuming 31GB of RAM at that
point.

99.9% of the queues are celery task queues and contain 0 or 1 messages;
the main celery task queue contains up to 4096 messages.

How do I figure out why rabbitmq is consuming that much RAM and why it's
crashing?

BTW, there was nothing in the logs but I do have a crash dump:

$ ll /var/lib/rabbitmq
total 487852
-r-------- 1 rabbitmq rabbitmq        20 2011-11-18 00:00 .erlang.cookie
-rw-r----- 1 rabbitmq rabbitmq 499544558 2011-11-23 10:23 erl_crash.dump
drwxr-xr-x 4 rabbitmq rabbitmq      4096 2011-11-23 07:38 mnesia/

Best regards/Mit freundlichen Grüßen

-- 
Muharem Hrnjadovic <mh at foldr3.com>
Public key id   : B2BBFCFC
Key fingerprint : A5A3 CC67 2B87 D641 103F  5602 219F 6B60 B2BB FCFC

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 900 bytes
Desc: OpenPGP digital signature
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20111123/19832ae2/attachment.pgp>