[rabbitmq-discuss] rabbitmq-server crashes hard while consuming 31GB of RAM
mh at foldr3.com
Wed Dec 21 12:31:09 GMT 2011
On 12/21/2011 12:44 PM, Matthias Radestock wrote:
> On 21/12/11 11:26, Muharem Hrnjadovic wrote:
>> On 12/21/2011 12:06 PM, Matthias Radestock wrote:
>>> Try doing that *while* it is wedged. And also send us the output of
>>> 'rabbitmqctl report'. Again, while rabbit is wedged.
>> Please see https://bugs.launchpad.net/openquake/+bug/894024/comments/8
>> and https://bugs.launchpad.net/openquake/+bug/894024/comments/9
>>> My hunch is that you probably have many thousands of celery result
>>> queue processes.
>> Hmm .. I can see that the celery result queues (which are all
>> auto-delete queues) are deleted after the result has been consumed.
>> Why would the respective processes still hang around?
> I was wrong. You have more than 47,000 connections! And the same
> number of channels (unsurprisingly). But you "only" have about 16,000
Wow! That means our celery workers are opening a connection per task and
not closing it. We *do* get stuck after approx. 47,000 tasks.
Thank you very much for looking into this and finding the actual
> Congratulations. This is by far the fattest rabbit I've ever seen
> (except those stuffed full of messages).
> I have no idea why all these connections, channels and queues are
> there. That's a question for the celery guys.
Right. Now that you have pointed out the root cause I want to check a
couple of things before involving the celery folks.
Best regards/Mit freundlichen Grüßen
Muharem Hrnjadovic <mh at foldr3.com>
Public key id : B2BBFCFC
Key fingerprint : A5A3 CC67 2B87 D641 103F 5602 219F 6B60 B2BB FCFC
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 900 bytes
Desc: OpenPGP digital signature
More information about the rabbitmq-discuss