[rabbitmq-discuss] Rabbitmq Crash

Thu Jun 17 13:54:23 BST 2010

Our server has 32GB ram. And most of the time 25GB is free. Please note
that it is occurring on production system. We try to reproduce it on
different machines but unable to do so. Ubuntu Server 9.04 is installed
on server. Please note that if we delete mnesia files then we are able
to start rabbitmq server. We suspect that mnesia files get corrupted due
to which rabbitmq server does not start. Can you please guide us on
exactly what log files do you need us to provide you?

On Thu, 2010-06-17 at 13:27 +0100, Matthew Sackman wrote:

> Hi Waqas,
> 
> Please try and keep the mailing list CC'd.
> 
> On Thu, Jun 17, 2010 at 05:13:23PM +0500, Waqas Badar wrote:
> > We are using RabbitMQ 1.7.1 and we are
> > using Java client 1.7.1 and for python pyamqp 0.6.1 and txamqp 0.3.
> 
> Ok. 1.7.1 was less capable about dealing with situations in which it was
> running out of memory. I'd recommend upgrading to 1.8.0, though be aware
> that none of your queues, exchanges, bindings or messages will be
> retained when you upgrade.
> 
> > Please note that most of the time it crashes after 200000 messages.
> > Following crash reports are found in rabbit-sasl.log file.
> > 
> > =CRASH REPORT==== 15-Jun-2010::22:41:10 ===
> >   crasher:
> >     pid: <0.155.0>
> >     registered_name: rabbit_persister
> >     exception exit: {{badmatch,eof},
> >                      [{rabbit_persister,internal_load_snapshot,2},
> >                       {rabbit_persister,init,1},
> >                       {gen_server,init_it,6},
> >                       {proc_lib,init_p_do_apply,3}]}
> >       in function  gen_server:init_it/6
> >     initial call: rabbit_persister:init/1
> >     ancestors: [rabbit_sup,<0.109.0>]
> >     messages: []
> >     links: [<0.112.0>,<support at rabbitmq.com 0.156.0>]
> >     dictionary: []
> >     trap_exit: true
> >     status: running
> >     heap_size: 987
> >     stack_size: 23
> >     reductions: 431
> >   neighbours:
> 
> Err, this looks like it crashing when it's starting up, not it crashing
> when it's just run out of memory. The eof is interesting though. This
> could just be an old bug where you needed to move the old persister log
> out of the way:
> 
> In /var/lib/rabbitmq/mnesia/$NODENAME/ you should find a
> rabbit_persister.LOG and a rabbit_persister.LOG.previous file. It's
> likely the .previous version is much bigger than the non .previous
> version. You could try moving the rabbit_persister.LOG out of the way,
> and then renaming rabbit_persister.LOG.previous to rabbit_persister.LOG
> and see if it starts up.
> 
> Again, I *think* this was a bug that was reported quite often with 1.7.1
> and older and something that we've subsequently fixed, so again, you
> might have better luck after upgrading Rabbit to 1.8.0.
> 
> I am still curious to see the logs of rabbit as it crashes on you if you
> can find/produce those. What platform are you running on, and how much
> RAM is available in the machine?
> 
> Matthew
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20100617/32ca0c66/attachment.htm>