[rabbitmq-discuss] RabbitMQ crash after few hours

[Artsiom]

Thanks for reply.
/
/usr/lib/rabbitmq/bin/rabbitmq-env: fork: retry: Resource temporarily
unavailable
/
I've checked that no limit of processes was reached (over 300 proc. were 
running),
I was able to start new processes. May be there was a file lock on one 
of the (configuration?) files?
This error disappeared when local rabbitmq broker have become able to 
see other brokers.

Most of all I'm interested why 3 nodes crashed at once and why following 
message appeared:

     =ERROR REPORT==== 14-Feb-2012::19:13:47 ===
     ** Generic server <0.1093.0> terminating
     ** Last message in was {mnesia_locker,wosnfs at dell39,granted}
     ** When Server state == 5
     ** Reason for termination ==
     ** {unexpected_info,{mnesia_locker,wosnfs at dell39,granted}}

Following error messages may be the consuquece of this.
Look forward to results of your investigation.

P.S. better way to recover cluster is (to leave cluster if possible), 
clear mnesia on crashed nodes and rejoin them?

On 02/22/2012 08:07 PM, Simon MacMullen wrote:
> On 22/02/12 16:13, Artsiom wrote:
>> /usr/lib/rabbitmq/bin/rabbitmq-env: fork: retry: Resource temporarily
>> unavailable
>
> This implies that you're running into some OS-imposed limit on the 
> number of (OS not Erlang) processes. When you do that, Bad Things are 
> likely to happen.
>
> I also see a bunch of errors along the lines of:
>
> =ERROR REPORT==== 14-Feb-2012::20:51:30 ===
> Mnesia(wosnfs at dell39): ** ERROR ** mnesia_event got 
> {inconsistent_database, running_partitioned_network, wosnfs at dell33}
>
> in machine 4's log, which is another bad sign - RabbitMQ clusters are 
> not designed to tolerate network partitions.
>
> I'll look into some of the other error messages that are showing up in 
> your logs; it's possible some of then are real, but you have some bad 
> things going on underneath RabbitMQ - you need to sort those out first,
>
> Cheers, Simon
>
--
Best regards,
Artsiom
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120222/97d33609/attachment.htm>