[rabbitmq-discuss] Clusters stop working/passing messages

[schaibaa]

Hi Guys,

I was hoping I might be able to get some help -- I'm running a 3 node 
rabbitmq cluster.  All 3 are currently in Disc mode.

The cluster isn't that busy ... I'm not sure how to obtain throughput 
stats, but my estimate is a few hundred messages per hour.

Anyway, every week or so, we notice an issue in our application, and I'll 
check the dashboard.  One of the nodes will be red and offline.  The 
RabbitMQ service on that machine is still running, but it appears offline 
to the other two nodes.

Restarting the nodes seems to immediately fix the problem... I'm just 
hoping to eliminate this issue.

OS:  Windows server 2008 R2
RabbitMQ Version: 2.8.0

The only thing I see in the logs is below:

=ERROR REPORT==== 19-Aug-2012::12:37:55 ===
Mnesia('rabbit at NODE3'): ** ERROR ** mnesia_event got 
{inconsistent_database, running_partitioned_network, 'rabbit at NODE1'}

=ERROR REPORT==== 19-Aug-2012::12:37:55 ===
Mnesia('rabbit at NODE3'): ** ERROR ** mnesia_event got 
{inconsistent_database, starting_partitioned_network, 'rabbit at NODE2'}

This when I restarted the service on NODE1 ... and a similar log message 
appears on the other nodes.

Any suggestions?  Relatively new to RabbitMQ ...

Thanks :)


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120819/8130c984/attachment.htm>