[rabbitmq-discuss] rabbitmq stopped working after removing a static route

[Kyle O'Donnell]

Has anyone experienced this before?

I added a static route to 2 of the 5 servers in my rabbitmq cluster to facilitate a network change.  No rabbit issues after the route was added.  A few days later we were done with the network change and I removed the static routes...

Almost immediately the 2 rabbit nodes with the added/removed static route were no longer able to maintain a connection to the cluster.  

*note* the static route was on a different subnet than the rabbitmq network and rabbi

Here are the logs (treehorn=localhost):

=ERROR REPORT==== 4-Feb-2012::17:17:10 ===
** Node rabbit at biglebowski not responding **
** Removing (timedout) connection **

=ERROR REPORT==== 4-Feb-2012::17:17:10 ===
** Node rabbit at thedude not responding **
** Removing (timedout) connection **

=ERROR REPORT==== 4-Feb-2012::17:17:10 ===
** Node rabbit at bunny not responding **
** Removing (timedout) connection **

=INFO REPORT==== 4-Feb-2012::17:17:10 ===
node rabbit at biglebowski lost 'rabbit'

=ERROR REPORT==== 4-Feb-2012::17:17:25 ===
** Node rabbit at donny not responding **
** Removing (timedout) connection **

=INFO REPORT==== 4-Feb-2012::17:17:42 ===
node rabbit at thedude lost 'rabbit'

=INFO REPORT==== 4-Feb-2012::17:17:42 ===
node rabbit at bunny lost 'rabbit'

=INFO REPORT==== 4-Feb-2012::17:17:42 ===
node rabbit at donny lost 'rabbit'

=ERROR REPORT==== 4-Feb-2012::17:18:54 ===
Mnesia(rabbit at treehorn): ** ERROR ** mnesia_event got {inconsistent_database, running_partitioned_network, rabbit at thedude}

=ERROR REPORT==== 4-Feb-2012::17:18:54 ===
Mnesia(rabbit at treehorn): ** ERROR ** mnesia_event got {inconsistent_database, running_partitioned_network, rabbit at bunny}

=ERROR REPORT==== 4-Feb-2012::17:18:54 ===
Mnesia(rabbit at treehorn): ** ERROR ** mnesia_event got {inconsistent_database, running_partitioned_network, rabbit at biglebowski}

=ERROR REPORT==== 4-Feb-2012::17:18:54 ===
Mnesia(rabbit at treehorn): ** ERROR ** mnesia_event got {inconsistent_database, running_partitioned_network, rabbit at donny}

I had to stop/start rabbit on the 2 nodes in order for them to rejoin the cluster.

I can't imagine this is expected behavior, but is it a bug?

Thanks,
Kyle