<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi, Simon.<br>
<blockquote cite="mid:4F22A81F.6050405@rabbitmq.com" type="cite">So
it looks like you are leaving the cluster and then resetting,
rather than the other way around (which is more normal). It looks
like the request to leave the cluster gets ignored in this case.
I'll file a bug.
<br>
</blockquote>
<br>
I've tried scenario, when removing node from cluster is performed
like this:<br>
<br>
1) join machine B (epbyminw2482t3) to machine A (epbyminw2482t2).<br>
<tt><br>
[root@epbyminw2482t3 ~]# rabbitmqctl stop_app &&
rabbitmqctl reset && rabbitmqctl cluster wosnfs@`hostname
-s` wosnfs@epbyminw2482t2 && rabbitmqctl start_app<br>
<-CUT-><br>
Clustering node wosnfs@epbyminw2482t3 with [wosnfs@epbyminw2482t3,<br>
wosnfs@epbyminw2482t2]
...<br>
...done.<br>
Starting node wosnfs@epbyminw2482t3 ...<br>
...done.</tt><br>
<br>
Everything is ok.<br>
<br>
2) remove node A from cluster<br>
<tt><br>
[root@epbyminw2482t2 ~]# <b>rabbitmqctl stop_app &&
rabbitmqctl reset && rabbitmqctl start_app</b><br>
Stopping node wosnfs@epbyminw2482t2 ...<br>
...done.<br>
Resetting node wosnfs@epbyminw2482t2 ...<br>
...done.<br>
Starting node wosnfs@epbyminw2482t2 ...<br>
...done.</tt><br>
<br>
From now node A and B are standalone, and operate<i> independently.</i><br>
<br>
3) restart broker on node B.<br>
<tt>Restarting rabbitmq-server: FAILED - check
/var/log/rabbitmq/startup_{log, _err}<br>
rabbitmq-server.</tt><br>
<br>
<tt>[root@epbyminw2482t3 ~]# cat /var/log/rabbitmq/startup_log <br>
Activating RabbitMQ plugins ...<br>
0 plugins activated:<br>
<-CUT-><br>
node : wosnfs@epbyminw2482t3<br>
app descriptor :
/usr/lib/rabbitmq/lib/rabbitmq_server-2.7.1/sbin/../ebin/rabbit.app<br>
home dir : /var/lib/rabbitmq<br>
config file(s) : (none)<br>
<-CUT-><br>
erlang version : 5.8.4<br>
<br>
-- rabbit boot start<br>
starting file handle cache
server ...done<br>
starting worker
pool ...done<br>
starting
database
...BOOT ERROR: FAILED<br>
Reason: {error,<br>
{unable_to_join_cluster,<br>
[wosnfs@epbyminw2482t3,wosnfs@epbyminw2482t2],<br>
{merge_schema_failed,<br>
"Bad cookie in table definition
mirrored_sup_childspec: wosnfs@epbyminw2482t3 =
{cstruct,mirrored_sup_childspec,ordered_set,[wosnfs@epbyminw2482t3],[],[],0,read_write,false,[],[],false,mirrored_sup_childspec,[key,mirroring_pid,childspec],[],[],{{1327,927347,889571},wosnfs@epbyminw2482t2},{{3,1},{wosnfs@epbyminw2482t3,{1327,927549,609065}}}},
wosnfs@epbyminw2482t2 =
{cstruct,mirrored_sup_childspec,ordered_set,[wosnfs@epbyminw2482t2],[],[],0,read_write,false,[],[],false,mirrored_sup_childspec,[key,mirroring_pid,childspec],[],[],{{1327,927550,935402},wosnfs@epbyminw2482t2},{{2,0},[]}}\n"}}}<br>
Stacktrace: [{rabbit_mnesia,init_db,3},<br>
{rabbit_mnesia,init,0},<br>
{rabbit,'-run_boot_step/1-lc$^1/1-1-',1},<br>
{rabbit,run_boot_step,1},<br>
{rabbit,'-start/2-lc$^0/1-0-',1},<br>
{rabbit,start,2},<br>
{application_master,start_it_old,4}]<br>
{"Kernel pid
terminated",application_controller,"{application_start_failure,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot}}}}}"}</tt><br>
<br>
We got this error.<br>
<br>
3.1) But, if we stop application on node A<br>
<tt>[root@epbyminw2482t2 ~]# rabbitmqctl stop_app</tt><br>
<br>
3.2)<tt> [root@epbyminw2482t3 ~]# service rabbitmq-server restart<br>
Restarting rabbitmq-server: RabbitMQ is not running<br>
SUCCESS<br>
rabbitmq-server.</tt><br>
<br>
Rabbitmq broker on node B(!) restarts successfully. <br>
Until broker(or application) on node A is down, node B work fine.<br>
When broker on node A is up, node B faces above mentioned troubles
on restart (or `rabbitmqctl reset`).<br>
Workaround is to remove mnesia db on node B, but it is bad idea,
while node B is still joined to the cluster.<br>
It seems that there is some kind of rule about order of joining and
removing nodes from cluster...<br>
<br>
In our application all rabbitmq brokers have to joined in cluster as
disk nodes, and it have to be possible<br>
to add\remove any node at any time. Could you suggest me, have to
achieve that and avoid "bad cookie" problem?<br>
<br>
</body>
</html>