[rabbitmq-discuss] unable to join the cluster node ..

Tue Oct 15 16:58:06 BST 2013

Hello Team

         while stoping the Ram node as master unable to fail over to
disk which alive more day than ram node and surprisingly it's why this
disc node not take over as master ( which uptime was higher than ram
node)

=ERROR REPORT==== 14-Oct-2013::10:57:01 ===
Mnesia('rabbit at linuxserv05'): ** ERROR ** (core dumped to file:
"/var/lib/rabbitmq/MnesiaCore.rabbit at linuxserv05_1381_809421_158007")
 ** FATAL ** Failed to merge schema: Bad cookie in table definition
rabbit_user_permission: 'rabbit at linuxserv05' =
 {cstruct,rabbit_user_permission,set,['rabbit at linuxserv01'],['rabbit at linuxserv06','rabbit at linuxserv05','rabbit at linuxserv04']
 ,[],0,read_write,false,[],[],false,user_permission,[user_vhost,permission],[],[],[],{{1376,164084,168369},'rabbit at linuxserv04'},
 {{4,2},{'rabbit at linuxserv05',{1381,784616,463441}}}},
'rabbit at linuxserv01' =
{cstruct,rabbit_user_permission,set,[],['rabbit at linuxserv01'],
 [],0,read_write,false,[],[],false,user_permission,[user_vhost,permission],[],[],[],{{1381,794520,133541},'rabbit at linuxserv01'},{{2,0},[]}}

=ERROR REPORT==== 14-Oct-2013::10:57:11 ===
** Generic server mnesia_monitor terminating
** Last message in was {'EXIT',<0.46.0>,killed}
** When Server state == {state,<0.46.0>,[],[],true,[],undefined,[]}
** Reason for termination ==
** killed

=ERROR REPORT==== 14-Oct-2013::10:57:11 ===
** Generic server mnesia_recover terminating
** Last message in was {'EXIT',<0.46.0>,killed}
** When Server state == {state,<0.46.0>,undefined,undefined,undefined,0,false,
                               true,[]}
** Reason for termination ==
** killed

=ERROR REPORT==== 14-Oct-2013::10:57:11 ===
** Generic server mnesia_subscr terminating
** Last message in was {'EXIT',<0.46.0>,killed}
** When Server state == {state,<0.46.0>,20502}
** Reason for termination ==
** killed

=INFO REPORT==== 14-Oct-2013::10:57:11 ===
    application: mnesia
    exited: {shutdown,{mnesia_sup,start,[normal,[]]}}
    type: permanent

=ERROR REPORT==== 14-Oct-2013::10:57:11 ===
** Generic server mnesia_snmp_sup terminating
** Last message in was {'EXIT',<0.46.0>,killed}
** When Server state == {state,
                            {local,mnesia_snmp_sup},
                            simple_one_for_one,
                            [{child,undefined,mnesia_snmp_sup,
                                 {mnesia_snmp_hook,start,[]},
                                 transient,3000,worker,
                                 [mnesia_snmp_sup,mnesia_snmp_hook,
                                  supervisor]}],
                            undefined,0,86400000,[],mnesia_snmp_sup,[]}
** Reason for termination ==
** killed

i have tried the
rabbit at linuxserv04% rabbitmqctl -n rabbit at linuxserv01 stop_app

rabbit at linuxserv04% rabbitmqctl -n rabbit at linuxserv01 reset
it lost all Queue informations ..
rabbit at linuxserv04%
failing to join the cluster ..

linuxserv05%rabbitmq-server -detached failed to  join ..

even i tired with manually but it was failed ..

is there any other method to remove this linuxserv01 from cluster and
bring back another node into cluster (linuxserv05) ? without shutdown
entire
cluster .

rabbitmqlinuxserv04:/opt/rabbitmq$ rabbitmqctl cluster_status
Cluster status of node 'rabbitlinuxserv04' ...
[{nodes,[{disc,['rabbitlinuxserv06',
                'rabbitlinuxserv05',  -> unable to bring this node as cluster
                'rabbitlinuxserv04']},
         {ram,['rabbitlinuxserv01']}]}, -> still pointing to old node
as STATS (DB) but this node is no more in cluster ..
 {running_nodes,['rabbitlinuxserv06',
                 'rabbitlinuxserv04']}

Regards
sagu