<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN">
<html><body>
<p>Hi,</p>
<p> </p>
<p>I Started again with a fresh installation of 2.7.1.<br /><br />There remain errors when I request it to join a cluster. <br /><br />There is nothing recorded in /var/log/rabbitmq/shutdown_err.<br /><br />I have replaced the server names with Node1 and Node2 respectivly to make it easier to follow.<br /><br />Any ideas?<br /><br /><br />***************************<br />**** ** Attempt # 1 ** **** <br />***************************</p>
<p>Rpm -e rabbit-server. Remove the content of /var/lib/rabbitmq and /etc/rabbitmq,. and rpm -i rabbit-server....rpm</p>
<p>**** Start Node1 ****<br />[root@Node1 ~]# /etc/init.d/rabbitmq-server start<br />Starting rabbitmq-server: SUCCESS<br />rabbitmq-server.<br /><br />** Verify that the server is really running.<br />[root@Node1 ~]# ps -aef|grep ra<br />root 3 2 0 Jun27 ? 00:00:00 [migration/0]<br />root 5 2 0 Jun27 ? 00:00:00 [migration/0]<br />rabbitmq 1710 1 0 Jun27 ? 00:00:00 /usr/lib64/erlang/erts-5.8.5/bin/epmd -daemon<br />root 5122 1 0 11:13 ? 00:00:00 /bin/sh /usr/sbin/rabbitmq-server<br />root 5133 5122 0 11:13 ? 00:00:00 su rabbitmq -s /bin/sh -c /usr/lib/rabbitmq/bin/rabbitmq-server <br />rabbitmq 5136 5133 3 11:13 ? 00:00:00 /usr/lib64/erlang/erts-5.8.5/bin/beam -W w -K true -A30 -P 1048576 -- -root /usr/lib64/erlang -progname erl -- -home /var/lib/rabbitmq -- -noshell -noinput -sname rabbit@Node1 -boot /var/lib/rabbitmq/mnesia/rabbit@Node1-plugins-expand/rabbit -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -sasl sasl_error_logger false -rabbit error_logger {file,"/var/log/rabbitmq/rabbit@Node1.log"} -rabbit sasl_error_logger {file,"/var/log/rabbitmq/rabbit@Node1-sasl.log"} -os_mon start_cpu_sup true -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@Node1"<br />rabbitmq 5216 5136 0 11:13 ? 00:00:00 /usr/lib64/erlang/lib/os_mon-2.2.7/priv/bin/cpu_sup<br />rabbitmq 5217 5136 0 11:13 ? 00:00:00 inet_gethost 4<br />rabbitmq 5218 5217 0 11:13 ? 00:00:00 inet_gethost 4<br /><br />** Server is listening<br />[root@Node1 ~]# lsof -P -i tcp |grep rabbit|grep LIST<br />epmd 1710 rabbitmq 3u IPv4 10926 0t0 TCP *:4369 (LISTEN)<br />beam 5136 rabbitmq 7u IPv4 27914 0t0 TCP *:45343 (LISTEN)<br />beam 5136 rabbitmq 17u IPv6 27951 0t0 TCP *:5672 (LISTEN)<br />beam 5136 rabbitmq 19r IPv4 27984 0t0 TCP *:55672 (LISTEN)<br /><br /><br /><br />**** Start Node2 ****<br />[root@Node2 ~]# /etc/init.d/rabbitmq-server start<br />Starting rabbitmq-server: SUCCESS<br />rabbitmq-server.<br /><br /><br />** Verify that it is running<br />-bash-4.1$ ps -eaf|grep rabbit|grep -v grep<br />rabbitmq 2172 1 0 Jun27 ? 00:00:00 /usr/lib64/erlang/erts-5.8.5/bin/epmd -daemon<br />root 5245 1 0 11:12 ? 00:00:00 /bin/sh /usr/sbin/rabbitmq-server<br />root 5257 5245 0 11:12 ? 00:00:00 su rabbitmq -s /bin/sh -c /usr/lib/rabbitmq/bin/rabbitmq-server <br />rabbitmq 5260 5257 0 11:12 ? 00:00:00 /usr/lib64/erlang/erts-5.8.5/bin/beam -W w -K true -A30 -P 1048576 -- -root /usr/lib64/erlang -progname erl -- -home /var/lib/rabbitmq -- -noshell -noinput -sname rabbit@iup-app008 -boot /var/lib/rabbitmq/mnesia/rabbit@iup-app008-plugins-expand/rabbit -kernel inet_default_connect_options [{nodelay,true}] -sasl errlog_type error -sasl sasl_error_logger false -rabbit error_logger {file,"/var/log/rabbitmq/rabbit@iup-app008.log"} -rabbit sasl_error_logger {file,"/var/log/rabbitmq/rabbit@iup-app008-sasl.log"} -os_mon start_cpu_sup true -os_mon start_disksup false -os_mon start_memsup false -mnesia dir "/var/lib/rabbitmq/mnesia/rabbit@iup-app008"<br />rabbitmq 5340 5260 0 11:13 ? 00:00:00 inet_gethost 4<br />rabbitmq 5341 5340 0 11:13 ? 00:00:00 inet_gethost 4<br />rabbitmq 5346 5345 0 11:13 pts/0 00:00:00 -bash<br /><br />** Server is listening<br />[root@Node2 ~]# lsof -P -i tcp |grep rabbit|grep LIST<br />epmd 2172 rabbitmq 3u IPv4 11583 0t0 TCP *:4369 (LISTEN)<br />beam 5260 rabbitmq 7u IPv4 164034 0t0 TCP *:51764 (LISTEN)<br /><br /><br />** Attempt to cluster Node2 with Node1 as rabbitmq user<br />-bash-4.1$ rabbitmqctl stop_app<br />Stopping node 'rabbit@Node2' ...<br />...done.<br />-bash-4.1$ rabbitmqctl reset<br />Resetting node 'rabbit@Node2' ...<br />...done.<br />-bash-4.1$ rabbitmqctl cluster rabbit@Node1 rabbit@Node2<br />Clustering node 'rabbit@Node2' with ['rabbit@Node1',<br /> 'rabbit@Node2'] ...<br />Error: {no_running_cluster_nodes,['rabbit@Node1'],['rabbit@Node1']}<br />-bash-4.1$ <br /><br /><br />***************************<br />**** ** Attempt # 2 ** **** <br />***************************<br />I tried once more from scratch, and thought I would get away with a shortcut... Fail:</p>
<p><br />Rpm -e rabbit-server. Remove the content of /var/lib/rabbitmq and /etc/rabbitmq,. </p>
<p>Deleted all the directories that rpm -e did not remove ( Debian's purge is far nicer) and rebooted both of the RedHat servers. These are virtual boxes & the reboot cycle is about 2 mins.<br /><br />After the servers came back I did this:</p>
<p># rpm -i rabbit-server....rpm</p>
<p>Create and add this line into /etc/rabbitmq/rabbitmq.config:<br /> [{rabbit, [{cluster_nodes, ['rabbit@Node1', 'rabbit@Node2']}]}].<br /><br />Node1: /etc/init.d/rabbitmq-server start<br />Starting rabbitmq-server: SUCCESS<br />rabbitmq-server<br /><br />Copy .erland.cookie to Node2<br />-r-------- 1 rabbitmq rabbitmq 20 Jun 28 11:35 .erlang.cookie<br /><br />Node1: /etc/init.d/rabbitmq-server start<br /># /etc/init.d/rabbitmq-server start<br />Starting rabbitmq-server: FAILED - check /var/log/rabbitmq/startup_{log, _err}<br /><br /># cat /var/log/rabbitmq/startup_err<br />Erlang has closed<br /><br />Crash dump was written to: erl_crash.dump<br />Kernel pid terminated (application_controller) ({application_start_failure,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot}}}}})<br /><br /><br /><br /></p>
<p>On 27.06.2012 15:35, Simon Loewenthal wrote:</p>
<blockquote type="cite" style="padding-left:5px; border-left:#1010ff 2px solid; margin-left:5px; width:100%"><!-- html ignored --><!-- head ignored --><!-- meta ignored -->
<div class="moz-cite-prefix">Nope, I'm not reissuing the commands. <br /><br /> One part I omitted was that there was an previous install of Rabbit on the server, but the rpm was removed, and an rm -r /var/lib/rabbitmq/* was done. The .erlang.cookie was also removed, and I double checked that the mnesia files were definitely removed afore I did anything. <br /><br /><br /><br /> On 27/06/12 15:31, Simon MacMullen wrote:</div>
<blockquote type="cite" style="padding-left:5px; border-left:#1010ff 2px solid; margin-left:5px; width:100%">By "reclustering" I just mean reissuing "rabbitmqctl reset" "rabbitmqctl cluster ..." commands, and thus redefining the cluster. <br /><br /> Alternatively, could something be removing files from /var/lib/rabbitmq/mnesia/ ? <br /><br /> Cheers, Simon <br /><br /> On 27/06/12 14:24, Simon Loewenthal wrote: <br />
<blockquote type="cite" style="padding-left:5px; border-left:#1010ff 2px solid; margin-left:5px; width:100%">Hi Simon, <br /><br /> I rebooted both servers : /# reboot/ <br /> You've given a good reason to move to 2.8.n <br /><br /> > Are you definitely not reclustering at any stage? <br /> Apologies, but I don't understand the term reclustering. <br /><br /> S. <br /><br /><br /><br /> On 27/06/12 15:20, Simon MacMullen wrote: <br />
<blockquote type="cite" style="padding-left:5px; border-left:#1010ff 2px solid; margin-left:5px; width:100%">Hi. <br /><br /> You say you "reboot the server" - is this node 1, node 2, or both? <br /><br /> The description you have after rebooting the server is consistent with <br /> having one RAM node and one disc node and restarting both of them - in <br /> 2.7.0 if the RAM node came back up first it would create a blank <br /> database. This was fixed in 2.8.x, in that now a RAM node will refuse <br /> to start if there are no disc nodes to connect to. <br /><br /> However, when you initially create the cluster you are creating both <br /> nodes as disc nodes. So I am puzzled as to how you could have got to <br /> the state where one node was a RAM node. Are you definitely not <br /> reclustering at any stage? <br /><br /> Cheers, Simon <br /><br /> On 27/06/12 12:48, S.Loewenthal wrote: <br />
<blockquote type="cite" style="padding-left:5px; border-left:#1010ff 2px solid; margin-left:5px; width:100%">Hi there, <br /><br /> I installed RabbitMQ server but cannot get one node to retain the <br /> cluster configuration. Step-by-step account follows for what I did to <br /> configure these nodes. <br /> Node 1: rabbit@iuu-7 <br /> Nide 2: rabbit@iuu-8 <br /><br /><br /> On both nodes install 2.7.0 - This version as all other environment run <br /> same version. <br /> # rpm -ivh rabbitmq-server-2.7.0-1.noarch.rpm <br /><br /> Start Node 2 (if not already running) <br /> # /etc/init.d/rabbitmq-server start <br /><br /> Shutdown RabbitMQ on Node 2 <br /> # /etc/init.d/rabbitmq-server stop <br /><br /> Copied the Erlang cookie so that both nodes have the same cookie, or <br /> else these won't talk with each other <br /> Copy (Node 1) iuu-7:/var/lib/rabbitmq/.erlang.cookie to (Node 2) <br /> iuu-8:/var/lib/rabbitmq/.erlang.cookie <br /> permissions and ownership are : Perms 400 -- Owner rabbitmq:rabbitmq <br /><br /> Start RabbitMQ on Node 2 <br /> # /etc/init.d/rabbitmq-server start <br /><br /> Verify that each node can communicate with each other: <br /> On Node 1 <br /> $ rabbitmqctl -n rabbit@iuu-8 status <br /> On Node 2 <br /> $ rabbitmqctl -n rabbit@iuu-7 status <br /><br /> Reset Node 2 so that it is ready to join the cluster on Node 1 <br /> $ rabbitmqctl stop_app <br /> $ rabbitmqctl reset <br /><br /> Add the Node 2 to the first node with disc writing enabled on both <br /> nodes: <br /> $ rabbitmqctl cluster rabbit@iuu-7 rabbit@iuu-8 <br /><br /> Start the app on Node 2 <br /> $ rabbitmqctl start_app <br /><br /> Verify the cluster status on Node 2 and the same on Node 1 <br /> $ rabbitmqctl cluster_status <br /> Cluster status of node 'rabbit@iuu-8' ... <br /> [{nodes,[{disc,['rabbit@iuu-8','rabbit@iuu-7']}]}, <br /> {running_nodes,['rabbit@iuu-7','rabbit@iuu-8']}] <br /> ...done. <br /><br /> next I add a few users, and reboot the server. Now the cluster is <br /> disconnection, and I cannot get ot to connnect. Also, the users added <br /> afore the reboot no longer exist. <br /><br /> It comes up without the cluster information nor the users: <br /> [root@iuu-7 ~]# rabbitmqctl cluster_status <br /> Cluster status of node 'rabbit@iup-7' ... <br /> [{nodes,[{ram,['rabbit@iuu-7']}]},{running_nodes,['rabbit@iuu-7']}] <br /> ...done. <br /><br /> [root@iup-8 ~]# rabbitmqctl cluster_status <br /> Cluster status of node 'rabbit@iup-' ... <br /> [{nodes,[{disc,['rabbit@iup-8']},{ram,['rabbit@iup-7']}]}, <br /> {running_nodes,['rabbit@iup-8']}] <br /> ...done. <br /><br /> The cluster configuration does not survive a server restart and in this <br /> case user account was was lost. I imagine that I have misconfigured <br /> something. <br /><br /> ** The question ** <br /> How can I ensure that the cluster reconnects after rabbitmq-server <br /> restarts, and that data that was written is retained? Or, what did I <br /> misconfigure <br /><br /><br /><br /> Many thanks, S. <br /><br /><br /> _______________________________________________ <br /> rabbitmq-discuss mailing list <br /><a class="moz-txt-link-abbreviated" href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a> <br /><a class="moz-txt-link-freetext" href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a> </blockquote>
<br /><br /></blockquote>
<br /><br /></blockquote>
<br /><br /></blockquote>
<br /><br />
<pre class="moz-signature">--
PGP is optional: 4BA78604
simon @ klunky . org
simon @ klunky . co.uk
I won't accept your confidentiality
agreement, and your Emails are kept.
~Ö¿Ö~
</pre>
</blockquote>
<div>
<pre>--
--
Dogs are tough.
I've been interrogating this one for hours
and he still won't tell me who's a good boy.
simon@klunky.co.uk www.klunky.org
--</pre>
</div>
</body></html>