<div dir="ltr">That was something new to try - thanks - but didn't help :)<div><br></div><div>I'm looking into this nxdomain (non-existing domain) issue - it's gotta have something to do with that. �Seems odd though... the hosts file is just fine.</div>
</div><div class="gmail_extra"><br><br><div class="gmail_quote">On 25 September 2013 11:08, Robin Lawrie - HostelBookers <span dir="ltr"><<a href="mailto:Robin.Lawrie@hostelbookers.com" target="_blank">Robin.Lawrie@hostelbookers.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-GB" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">The other issue I had in creating a cluster was that it wouldn�t create if the web management plugin was running/installed on both nodes.</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">�</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">I needed to enter the following commands on each node:</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">�</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">rabbitmq-plugins disable rabbitmq_management</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">service rabbitmq-server restart</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">�</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">Regards</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">�</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">Robin</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">�</span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:rabbitmq-discuss-bounces@lists.rabbitmq.com" target="_blank">rabbitmq-discuss-bounces@lists.rabbitmq.com</a> [mailto:<a href="mailto:rabbitmq-discuss-bounces@lists.rabbitmq.com" target="_blank">rabbitmq-discuss-bounces@lists.rabbitmq.com</a>]
<b>On Behalf Of </b>Derek Wyatt<br>
<b>Sent:</b> 25 September 2013 16:04<br>
<b>To:</b> Discussions about RabbitMQ<br>
<b>Subject:</b> Re: [rabbitmq-discuss] Clustering - just can't get it going</span></p>
<p class="MsoNormal">�</p>
<div>
<p class="MsoNormal">Damn. That's exactly the setup I have.</p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">�</p>
<div>
<p class="MsoNormal">On 25 September 2013 10:55, Robin Lawrie - HostelBookers <<a href="mailto:Robin.Lawrie@hostelbookers.com" target="_blank">Robin.Lawrie@hostelbookers.com</a>> wrote:</p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">Hi,</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">�</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">In my case, I have a 2 node cluster (called cache1 and cache2) and I needed to add an entry to the hosts file on both nodes to ensure each node can resolve
the name of the other node before clustering worked for me.</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">�</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">My hosts file is in /etc and is called hosts</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">�</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">In there I entered the following:</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">�</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">On cache1.lon.hosting, enter the line 192.168.3.1
<a href="http://Cache2.domain.com" target="_blank">Cache2.domain.com</a> Cache2</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">On cache2.lon.hosting, enter the line 192.168.3.0
<a href="http://Cache1.domain.com" target="_blank">Cache1.domain.com</a> Cache1</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">�</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">Once done, I needed to confirm I could ping each node using it�s hostname from the other node. I don�t care about DNS or nslookup working/resolving the
name.</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">�</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">HTH</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">�</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">Robin</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy">�</span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
<a href="mailto:rabbitmq-discuss-bounces@lists.rabbitmq.com" target="_blank">rabbitmq-discuss-bounces@lists.rabbitmq.com</a> [mailto:<a href="mailto:rabbitmq-discuss-bounces@lists.rabbitmq.com" target="_blank">rabbitmq-discuss-bounces@lists.rabbitmq.com</a>]
<b>On Behalf Of </b>Derek Wyatt<br>
<b>Sent:</b> 25 September 2013 15:47<br>
<b>To:</b> Discussions about RabbitMQ<br>
<b>Subject:</b> Re: [rabbitmq-discuss] Clustering - just can't get it going</span></p>
<p class="MsoNormal">�</p>
<div>
<p class="MsoNormal">Ah, I do have more information though:</p>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">DIAGNOSTICS</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">===========</span></p>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">nodes in question: ['RMQ1']</span></p>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">hosts, their running nodes and ports:</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">- unable to connect to epmd on RMQ1: nxdomain (non-existing domain)</span></p>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">current node details:</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">- node name: 'rabbitmqctl1577@RMQ2'</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">- home dir: /var/lib/rabbitmq</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">- cookie hash: ohQKEF09peb6bAgNqawvKA==</span></p>
</div>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal">And just to be clear, the cookie is the same:</p>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<div>
<p class="MsoNormal"><b><span style="font-family:"Courier New"">01</span></b><span style="font-family:"Courier New"">:~$ sudo md5sum /var/lib/rabbitmq/.erlang.cookie�</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">a2140a105d3da5e6fa6c080da9ac2f28 �/var/lib/rabbitmq/.erlang.cookie</span></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><b><span style="font-family:"Courier New"">02</span></b><span style="font-family:"Courier New"">:~$ sudo md5sum /var/lib/rabbitmq/.erlang.cookie</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">a2140a105d3da5e6fa6c080da9ac2f28 �/var/lib/rabbitmq/.erlang.cookie</span></p>
</div>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal">Somehow, telnet to epmd works just fine, but something that RMQ is doing fails to make that happen. �Is there some sort of DNS work that it's doing, instead of using the hosts files?</p>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal">i.e. one thing I found is that nslookup fails:</p>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">02:~$ nslookup RMQ1</span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">;; Got SERVFAIL reply from <ipaddress>, trying next server</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Server:������ <ipaddress></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Address:� <ipaddress></span></p>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">** server can't find RMQ1: SERVFAIL</span></p>
</div>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal">But if I ping RMQ1 it works fine. �<span style="font-family:"Courier New"">/etc/nsswitch.conf</span> specifies that files should be tried first, before DNS w.r.t. hosts.</p>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal">So, it looks like RMQ is doing something more rigorous to resolve the host, and I don't know how to change that. �I also don't have access to the DNS server configuration in order to modify it in any way.</p>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">�</p>
<div>
<p class="MsoNormal">On 25 September 2013 09:57, Jason McIntosh <<a href="mailto:mcintoshj@gmail.com" target="_blank">mcintoshj@gmail.com</a>> wrote:</p>
<div>
<p class="MsoNormal">Check your erlang cookie on both servers to make sure it matches I think it's in - /var/lib/rabbitmq/ - then you can use rabbitmqctl from one machine and see if you can connect to another to list queues. �I THINK that's rabbitmqctl
-n <servernode> list_queues for example. �If both servers can talk to each other then it should be rabbitmqctl stop_app, join_cluster, start_app.<br>
<br>
Jason</p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">�</p>
<div>
<p class="MsoNormal">On Wed, Sep 25, 2013 at 8:50 AM, Derek Wyatt <<a href="mailto:derek@derekwyatt.org" target="_blank">derek@derekwyatt.org</a>> wrote:</p>
<div>
<p class="MsoNormal">Hi,</p>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal">I've seen a number of people failing to get clustering running and, unfortunately, I can't get it going either. �Here's the summary of what I've got:</p>
</div>
<div>
<div>
<ul type="disc">
<li class="MsoNormal">Two nodes - RMQ1 and RMQ2</li><li class="MsoNormal">I can ping RMQ1 from RMQ2, and vice versa</li><li class="MsoNormal">I can telnet from RMQ1 to RMQ2:epmd, and vice versa</li><li class="MsoNormal">
I can telnet from RMQ1 to RMQ2:amqp, and vice versa</li><li class="MsoNormal">The cookie file is identical, as is clear from the startup INFO</li></ul>
<div>
<p class="MsoNormal">My goal is to have RMQ2 join RMQ1 in a cluster.</p>
</div>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal">The servers are started using the init script in Ubuntu (i.e. service rabbitmq-server start). �This is different than the script at�<a href="http://www.rabbitmq.com/clustering.html" target="_blank">http://www.rabbitmq.com/clustering.html</a>,
which says to start with "rabbitmq-server -detached". �I've tried that and it doesn't seem to make any difference so I always use the init script instead.</p>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal">So, the script says to stop the RMQ2 server and then join the cluster. �The following transcript shows how well all this goes:</p>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">02:~$ sudo rabbitmqctl stop_app</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Stopping node 'rabbit@RMQ2' ...</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">...done.</span></p>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">02:~$ sudo rabbitmqctl join_cluster --ram rabbit@RMQ1</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Clustering node 'rabbit@RMQ2' with 'rabbit@RMQ1' ...</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Error: {cannot_discover_cluster,"The nodes provided are either offline or not running"}</span></p>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Arial","sans-serif"">However, as I said above, telnetting to the ports works just fine:</span></p>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">02:~$ telnet RMQ1 epmd</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Trying <ip address>...</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Connected to RMQ1</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Escape character is '^]'.</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">booger!</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Connection closed by foreign host.</span></p>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">02:~$ telnet RMQ1 amqp</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Trying <ip address>...</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Connected to RMQ1</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">Escape character is '^]'.</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">booger!</span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Courier New"">AMQP Connection closed by foreign host.</span></p>
</div>
</div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal">I'm stuck for what else to test. �Does anyone know how to troubleshoot this thing further?</p>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
<div>
<p class="MsoNormal">Thanks,</p>
</div>
<div>
<p class="MsoNormal">Derek</p>
</div>
<div>
<p class="MsoNormal">�</p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com" target="_blank">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a></p>
</div>
<p class="MsoNormal"><span style="color:#888888"><br>
<br clear="all">
</span></p>
<div>
<p class="MsoNormal">�</p>
</div>
<p class="MsoNormal"><span style="color:#888888">-- <br>
Jason McIntosh<br>
<a href="http://mcintosh.poetshome.com/blog/" target="_blank">http://mcintosh.poetshome.com/blog/</a><br>
<a href="tel:573-424-7612" target="_blank">573-424-7612</a> </span></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com" target="_blank">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a></p>
</div>
<p class="MsoNormal">�</p>
</div>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
<p>This email is from Hostelbookers.com Limited. Registered office: 52-54 High Holborn, London, WC1V 6RL, UK. Registered in England under Company No.: 2841908. This email and any files transmitted with it are confidential and may be privileged and are intended
solely for the use of the individual or entity to whom they are addressed. As email can be subject to operational or technical difficulties and time delays, communications that are subject to deadlines should also be sent by post. Any unauthorised direct or
indirect dissemination, distribution or copying of this message and any attachments is strictly prohibited. If you have received the email in error, please notify
<a href="mailto:postmaster@hostelbookers.com" target="_blank">postmaster@hostelbookers.com</a></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com" target="_blank">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a></p>
</div>
<p class="MsoNormal">�</p>
</div>
</div>
<hr>
<p>This email is from Hostelbookers.com Limited. Registered office: 52-54 High Holborn, London, WC1V 6RL, UK. Registered in England under Company No.: 2841908. This email and any files transmitted with it are confidential and may be privileged and are intended
solely for the use of the individual or entity to whom they are addressed. As email can be subject to operational or technical difficulties and time delays, communications that are subject to deadlines should also be sent by post. Any unauthorised direct or
indirect dissemination, distribution or copying of this message and any attachments is strictly prohibited. If you have received the email in error, please notify <a href="mailto:postmaster@hostelbookers.com" target="_blank">postmaster@hostelbookers.com</a></p>
<hr>
</div>
<br>_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
<br></blockquote></div><br></div>