<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div>On 19 Jul 2012, at 02:15, mrajkovic wrote:</div><blockquote type="cite"><meta http-equiv="Content-Type" content="text/html; charset=utf-8"><meta name="Generator" content="Microsoft Word 14 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--><div class="WordSection1"><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">Hi Tim,<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4"><o:p> </o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">Sorry it was 1am locally here when we were emailing last night, and I needed to getup for an early morning appointment�.<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4"></span></p></div></blockquote><div><br></div>Sleeping is pretty important, so fair enough! :)<br><blockquote type="cite"><div class="WordSection1"><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4"><o:p> </o:p></span></p><div><br></div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">telnet has blocked it on that port�<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">root@mf-01:~# telnet mf-02 4369<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">Trying 119.31.227.106...<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">telnet: Unable to connect to remote host: Connection refused<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4"></span></p></div></blockquote><div><br></div><div>That's more like what we'd expect, given that epmd isn't listening other than on localhost.</div><br><blockquote type="cite"><div class="WordSection1"><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4"><o:p> </o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">then I ran netstat on mf-02<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">root@mf-02:~# netstat -p -l --numeric-ports<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">Active Internet connections (only servers)<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">tcp 0 0 0.0.0.0:3306 0.0.0.0:* LISTEN 995/mysqld<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">tcp 0 0 0.0.0.0:10000 0.0.0.0:* LISTEN 2028/perl<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 1982/apache2<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">tcp 0 0 127.0.0.1:4369 0.0.0.0:* LISTEN 1868/epmd</span></p></div></blockquote><div><br></div><div>So only listening for local traffic. That's not what we should be seeing:</div><div><br></div><div><div>iske:~ root# netstat -a | grep LISTEN</div><div>tcp4 0 0 *.epmd *.* LISTEN </div><div>tcp4 0 0 *.62693 *.* LISTEN </div><div>tcp4 0 0 localhost.ipp *.* LISTEN </div><div>tcp6 0 0 localhost.ipp *.* LISTEN </div><div>iske:~ root# </div></div><br><blockquote type="cite"><div class="WordSection1"><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4"><o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">tcp 0 0 0.0.0.0:53 0.0.0.0:* LISTEN 935/dnsmasq<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">tcp 0 0 0.0.0.0:21 0.0.0.0:* LISTEN 764/vsftpd<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 800/sshd<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">tcp 0 0 0.0.0.0:25 0.0.0.0:* LISTEN 1755/master<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">tcp 0 0 0.0.0.0:39616 0.0.0.0:* LISTEN 26602/beam.smp<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">tcp 0 0 119.31.227.106:5672 0.0.0.0:* LISTEN 26602/beam.smp</span></p></div></blockquote><div><br></div><div>And clearly not all programs are having this problem. Indeed beam is running and listening happily enough.</div><br><blockquote type="cite"><div class="WordSection1"><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4"><o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">tcp6 0 0 :::53 :::* LISTEN 935/dnsmasq<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">tcp6 0 0 :::22 :::* LISTEN 800/sshd<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">tcp6 0 0 :::25 :::* LISTEN 1755/master<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">udp 0 0 0.0.0.0:53 0.0.0.0:* 935/dnsmasq<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">udp 0 0 0.0.0.0:43242 0.0.0.0:* 1613/dccifd<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">udp 0 0 0.0.0.0:10000 0.0.0.0:* 2028/perl<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">udp6 0 0 :::53 :::* 935/dnsmasq<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4"><o:p> </o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4">it looks like its only listening on localhost and not all ports<o:p></o:p></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#548DD4"></span></p></div></blockquote><div>Clearly. The environment variables that govern epmd's behaviour (taken from <a href="http://www.erlang.org/doc/man/epmd.html">http://www.erlang.org/doc/man/epmd.html</a>) are:</div><div><br></div><div><dt><strong><span class="code">ERL_EPMD_ADDRESS</span></strong></dt>
<dd><p>This environment variable may be set to a comma-separated
list of IP addresses, in which case the <span class="code">epmd</span> daemon
will listen only on the specified address(es) and on the
loopback address (which is implicitly added to the list if it
has not been specified). The default behaviour is to listen on
all available IP addresses.</p>
</dd>
<dt>And in terms of access restrictions, the same source gives us: </dt><dt><br></dt><dt><quote></dt><dt><span class="Apple-style-span" style="font-size: 14px; font-weight: bold; "><a name="id152489">Access restrictions</a></span></dt><div>
<div class="REFBODY"><p>The <span class="code">epmd</span> daemon accepts messages from both localhost and
remote hosts. However, only the query commands are answered (and
acted upon) if the query comes from a remote host. It is always an
error to try to register a nodename if the client is not a process
located on the same host as the <span class="code">epmd</span> instance is running on,
why such requests are considered hostile and the connection is
immediately closed.</p><p>The queries accepted from remote nodes are:</p>
<ul>
<li><p>Port queries - i.e. on which port does the node with a given
name listen</p>
</li>
<li><p>Name listing - i.e. give a list of all names registered on
the host</p>
</li>
</ul><p>To restrict access further, firewall software has to be used.</p><p></quote></p></div></div></div><div>Finally, you can control the list of IP addresses epmd will listen on using the `-address List` flags on startup. Rabbit does not do any of these things, your environment config file isn't doing this and so I'm left a little confused. For the beam (emulator) you can set a parameter in the kernel application (inet_dist_use_interface) to control this, but you're not doing that AFAICT - you don't have any other config files (e.g., in wherever RABBITMQ_CONFIG_FILE is pointing) do you? /etc/rabbitmq can contain a rabbitmq.config file as well as the environment config file you posted originally. Is there anything in there? Even so, inet_dist_use_interface doesn't actually affect what addresses epmd will LISTEN on anyway AFAICT.</div><div><br></div><div>Is it possible that some other software in your environment could be starting epmd with -address *before* the rabbit nodes are started up? So really I'm at a bit of a loss here. Could you add to the mix by letting me know what Erlang/OTP version you've got installed, where it came from (i.e., apt, compiled from source, etc)? </div><div><div><br></div><div>Having looked at the epmd source code, if there are no explicit addresses passed to the server then epmd uses INADDR_ANY, which listens on all interfaces (on most linux variants anyway). Unless you can find something in the configuration and/or environment, or there is something going on that causes INADDR_ANY to only bind to localhost (the result of muti-homing or some such) then I don't know what's going on with this one. Unless someone else has any light to shed, or you're able to locate some extraneous config/environment then we might have to take a wander over to the erlang-questions mailing list and ask there, though I will need you to come along and ask the question initially as I'm not the one who's able to replicate this behaviour.</div><div><br></div><div>One last question. You're not trying to run rabbit in a restricted environment (i.e., in a jail) are you? It could just be possible that security permissions are restricting the available addresses you can bind to. </div><div><br></div><div>Tim</div></div></div><br></body></html>