<div dir="ltr"><div><div>Here's the output of the command around (within 10s) when it segfaults. I've truncated a bunch of similar lines, since there are more than 25k lines.<br><br>USER CMD PID FD MOUNT INUM MODE SZ|DV R/W<br>
brandon beam.smp 65462 text / 17977559 -r-xr-xr-x 2397040 r<br>brandon beam.smp 65462 ctty /dev 89 crw--w---- pts/0 rw<br>brandon beam.smp 65462 wd / 15810621 drwxr-xr-x 1024 r<br>
brandon beam.smp 65462 root / 2 drwxr-xr-x 1024 r<br>brandon beam.smp 65462 0 /dev 89 crw--w---- pts/0 rw<br>brandon beam.smp 65462 1 /dev 89 crw--w---- pts/0 rw<br>brandon beam.smp 65462 2 /dev 89 crw--w---- pts/0 rw<br>
brandon beam.smp 65462 3<br>brandon beam.smp 65462 4* pipe fffff8000a978000 <-> fffff8000a978160 0 rw<br>brandon beam.smp 65462 5* pipe fffff8000a978160 <-> fffff8000a978000 0 rw<br>
brandon beam.smp 65462 6* pipe fffff8008b5b22e8 <-> fffff8008b5b2448 0 rw<br>brandon beam.smp 65462 7* pipe fffff8008b5b2448 <-> fffff8008b5b22e8 0 rw<br>brandon beam.smp 65462 8* internet stream tcp fffff8027d821000<br>
brandon beam.smp 65462 9* internet stream tcp fffff80566596800<br>brandon beam.smp 65462 10 - 15810857 -rw-r--r-- 3093764 w<br>brandon beam.smp 65462 11 - 15810858 -rw-r--r-- 0 w<br>
brandon beam.smp 65462 12 - 15891126 -rw-r--r-- 96 rw<br>brandon beam.smp 65462 13 - 15891137 -rw-r--r-- 0 rw<br>brandon beam.smp 65462 14 - 15891139 -rw-r--r-- 0 rw<br>
brandon beam.smp 65462 15* internet6 stream tcp fffff80458487800<br>brandon beam.smp 65462 16* pipe fffff803c3f4b000 <-> fffff803c3f4b160 0 rw<br>brandon beam.smp 65462 17* internet stream tcp fffff80458449400<br>
brandon beam.smp 65462 18* internet stream tcp fffff802be9b4c00<br>brandon beam.smp 65462 19* pipe fffff8000a9bca18 <-> fffff8000a9bc8b8 0 rw<br>brandon beam.smp 65462 20* internet stream tcp fffff802bee95400<br>
brandon beam.smp 65462 21* internet stream tcp fffff802be4ad400<br>brandon beam.smp 65462 22* pipe fffff801ea5f52e8 <-> fffff801ea5f5448 0 rw<br>brandon beam.smp 65462 23* internet stream tcp fffff804edad0800<br>
brandon beam.smp 65462 24* internet stream tcp fffff80249ff8400<br>brandon beam.smp 65462 25* internet stream tcp fffff80566026400<br>...<br>brandon beam.smp 65462 25018* internet stream tcp fffff8045848d000<br>
brandon beam.smp 65462 25019* internet stream tcp fffff805662fb400<br>brandon beam.smp 65462 25020* internet stream tcp fffff804a4633400<br>brandon beam.smp 65462 25021* internet stream tcp fffff802495a8000<br>brandon beam.smp 65462 25024* pipe fffff8049522a160 <-> fffff8049522a000 0 rw<br>
<br></div>Let me know if you need anything else, or if you spot anything that I can do to mitigate the issue.<br><br></div>-Brandon<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Apr 18, 2014 at 8:16 AM, Matthias Radestock <span dir="ltr"><<a href="mailto:matthias@rabbitmq.com" target="_blank">matthias@rabbitmq.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Brandon,<div class=""><br>
<br>
On 15/04/14 17:25, Brandon Munroe wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
The message in the logs reads:<br>
<br>
** Generic server rabbit_mgmt_external_stats terminating<br>
** Last message in was emit_update<br>
** When Server state == {state,707130}<br>
** Reason for termination ==<br>
** {function_clause,[{lists,nth,[<u></u>1,[]],[{file,"lists.erl"},{<u></u>line,168}]},<br>
<br>
{rabbit_mgmt_external_stats,'-<u></u>get_used_fd/1-fun-1-',2,[]},<br>
{lists,'-filter/2-lc$^0/1-0-',<u></u>2,<br>
[{file,"lists.erl"},{line,<u></u>1271}]},<br>
{lists,'-filter/2-lc$^0/1-0-',<u></u>2,<br>
[{file,"lists.erl"},{line,<u></u>1271}]},<br>
{rabbit_mgmt_external_stats,<u></u>get_used_fd,1,[]},<br>
<br>
{rabbit_mgmt_external_stats,'-<u></u>infos/2-lc$^0/1-0-',2,[]},<br>
<br>
{rabbit_mgmt_external_stats,'-<u></u>infos/2-lc$^0/1-0-',2,[]},<br>
{rabbit_mgmt_external_stats,<u></u>emit_update,1,[]}]}<br>
</blockquote>
<br></div>
On BSD RabbitMQ invokes 'fstat -p <pid>' to determine the number of used file descriptors. In the above it is tripping over the parsing of the output. If you can, please post the output of that command just before rabbit dies. Meanwhile I've filed a bug to make our parsing code more robust.<span class="HOEnZb"><font color="#888888"><br>
<br>
<br>
Matthias.<br>
<br>
</font></span></blockquote></div><br></div>