[rabbitmq-discuss] "Dead" beam.smp threads under high load
paul.bowsher at gmail.com
Tue Oct 8 14:35:57 BST 2013
Hi Emile, thanks for the reply.
Nothing interesting in any logs, just standard. It's difficult to pinpoint
at what point these threads die (or I guess get put to sleep and never
woken up again) so we can't search the logs specifically.
We're only running self-compiled RPMs because we're on RHEL5.6 and there
aren't vendor-supplied RPMs. We've not made any modifications to Erlang or
https://www.dropbox.com/s/tcvkjd87iwre093/straces.tar.gz is a tarball of
straces for one of the nodes exhibiting this issue. The largest strace is,
unsurprisingly, the thread with the high load. I've set -s 1 to protect any
sensitive data. The output of rabbitmqctl report is available, is there
anything specific you want? It's a lot of output, and I want to be sure not
to accidentally disclose sensitive data publicly. I can send you it
directly though, PM me on Freenode if this would be useful.
etop output: https://gist.github.com/boffbowsh/116c4c2d953350111bc7
Thanks again for the help.
On Tue, Oct 8, 2013 at 1:33 PM, Emile Joubert <emile at rabbitmq.com> wrote:
> Hi Paul,
> Thanks for the detailed diagnostics.
> On 08/10/13 11:21, Paul Bowsher wrote:
> > We currently have 4 nodes in this "stuck" situation on our less-critical
> > workloads, so we are able to provide any debugging information required.
> Can you supply the output of etop? e.g.
> etop -node rabbit at rabbit-node-name -lines 100 -sort reductions
> If you can get the output of "rabbitmqctl report" that will also be useful.
> Is there anything noteworthy in the broker or system logfiles?
> Can you tell what system calls are being executed by the threads with
> high load? e.g. "strace -pid 6017"
> > We're running RabbitMQ both 3.1.0 and 3.1.5 on a self-compiled RPM
> Were any changes made to the sources?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rabbitmq-discuss