<html><body bgcolor="#FFFFFF"><div>DTrace is pretty amazing. We've been using it for troubleshooting MySQL since 2006 (OpenSolaris/OpenIndiana). Once you've used it, it's hard to live without. Strace doesn't even hold a candle. </div><div><br></div><div>-J<br><br>Sent via iPhone<div><br></div><div>Is your email Premiere?</div></div><div><br>On Sep 15, 2011, at 10:11, Alvaro Videla <<a href="mailto:videlalvaro@gmail.com">videlalvaro@gmail.com</a>> wrote:<br><br></div><div></div><blockquote type="cite"><div>I hope this DTrace branch gets merged with the default Erlang. I think is quite useful<br><br><div class="gmail_quote">On Thu, Sep 15, 2011 at 6:00 PM, Jason J. W. Williams <span dir="ltr"><<a href="mailto:jasonjwwilliams@gmail.com"><a href="mailto:jasonjwwilliams@gmail.com">jasonjwwilliams@gmail.com</a></a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;"><div bgcolor="#FFFFFF"><div>OpenIndiana and (open Solaris) are great Rabbit platforms. FreeBSD has DTrace too. DTrace for Linux isn't up to snuff with the other platforms yet, but it's still worlds ahead of SystemTap. </div>
<div><br></div><div>-J<br><br>Sent via iPhone<div><br></div><div>Is your email Premiere?</div></div><div><div></div><div class="h5"><div><br>On Sep 15, 2011, at 8:03, Alvaro Videla <<a href="mailto:videlalvaro@gmail.com" target="_blank"><a href="mailto:videlalvaro@gmail.com">videlalvaro@gmail.com</a></a>> wrote:<br>
<br></div><div></div><blockquote type="cite"><div><a href="https://twitter.com/hylomorphism/status/114000680875851776" target="_blank"></a><a href="https://twitter.com/hylomorphism/status/114000680875851776" target="_blank"></a><a href="https://twitter.com/hylomorphism/status/114000680875851776" target="_blank"></a><a href="https://twitter.com/hylomorphism/status/114000680875851776" target="_blank"></a><a href="https://twitter.com/hylomorphism/status/114000680875851776" target="_blank"></a><div>
Hi,</div><div><br></div><div>I saw Matthew tweeting "Oh RabbitMQ, why are you taking over an hour to recover 100k queues? What /are/ you doing?!" <a href="https://twitter.com/hylomorphism/status/114000680875851776" target="_blank"></a><a href="https://twitter.com/hylomorphism/status/114000680875851776" target="_blank"><a href="https://twitter.com/hylomorphism/status/114000680875851776">https://twitter.com/hylomorphism/status/114000680875851776</a></a> </div>
<div><br></div><div>I've been doing some stuff at work with DTrace and I think is pretty cool when it comes to find out what a process is actually doing.</div><div><br></div><div>At the same time there's an experimental OTP fork on Github that adds Dtrace probes to Erlang: <a href="https://github.com/dustin/otp/wiki/DTrace" target="_blank"></a><a href="https://github.com/dustin/otp/wiki/DTrace" target="_blank"><a href="https://github.com/dustin/otp/wiki/DTrace">https://github.com/dustin/otp/wiki/DTrace</a></a></div>
<div><br></div><div>See here for the available probes: <a href="https://github.com/dustin/otp/blob/dtrace/erts/emulator/beam/erlang_dtrace.d" target="_blank"></a><a href="https://github.com/dustin/otp/blob/dtrace/erts/emulator/beam/erlang_dtrace.d" target="_blank"><a href="https://github.com/dustin/otp/blob/dtrace/erts/emulator/beam/erlang_dtrace.d">https://github.com/dustin/otp/blob/dtrace/erts/emulator/beam/erlang_dtrace.d</a></a></div>
<div><br></div><div>So for example running RabbitMQ based on that Erlang branch you could do things like in this gist: <a href="https://gist.github.com/1219266" target="_blank"></a><a href="https://gist.github.com/1219266" target="_blank"><a href="https://gist.github.com/1219266">https://gist.github.com/1219266</a></a></div>
<div><br></div><div>
<span style="font-family:helvetica, arial, freesans, clean, sans-serif;font-size:11px;line-height:14px"><pre style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em;font-family:'Bitstream Vera Sans Mono', Courier, monospace"><div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:1em;line-height:1.4em"><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">erlang</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">$</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">target</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">:::</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em;font-weight:bold">function</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">-</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">entry</span></div>
<div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:1em;line-height:1.4em"><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">/</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">copyinstr</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">(</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">arg1</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">)</span> <span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">==</span> <span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em;color:rgb(221, 17, 68)">"rabbit_exchange_type_direct:route/2"</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">/</span></div>
<div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:1em;line-height:1.4em"><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">{</span></div>
<div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:1em;line-height:1.4em"> <span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">printf</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">(</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em;color:rgb(221, 17, 68)">"%s %s"</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">,</span> <span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">probemod</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">,</span> <span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">probefunc</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">);</span></div>
<div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:1em;line-height:1.4em"> <span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">self</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">-></span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">traceIt</span> <span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">=</span> <span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em;color:rgb(0, 153, 153)">1</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">;</span></div>
<div style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:1em;line-height:1.4em"><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">}</span></div>
</pre></span></div><div><br></div><div>There I enable the flag traceIt only when route/2 is called on the direct exchange.</div><div><br></div><div>From there is just a matter of adding probes to different syscalls or io access etc based using patterns on that flag like:</div>
<div><span style="font-family:'Bitstream Vera Sans Mono', Courier, monospace;font-size:12px;line-height:16px;white-space:pre-wrap"><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em"><br>
</span></span></div><div><span style="font-family:'Bitstream Vera Sans Mono', Courier, monospace;font-size:12px;line-height:16px;white-space:pre-wrap"><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">erlang</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">$</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">target</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">:::</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em;font-weight:bold">function</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">-</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">entry</span></span></div>
<div><span style="font-family:'Bitstream Vera Sans Mono', Courier, monospace;font-size:12px;line-height:16px;white-space:pre-wrap"><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">/</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">self</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">-></span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">traceIt</span><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">/</span></span></div>
<div><span style="font-family:'Bitstream Vera Sans Mono', Courier, monospace;font-size:12px;line-height:16px;white-space:pre-wrap"><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">{</span></span></div>
<div><span style="font-family:'Bitstream Vera Sans Mono', Courier, monospace;font-size:12px;line-height:16px;white-space:pre-wrap"><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em"> //do something here</span></span></div>
<div><span style="font-family:'Bitstream Vera Sans Mono', Courier, monospace;font-size:12px;line-height:16px;white-space:pre-wrap"><span style="margin-top:0px;margin-right:0px;margin-bottom:0px;margin-left:0px;padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;line-height:1.4em">}</span></span></div>
<div><br></div><div>and so on. </div><div><br></div><div>You can run the gist like: <span style="color:rgb(51, 51, 51);font-family:helvetica, arial, freesans, clean, sans-serif;font-size:12px;line-height:18px">sudo dtrace -s route.d -p `pgrep beam.smp`</span></div>
<div><br></div><div>The examples are really basic but I think they illustrate the point. With that Erlang branch you could enable more granular probes than with the usual pid provider or dtruss.</div><div><br></div><div>
BTW, this works on Mac and Solaris. AFAIK on Linux some Kernel modules are required.</div>
<div><br></div><div>Cheers,</div><div><br></div><div>Alvaro</div>
</div></blockquote></div></div><blockquote type="cite"><div><span>_______________________________________________</span><div class="im"><br><span>rabbitmq-discuss mailing list</span><br><span><a href="mailto:rabbitmq-discuss@lists.rabbitmq.com" target="_blank"><a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a></a></span><br>
<span><a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank"><a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a></a></span><br></div></div></blockquote></div></blockquote>
</div><br>
</div></blockquote></body></html>