[rabbitmq-discuss] Measuring end-to-end message delivery time

[dburgan at peopleanswers.com]

Firstly, kudos to the RabbitMQ team for an astonishingly good product. We 
are using it as the messaging bus backbone for our flagship application and 
have had literally zero problems with it. Can't say enough good things.

I'm hoping someone can help me figure out a solution for this. When we send 
a message, it generally goes through a pipeline something like this:

1. (Sending JVM) A Java thread will push the message into a 
ThreadPoolExecutor queue.
2. (Sending JVM) The ThreadPoolExecutor sends the message to RabbitMQ.
3. RabbitMQ delivers the message to the consumer.
4. (Receiving JVM) The consumer receives the message and pushes it into a 
ThreadPoolExecutor queue
5. (Receiving JVM) The ThreadPoolExecutor handles the message

My goal is to measure the total time spent, per message, with millisecond 
accuracy, from one end of this pipeline to the other. I want to be able to 
calculate this for every message sent through our message bus, so then I 
can do analysis on those times, in real-time, and draw useful conclusions.

The trouble is that the part pre-RabbitMQ (1-2) could be in a different JVM 
on a different physical server than the part post-RabbitMQ (4-5). So I can 
measure the time for steps 1, 2, 4, and 5. But how do I measure the time it 
took for the message to actually travel through RabbitMQ (step 3)?

I can't just time-stamp the message because the local time on the two 
physical servers cannot be guaranteed to be synchronized to the 
millisecond. We use NTP but it does not keep the two servers that closely 
synchronized.

So my question is: does RabbitMQ offer anything I can use that will allow 
me to figure out how long step 4 took for each message I send through 
RabbitMQ, or at least approximate it closely? Or is there some other clever 
technoloogy I could use to accomplish the same thing?

Thanks in advance,
Darrell Burgan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20121222/d0c5f8dc/attachment.htm>