<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thank you!!<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D">Thara Vadakkeveedu | Performance Engineer |
</span></b><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#F15D22">Kronos Incorporated</span></b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D"><br>
tel: +1 978 947 4123<br>
Kronos | Time & Attendance • Scheduling • Absence Management • HR & Payroll • Hiring • Labor Analytics<b><o:p></o:p></b></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D">Join Kronos on:
<a href="http://www.kronos.com/">kronos.com</a> | <a href="http://www.kronos.com/facebook">
<span style="color:#0011FF">Facebook</span></a></span></b><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#0011FF">
</span></b><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D">|</span></b><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#0D396B">
<a href="http://www.kronos.com/twitter"><span style="color:#0011FF">Twitter</span></a></span></b><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#0011FF">
</span></b><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D">|</span></b><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#0011FF">
<a href="http://www.kronos.com/linkedin"><span style="color:#0011FF">LinkedIn</span></a></span></b><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D"> |</span></b><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#0011FF">
<a href="http://www.kronos.com/youtube"><span style="color:#0011FF">YouTube</span></a></span></b><b><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#D63F20"><o:p></o:p></span></b></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> rabbitmq-discuss [mailto:rabbitmq-discuss-bounces@lists.rabbitmq.com]
<b>On Behalf Of </b>Gary Russell<br>
<b>Sent:</b> Thursday, February 06, 2014 1:22 PM<br>
<b>To:</b> Discussions about RabbitMQ<br>
<b>Subject:</b> Re: [rabbitmq-discuss] multiple message listeners on a single reply queue<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">You cannot share reply queues across instances; unlike JMS, AMQP has no notion of a message selector to choose messages from a queue, so each instance needs its own reply queue.<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">When the reply goes to the wrong instance, the originating instance thread will be suspended for (default) 5 seconds (modifiable by setting
<span style="font-family:"Courier New"">replyTimeout</span>).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">You can increase throughput by increasing the <span style="font-family:"Courier New"">
concurrency</span> on the listener containers.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The upcoming 1.3 release of spring-AMQP allows the concurrency to vary on demand; with 1.2.x you have to stop/start the container to change the concurrency.<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Thu, Feb 6, 2014 at 1:07 PM, tgv amni <<a href="mailto:tgvt53@gmail.com" target="_blank">tgvt53@gmail.com</a>> wrote:<o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">We are beginners in rabbitmq. We use rabbitmq 3.2.2. We use spring amqp.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I recently observed an issue with multiple listeners on the same reply queue when performance testing <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">our application.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Here is the test scenario:<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">50 Concurrent users post a request to Service F to create an entity A. The message contains a user id and a timestamp. Each concurrent user posts roughly two messages per second.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">F publishes this message to a non-durable queue via a topic exchange (using convertSendAndReceive) and waits for the response on a fixed reply queue ReplyQ1.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Service A picks up this message and creates this entity (basically an entry in a table) after<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">validating that the user id wihin that message is valid.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">For the validation, Service A sends this message to another non-durable queue via a topic exchange (convertSendAndReceive) and waits for a response on <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">another fixed reply queue, ReplyQ2. <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">A validating service V picks up this user id, validates it (basically a select from a db table) and then sends the response back to ReplyQ2.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Service A picks this up, based on the message, creates the entity and then sends response to ReplyQ1. <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">All messages are non-persistent. No concurrency or prefetch settings configured on the listeners. <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>CreateEntity A Request -> Service F -> Q1 -> Service A -> Q2 - Service V</b><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>Service V-> ReplyQ2 -> Service A (creates entity A if user valid) -> ReplyQ1 -> Service F</b><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">F, A and V all live within the same tomcat instance. When a second tomcat instance was started with service A and V, I started to notice warning messages in tomcat logs:<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>WARN : org.springframework.amqp.rabbit.core.RabbitTemplate - Reply received after timeout for b3fd4f57-0403-4704-8f16-8c3055ca0cd9</b><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">It looked like messages were not picked up by the correct listener from the validation response queues. Rabbitmq logs did not show any errors. From the management console i saw
40-60 messages always staying in ready state in Q1, with a few in unacked state.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Response times were very high for this create Entity A request (10 seconds) and only about 5 transactions were processed per second. Tests were run in AWS on m1.large instances.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">High response time was not caused by network latency. Network latency was really small. <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On testing with separate reply queues for the user id validation response, <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">(ReplyQ21 and ReplyQ22 instead of ReplyQ2), response times improved (1 second) and more transactions (roughly 70) were processed per second. <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Is this a good design approach for this scenario? Or is there a better way to handle this ?<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Please let me know if you need any other information.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thanks.<o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</body>
</html>