<html><head><meta http-equiv="Content-Type" content="text/html charset=windows-1252"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">Hi Michael,<div><br></div><div>Thanks very much for your response. I have a bit more data about this that could be interesting.</div><div><br></div><div>Our producer node dysfunction seems to be correlated with a heartbeat timeout event:</div><div><br></div><div><div>5/7/14 12:44:11.000 AM<span class="Apple-tab-span" style="white-space:pre"> </span></div><div>=INFO REPORT==== 7-May-2014::00:44:11 ===</div><div>accepting AMQP connection <0.17458.855> (172.20.153.34:36684 -> 172.20.153.185:5672)</div><div><br></div><div><br></div><div>5/7/14 12:44:11.000 AM<span class="Apple-tab-span" style="white-space:pre"> </span></div><div>=ERROR REPORT==== 7-May-2014::00:44:11 ===</div><div>closing AMQP connection <0.30506.852> (172.20.153.34:37769 -> 172.20.153.185:5672):</div><div>{heartbeat_timeout,running}</div><div><br></div><div><br></div><div>If we check netstat on the producer / rabbit-server sides, we see a very large Send-Q value on the producer, and nothing on the rabbit-server side; the producer Send-Q size just continues to grow larger, with no response from the server:</div><div><br></div><div>producer:</div><div><div>[ccpmer-po-cb48-p ~ ]$ netstat -an | grep 5672</div><div>tcp 0 1066985 ::ffff:172.20.153.34:36684 ::ffff:172.20.153.185:5672 ESTABLISHED </div><div>tcp 1 0 ::ffff:172.20.153.34:60075 ::ffff:172.20.149.41:15672 CLOSE_WAIT </div><div><br></div><div>rabbit side:</div><div>[ccpmer-po-cb176-p ~ ]$ netstat -an | grep 5672 | grep 34</div><div>tcp 0 0 ::ffff:172.20.153.185:5672 ::ffff:172.20.153.34:36684 ESTABLISHED </div><div><br></div><div>producer Send-Q keeps growing.</div><div>[ccpmer-po-cb48-p ~ ]$ netstat -an | grep 5672</div><div>tcp 0 1074230 ::ffff:172.20.153.34:36684 ::ffff:172.20.153.185:5672 ESTABLISHED </div><div>tcp 1 0 ::ffff:172.20.153.34:60075 ::ffff:172.20.149.41:15672 CLOSE_WAIT</div></div><div><br></div><div><br></div><div>Again we�re running the 3.2.2 version of the server - I do see that bugfix 25457 fixed a similar issue in the 3.0.3 release, but we�re past that now.</div><div><br></div><div>Is this still indicative of the socketWrite timeout issue you referred me to in your last response, or could this be something else? Is there more data you�d like to see that could help diagnose this further?</div><div><br></div><div>Are there any plans to fix <a href="https://github.com/rabbitmq/rabbitmq-java-client/issues/11">https://github.com/rabbitmq/rabbitmq-java-client/issues/11</a> soon? There�s not another more recent version of rabbit we could upgrade to that would help us, is there? </div><div><br></div><div>Thanks again,</div><div><br></div><div>Jeff</div><div><br></div><div><br></div><div><br></div><div><div>On Apr 28, 2014, at 6:31 PM, Michael Klishin <<a href="mailto:mklishin@gopivotal.com">mklishin@gopivotal.com</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">On 29 April 2014 at 02:29:30, Jeff Doto (<a href="mailto:jeffdoto@gmail.com">jeffdoto@gmail.com</a>) wrote:<br><blockquote type="cite"><blockquote type="cite">We�re wondering if anyone has encountered an issue similar <br></blockquote>to this running the versions of Rabbit server / client we are. <br>Are there any suggestions for what we can do to prevent this? Could <br>upgrading the server version help?<br></blockquote><br>This is a known Java client (or, well, JDK Socket) limitation:<br><a href="http://markmail.org/thread/w7ay4hslxwqirfsc">http://markmail.org/thread/w7ay4hslxwqirfsc</a> <br>-- <br>MK <br><br>Software Engineer, Pivotal/RabbitMQ<br></blockquote></div><br></div></body></html>