<div dir="ltr"><div><div><div><div><div><div><div>Hello,<br><br></div>I tried to search some discussion on this topic but didn't find.<br><br></div>The
problem : the client sends some messages to the server on another host
and at some moment server host becomes unreachable (network is down,
host is powered off, etc). <br>
</div>As a result the sender thread is blocked while writing to socket
and recognizes network failure after some time which is not comfortable
(> 15 min for our rhel machine). I tried to tune some tcp/socket
settings but didn't succeed.<br>
<br>Here is a part of thread dump when the thread is stuck:<br><br>"main" prio=10 tid=0xf7505800 nid=0x2c3e runnable [0xf7728000]<br> java.lang.Thread.State: RUNNABLE<br> at java.net.SocketOutputStream.socketWrite0(Native Method)<br>
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)<br> at java.net.SocketOutputStream.write(SocketOutputStream.java:153)<br> at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)<br>
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)<br> - locked <0xb4532608> (a java.io.BufferedOutputStream)<br> at java.io.DataOutputStream.flush(DataOutputStream.java:123)<br>
at com.rabbitmq.client.impl.SocketFrameHandler.flush(SocketFrameHandler.java:142)<br> at com.rabbitmq.client.impl.AMQConnection.flush(AMQConnection.java:488)<br> at com.rabbitmq.client.impl.AMQCommand.transmit(AMQCommand.java:125)<br>
at com.rabbitmq.client.impl.AMQChannel.quiescingTransmit(AMQChannel.java:316)<br> - locked <0xb4532ee8> (a java.lang.Object)<br> at com.rabbitmq.client.impl.AMQChannel.transmit(AMQChannel.java:292)<br>
- locked <0xb4532ee8> (a java.lang.Object)<br> at com.rabbitmq.client.impl.ChannelN.basicPublish(ChannelN.java:634)<br> at com.rabbitmq.client.impl.ChannelN.basicPublish(ChannelN.java:617)<br>
at com.rabbitmq.client.impl.ChannelN.basicPublish(ChannelN.java:608)<br>
<br>"AMQP Connection <a href="http://172.17.32.170:5672" target="_blank">172.17.32.170:5672</a>" prio=10 tid=0xac15d800 nid=0x2c5c waiting for monitor entry [0xacb56000]<br> java.lang.Thread.State: BLOCKED (on object monitor)<br>
at com.rabbitmq.client.impl.AMQChannel.processShutdownSignal(AMQChannel.java:263)<br> - waiting to lock <0xb4532ee8> (a java.lang.Object)<br> at com.rabbitmq.client.impl.ChannelN.startProcessShutdownSignal(ChannelN.java:259)<br>
at com.rabbitmq.client.impl.ChannelN.processShutdownSignal(ChannelN.java:283)<br> at com.rabbitmq.client.impl.ChannelManager.handleSignal(ChannelManager.java:90)<br> at com.rabbitmq.client.impl.AMQConnection.finishShutdown(AMQConnection.java:696)<br>
at com.rabbitmq.client.impl.AMQConnection.shutdown(AMQConnection.java:669)<br> at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:550)<br><br>"pool-2-thread-1" prio=10 tid=0xac167000 nid=0x2c60 waiting for monitor entry [0xac77d000]<br>
java.lang.Thread.State: BLOCKED (on object monitor)<br> at java.io.BufferedOutputStream.write(BufferedOutputStream.java:94)<br> - waiting to lock <0xb4532608> (a java.io.BufferedOutputStream)<br> at java.io.DataOutputStream.writeByte(DataOutputStream.java:153)<br>
at com.rabbitmq.client.impl.Frame.writeTo(Frame.java:189)<br> at com.rabbitmq.client.impl.SocketFrameHandler.writeFrame(SocketFrameHandler.java:137)<br> - locked <0xb45325f0> (a java.io.DataOutputStream)<br>
at com.rabbitmq.client.impl.HeartbeatSender$HeartbeatRunnable.run(HeartbeatSender.java:133)<br> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)<br> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)<br>
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)<br> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)<br> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)<br>
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)<br> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)<br> at java.lang.Thread.run(Thread.java:722)<br>
<br></div>RabbitMQ server v 3.0.2, client (java) v 3.0.1, both run on rhel 6.4.<br><br></div>Do you know if there is some setting which can manage this timeout period (OS level / JVM level / client library level)?<br></div>
<div>Also the thread "AMQP Connection ..." seems is blocked on monitor
object while trying to make shutdown - could it be a bug which prevents to
make shutdown in time?<br></div><div><br></div>Thanks,<br></div>Oleg</div>