Hello,<div><div>We have a rabbitMQ consumer that takes messages from a queue and write them in a HBase store.</div><div>When&nbsp;HBase connection&nbsp;fails, the consumer get an exception and messages are not acknowledged, that's ok.</div></div><div>After a few minutes, the consumer doesn't get messages anymore, even after restarting Hbase.</div><div><br></div><div>When debugging rabbitMQ client, we have noticed, that it is blocked on the following line:&nbsp;SocketInputStream:129 ( Java build 1.6.0_33-b03)</div><div>The Frame class is waiting for messages, but the used socket doesn't receive any data anymore.</div><div>We also notice, that there is no timeout on this socket.</div><div><br></div><div><div>&nbsp; public static Frame readFrom(DataInputStream is) throws IOException {</div><div>&nbsp; &nbsp; &nbsp; &nbsp; int type;</div><div>&nbsp; &nbsp; &nbsp; &nbsp; int channel;</div><div><br></div><div>&nbsp; &nbsp; &nbsp; &nbsp; try {</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; type = is.readUnsignedByte();</div><div>&nbsp; &nbsp; &nbsp; &nbsp; } catch (SocketTimeoutException ste) {</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; // System.err.println("Timed out waiting for a frame.");</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; return null; // failed</div><div>&nbsp; &nbsp; &nbsp; &nbsp; }</div></div><div>&nbsp; &nbsp; &nbsp; &nbsp; ....</div><div>}</div><div><br></div><div>Here is my rabbitMQ report :&nbsp;</div><div><br></div><div><div>Reporting server status on {{2012,10,18},{15,39,34}}</div><div><br></div><div>Status of node rabbit@localhost</div><div>[{pid,20127},</div><div>&nbsp;{running_applications,[{rabbit,"RabbitMQ","2.7.1"},</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {os_mon,"CPO &nbsp;CXC 138 46","2.2.7"},</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {sasl,"SASL &nbsp;CXC 138 11","2.1.10"},</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {mnesia,"MNESIA &nbsp;CXC 138 12","4.5"},</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {stdlib,"ERTS &nbsp;CXC 138 10","1.17.5"},</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {kernel,"ERTS &nbsp;CXC 138 10","2.14.5"}]},</div><div>&nbsp;{os,{unix,linux}},</div><div>&nbsp;{erlang_version,"Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:30] [kernel-poll:true]\n"},</div><div>&nbsp;{memory,[{total,47487536},</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {processes,12533768},</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {processes_used,12228104},</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {system,34953768},</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {atom,1183993},</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {atom_used,1157601},</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {binary,16573424},</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {code,11134417},</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {ets,4108160}]},</div><div>&nbsp;{vm_memory_high_watermark,0.39999999990305685},</div><div>&nbsp;{vm_memory_limit,3300904140}]</div><div><br></div><div>Cluster status of node rabbit@samcrow ...</div><div>[{nodes,[{disc,[rabbit@samcrow]}]},{running_nodes,[rabbit@samcrow]}]</div><div><br></div><div>Application environment of node rabbit@samcrow ...</div><div>[{auth_backends,[rabbit_auth_backend_internal]},</div><div>&nbsp;{auth_mechanisms,['PLAIN','AMQPLAIN']},</div><div>&nbsp;{backing_queue_module,rabbit_variable_queue},</div><div>&nbsp;{cluster_nodes,[]},</div><div>&nbsp;{collect_statistics,none},</div><div>&nbsp;{collect_statistics_interval,5000},</div><div>&nbsp;{default_permissions,[&lt;&lt;".*"&gt;&gt;,&lt;&lt;".*"&gt;&gt;,&lt;&lt;".*"&gt;&gt;]},</div><div>&nbsp;{default_user,&lt;&lt;"guest"&gt;&gt;},</div><div>&nbsp;{default_user_tags,[administrator]},</div><div>&nbsp;{default_vhost,&lt;&lt;"/"&gt;&gt;},</div><div>&nbsp;{delegate_count,16},</div><div>&nbsp;{error_logger,{file,"/var/log/rabbitmq/rabbit@samcrow.log"}},</div><div>&nbsp;{frame_max,131072},</div><div>&nbsp;{hipe_compile,false},</div><div>&nbsp;{included_applications,[]},</div><div>&nbsp;{msg_store_file_size_limit,16777216},</div><div>&nbsp;{msg_store_index_module,rabbit_msg_store_ets_index},</div><div>&nbsp;{queue_index_max_journal_entries,262144},</div><div>&nbsp;{sasl_error_logger,{file,"/var/log/rabbitmq/rabbit@samcrow-sasl.log"}},</div><div>&nbsp;{server_properties,[]},</div><div>&nbsp;{ssl_listeners,[]},</div><div>&nbsp;{ssl_options,[]},</div><div>&nbsp;{tcp_listen_options,[binary,</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {packet,raw},</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {reuseaddr,true},</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {backlog,128},</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {nodelay,true},</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; {exit_on_close,false}]},</div><div>&nbsp;{tcp_listeners,[5672]},</div><div>&nbsp;{trace_vhosts,[]},</div><div>&nbsp;{vm_memory_high_watermark,0.4}]</div><div><br></div><div>Connections:</div><div>pid<span class="Apple-tab-span" style="white-space:pre">        </span>address<span class="Apple-tab-span" style="white-space:pre">        </span>port<span class="Apple-tab-span" style="white-space:pre">        </span>peer_address<span class="Apple-tab-span" style="white-space:pre">        </span>peer_port<span class="Apple-tab-span" style="white-space:pre">        </span>ssl<span class="Apple-tab-span" style="white-space:pre">        </span>peer_cert_subject<span class="Apple-tab-span" style="white-space:pre">        </span>peer_cert_issuer<span class="Apple-tab-span" style="white-space:pre">        </span>peer_cert_validity<span class="Apple-tab-span" style="white-space:pre">        </span>auth_mechanism<span class="Apple-tab-span" style="white-space:pre">        </span>ssl_protocol<span class="Apple-tab-span" style="white-space:pre">        </span>ssl_key_exchange<span class="Apple-tab-span" style="white-space:pre">        </span>ssl_cipher<span class="Apple-tab-span" style="white-space:pre">        </span>ssl_hash<span class="Apple-tab-span" style="white-space:pre">        </span>protocol<span class="Apple-tab-span" style="white-space:pre">        </span>user<span class="Apple-tab-span" style="white-space:pre">        </span>vhost<span class="Apple-tab-span" style="white-space:pre">        </span>timeout<span class="Apple-tab-span" style="white-space:pre">        </span>frame_max<span class="Apple-tab-span" style="white-space:pre">        </span>client_properties<span class="Apple-tab-span" style="white-space:pre">        </span>recv_oct<span class="Apple-tab-span" style="white-space:pre">        </span>recv_cnt<span class="Apple-tab-span" style="white-space:pre">        </span>send_oct<span class="Apple-tab-span" style="white-space:pre">        </span>send_cnt<span class="Apple-tab-span" style="white-space:pre">        </span>send_pend<span class="Apple-tab-span" style="white-space:pre">        </span>state<span class="Apple-tab-span" style="white-space:pre">        </span>channels</div><div>&lt;rabbit@samcrow.2.2883.0&gt;<span class="Apple-tab-span" style="white-space:pre">        </span>127.0.0.1<span class="Apple-tab-span" style="white-space:pre">        </span>5672<span class="Apple-tab-span" style="white-space:pre">        </span>127.0.0.1<span class="Apple-tab-span" style="white-space:pre">        </span>15220<span class="Apple-tab-span" style="white-space:pre">        </span>false<span class="Apple-tab-span" style="white-space:pre">                                </span>PLAIN<span class="Apple-tab-span" style="white-space:pre">                                        </span>{0,9,1}<span class="Apple-tab-span" style="white-space:pre">        </span>guest<span class="Apple-tab-span" style="white-space:pre">        </span>/<span class="Apple-tab-span" style="white-space:pre">        </span>0<span class="Apple-tab-span" style="white-space:pre">        </span>131072[{"product","RabbitMQ"},{"information","Licensed under the MPL. See http://www.rabbitmq.com/"},{"platform","Java"},{"capabilities",[{"exchange_exchange_bindings",true},{"consumer_cancel_notify",true},{"basic.nack",true},{"publisher_confirms",true}]},{"copyright","Copyright (C) 2007-2012 VMware, Inc."},{"version","2.8.1"}]<span class="Apple-tab-span" style="white-space:pre">        </span>11864749<span class="Apple-tab-span" style="white-space:pre">        </span>27712<span class="Apple-tab-span" style="white-space:pre">        </span>26676264<span class="Apple-tab-span" style="white-space:pre">        </span>11607<span class="Apple-tab-span" style="white-space:pre">        </span>0<span class="Apple-tab-span" style="white-space:pre">        </span>running<span class="Apple-tab-span" style="white-space:pre">        </span>23</div></div><div><br></div><div>I use rabbitMQ client version 2.8.1.</div><div><br></div><div>Do you have any idea ?</div><div>Thanks in advance,</div><div>Nicolas</div><div><br></div><div><br></div><div><br></div><div><br></div>