[rabbitmq-discuss] RabbitMQ blocking issue
Konar Peter (Nokia-LC/Chicago)
peter.konar at nokia.com
Tue Feb 12 13:27:19 GMT 2013
'rabbitmqctl report
[pkonar at ajtrafp063 ~]$ sudo rabbitmqctl report
Reporting server status on {{2013,2,12},{13,12,55}}
...
Status of node rabbit at ajtrafp064 ...
[{pid,3884},
{running_applications,
[{rabbitmq_management_visualiser,"RabbitMQ Visualiser","3.0.2"},
{rabbitmq_management,"RabbitMQ Management Console","3.0.2"},
{rabbitmq_management_agent,"RabbitMQ Management Agent","3.0.2"},
{rabbit,"RabbitMQ","3.0.2"},
{os_mon,"CPO CXC 138 46","2.2.7"},
{rabbitmq_mochiweb,"RabbitMQ Mochiweb Embedding","3.0.2"},
{webmachine,"webmachine","1.9.1-rmq3.0.2-git52e62bc"},
{mochiweb,"MochiMedia Web Server","2.3.1-rmq3.0.2-gitd541e9a"},
{xmerl,"XML parser","1.2.10"},
{inets,"INETS CXC 138 49","5.7.1"},
{mnesia,"MNESIA CXC 138 12","4.5"},
{amqp_client,"RabbitMQ AMQP Client","3.0.2"},
{sasl,"SASL CXC 138 11","2.1.10"},
{stdlib,"ERTS CXC 138 10","1.17.5"},
{kernel,"ERTS CXC 138 10","2.14.5"}]},
{os,{unix,linux}},
{erlang_version,
"Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:30] [kernel-poll:true]\n"},
{memory,
[{total,35280904},
{connection_procs,395120},
{queue_procs,5408},
{plugins,306296},
{other_proc,9714296},
{mnesia,78592},
{mgmt_db,153144},
{msg_index,22696},
{other_ets,1142280},
{binary,2462008},
{code,17446642},
{atom,1553689},
{other_system,2000733}]},
{vm_memory_high_watermark,0.4},
{vm_memory_limit,6689898496},
{disk_free_limit,1000000000},
{disk_free,62383673344},
{file_descriptors,
[{total_limit,924},{total_used,7},{sockets_limit,829},{sockets_used,5}]},
{processes,[{limit,1048576},{used,236}]},
{run_queue,0},
{uptime,69961}]
Cluster status of node rabbit at ajtrafp064 ...
[{nodes,[{disc,[rabbit at ajtrafp063,rabbit at ajtrafp064]}]},
{running_nodes,[rabbit at ajtrafp063,rabbit at ajtrafp064]},
{partitions,[]}]
Application environment of node rabbit at ajtrafp064 ...
[{auth_backends,[rabbit_auth_backend_internal]},
{auth_mechanisms,['PLAIN','AMQPLAIN']},
{backing_queue_module,rabbit_variable_queue},
{cluster_nodes,[rabbit at ajtrafp063,rabbit at ajtrafp064]},
{collect_statistics,fine},
{collect_statistics_interval,5000},
{default_permissions,[<<".*">>,<<".*">>,<<".*">>]},
{default_user,<<"guest">>},
{default_user_tags,[administrator]},
{default_vhost,<<"/">>},
{delegate_count,16},
{disk_free_limit,1000000000},
{enabled_plugins_file,"/etc/rabbitmq/enabled_plugins"},
{error_logger,{file,"/var/log/rabbitmq/rabbit at ajtrafp064.log"}},
{frame_max,131072},
{heartbeat,600},
{hipe_compile,false},
{included_applications,[]},
{log_levels,[{connection,info}]},
{msg_store_file_size_limit,16777216},
{msg_store_index_module,rabbit_msg_store_ets_index},
{plugins_dir,"/usr/lib/rabbitmq/lib/rabbitmq_server-3.0.2/sbin/../plugins"},
{plugins_expand_dir,"/var/lib/rabbitmq/mnesia/rabbit at ajtrafp064-plugins-expand"},
{queue_index_max_journal_entries,262144},
{reverse_dns_lookups,false},
{sasl_error_logger,{file,"/var/log/rabbitmq/rabbit at ajtrafp064-sasl.log"}},
{server_properties,[]},
{ssl_cert_login_from,distinguished_name},
{ssl_listeners,[]},
{ssl_options,[]},
{tcp_listen_options,[binary,
{packet,raw},
{reuseaddr,true},
{backlog,128},
{nodelay,true},
{exit_on_close,false}]},
{tcp_listeners,[5672]},
{trace_vhosts,[]},
{vm_memory_high_watermark,0.4}]
Status of node rabbit at ajtrafp063 ...
[{pid,3873},
{running_applications,
[{rabbitmq_management_visualiser,"RabbitMQ Visualiser","3.0.2"},
{rabbitmq_management,"RabbitMQ Management Console","3.0.2"},
{rabbitmq_management_agent,"RabbitMQ Management Agent","3.0.2"},
{rabbit,"RabbitMQ","3.0.2"},
{os_mon,"CPO CXC 138 46","2.2.7"},
{rabbitmq_mochiweb,"RabbitMQ Mochiweb Embedding","3.0.2"},
{webmachine,"webmachine","1.9.1-rmq3.0.2-git52e62bc"},
{mochiweb,"MochiMedia Web Server","2.3.1-rmq3.0.2-gitd541e9a"},
{xmerl,"XML parser","1.2.10"},
{inets,"INETS CXC 138 49","5.7.1"},
{mnesia,"MNESIA CXC 138 12","4.5"},
{amqp_client,"RabbitMQ AMQP Client","3.0.2"},
{sasl,"SASL CXC 138 11","2.1.10"},
{stdlib,"ERTS CXC 138 10","1.17.5"},
{kernel,"ERTS CXC 138 10","2.14.5"}]},
{os,{unix,linux}},
{erlang_version,
"Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:4:4] [rq:4] [async-threads:30] [kernel-poll:true]\n"},
{memory,
[{total,57276528},
{connection_procs,134480},
{queue_procs,21076560},
{plugins,81960},
{other_proc,9372096},
{mnesia,79040},
{mgmt_db,9904},
{msg_index,249152},
{other_ets,2159288},
{binary,3100872},
{code,17446642},
{atom,1553689},
{other_system,2012845}]},
{vm_memory_high_watermark,0.4},
{vm_memory_limit,6689898496},
{disk_free_limit,1000000000},
{disk_free,62375903232},
{file_descriptors,
[{total_limit,924},{total_used,7},{sockets_limit,829},{sockets_used,4}]},
{processes,[{limit,1048576},{used,226}]},
{run_queue,0},
{uptime,56951}]
Cluster status of node rabbit at ajtrafp063 ...
[{nodes,[{disc,[rabbit at ajtrafp063,rabbit at ajtrafp064]}]},
{running_nodes,[rabbit at ajtrafp064,rabbit at ajtrafp063]},
{partitions,[]}]
Application environment of node rabbit at ajtrafp063 ...
[{auth_backends,[rabbit_auth_backend_internal]},
{auth_mechanisms,['PLAIN','AMQPLAIN']},
{backing_queue_module,rabbit_variable_queue},
{cluster_nodes,[rabbit at ajtrafp063,rabbit at ajtrafp064]},
{collect_statistics,fine},
{collect_statistics_interval,5000},
{default_permissions,[<<".*">>,<<".*">>,<<".*">>]},
{default_user,<<"guest">>},
{default_user_tags,[administrator]},
{default_vhost,<<"/">>},
{delegate_count,16},
{disk_free_limit,1000000000},
{enabled_plugins_file,"/etc/rabbitmq/enabled_plugins"},
{error_logger,{file,"/var/log/rabbitmq/rabbit at ajtrafp063.log"}},
{frame_max,131072},
{heartbeat,600},
{hipe_compile,false},
{included_applications,[]},
{log_levels,[{connection,info}]},
{msg_store_file_size_limit,16777216},
{msg_store_index_module,rabbit_msg_store_ets_index},
{plugins_dir,"/usr/lib/rabbitmq/lib/rabbitmq_server-3.0.2/sbin/../plugins"},
{plugins_expand_dir,"/var/lib/rabbitmq/mnesia/rabbit at ajtrafp063-plugins-expand"},
{queue_index_max_journal_entries,262144},
{reverse_dns_lookups,false},
{sasl_error_logger,{file,"/var/log/rabbitmq/rabbit at ajtrafp063-sasl.log"}},
{server_properties,[]},
{ssl_cert_login_from,distinguished_name},
{ssl_listeners,[]},
{ssl_options,[]},
{tcp_listen_options,[binary,
{packet,raw},
{reuseaddr,true},
{backlog,128},
{nodelay,true},
{exit_on_close,false}]},
{tcp_listeners,[5672]},
{trace_vhosts,[]},
{vm_memory_high_watermark,0.4}]
Connections:
pid name port peer_port host peer_host ssl peer_cert_subject peer_cert_issuer peer_cert_validity auth_mechanism ssl_protocol ssl_key_exchange ssl_cipher ssl_hash protocol user vhost timeout frame_max client_properties recv_oct recv_cnt send_oct send_cnt send_pend state last_blocked_by last_blocked_age channels
<rabbit at ajtrafp063.3.3992.0> 10.196.42.119:48485 -> 10.196.42.21:5672 5672 48485 10.196.42.21 10.196.42.119 false PLAIN {0,9,1} guest / 600 131072 [{"product","RabbitMQ"},{"information","Licensed under the MPL. See http://www.rabbitmq.com/"},{"platform","Java"},{"capabilities",[{"exchange_exchange_bindings",true},{"consumer_cancel_notify",true},{"basic.nack",true},{"publisher_confirms",true}]},{"copyright","Copyright (C) 2007-2012 VMware, Inc."},{"version","2.8.4"}] 149398118 6460971 2122464675 7082940 0 running none infinity 1
<rabbit at ajtrafp063.3.4254.0> 10.196.42.120:59485 -> 10.196.42.21:5672 5672 59485 10.196.42.21 10.196.42.120 false PLAIN {0,9,1} guest / 600 131072 [{"product","RabbitMQ"},{"information","Licensed under the MPL. See http://www.rabbitmq.com/"},{"platform","Java"},{"capabilities",[{"exchange_exchange_bindings",true},{"consumer_cancel_notify",true},{"basic.nack",true},{"publisher_confirms",true}]},{"copyright","Copyright (C) 2007-2012 VMware, Inc."},{"version","2.8.4"}] 109362206 4866571 3624114636 5187935 0 running none infinity 1
<rabbit at ajtrafp063.3.20125.0> 147.243.130.14:34613 -> 10.196.42.21:5672 5672 34613 10.196.42.21 147.243.130.14 false PLAIN {0,9,1} guest / 600 131072 [{"product","RabbitMQ"},{"information","Licensed under the MPL. See http://www.rabbitmq.com/"},{"platform","Java"},{"capabilities",[{"exchange_exchange_bindings",true},{"consumer_cancel_notify",true},{"basic.nack",true},{"publisher_confirms",true}]},{"copyright","Copyright (C) 2007-2012 VMware, Inc."},{"version","2.8.4"}] 2254516 41540 96969022 93165 0 running none infinity 1
<rabbit at ajtrafp064.1.14970.129> 10.196.42.182:50012 -> 10.196.42.22:5672 5672 50012 10.196.42.22 10.196.42.182 false PLAIN {0,9,1} guest / 600 131072 [{"product","RabbitMQ"},{"information","Licensed under the MPL. See http://www.rabbitmq.com/"},{"platform","Java"},{"capabilities",[{"exchange_exchange_bindings",true},{"consumer_cancel_notify",true},{"basic.nack",true},{"publisher_confirms",true}]},{"copyright","Copyright (C) 2007-2013 VMware, Inc."},{"version","3.0.2"}] 892971313 13406695 65127580 4651947 0 running flow 708.153401 2
<rabbit at ajtrafp064.1.13886.130> 10.196.42.180:57261 -> 10.196.42.22:5672 5672 57261 10.196.42.22 10.196.42.180 false PLAIN {0,9,1} guest / 600 131072 [{"product","RabbitMQ"},{"information","Licensed under the MPL. See http://www.rabbitmq.com/"},{"platform","Java"},{"capabilities",[{"exchange_exchange_bindings",true},{"consumer_cancel_notify",true},{"basic.nack",true},{"publisher_confirms",true}]},{"copyright","Copyright (C) 2007-2013 VMware, Inc."},{"version","3.0.2"}] 991521388 13254940 64474148 4605273 0 running flow 1012.465014 1
<rabbit at ajtrafp064.1.5604.131> 10.196.42.128:38983 -> 10.196.42.22:5672 5672 38983 10.196.42.22 10.196.42.128 false PLAIN {0,9,1} guest / 600 131072 [{"product","RabbitMQ"},{"information","Licensed under the MPL. See http://www.rabbitmq.com/"},{"platform","Java"},{"capabilities",[{"exchange_exchange_bindings",true},{"consumer_cancel_notify",true},{"basic.nack",true},{"publisher_confirms",true}]},{"copyright","Copyright (C) 2007-2013 VMware, Inc."},{"version","3.0.2"}] 1260443690 13146545 63815924 4558257 0 running flow 713.080137 1
<rabbit at ajtrafp064.1.31824.131> 10.196.42.179:41165 -> 10.196.42.22:5672 5672 41165 10.196.42.22 10.196.42.179 false PLAIN {0,9,1} guest / 600 131072 [{"product","RabbitMQ"},{"information","Licensed under the MPL. See http://www.rabbitmq.com/"},{"platform","Java"},{"capabilities",[{"exchange_exchange_bindings",true},{"consumer_cancel_notify",true},{"basic.nack",true},{"publisher_confirms",true}]},{"copyright","Copyright (C) 2007-2013 VMware, Inc."},{"version","3.0.2"}] 1207139221 13617335 66191024 4727907 0 running flow 1314.712837 1
Channels:
pid name connection number user vhost transactional confirm consumer_count messages_unacknowledged messages_unconfirmed messages_uncommitted acks_uncommitted prefetch_count client_flow_blocked
<rabbit at ajtrafp063.3.3999.0> 10.196.42.119:48485 -> 10.196.42.21:5672 (1) <rabbit at ajtrafp063.3.3992.0> 1 guest / false false 1 0 0 0 0 8000 false
<rabbit at ajtrafp063.3.4260.0> 10.196.42.120:59485 -> 10.196.42.21:5672 (1) <rabbit at ajtrafp063.3.4254.0> 1 guest / false false 1 0 0 0 0 8000 false
<rabbit at ajtrafp063.3.20131.0> 147.243.130.14:34613 -> 10.196.42.21:5672 (1) <rabbit at ajtrafp063.3.20125.0> 1 guest / false false 1 0 0 0 0 8000 false
<rabbit at ajtrafp064.1.8630.1251> 10.196.42.182:50012 -> 10.196.42.22:5672 (44154) <rabbit at ajtrafp064.1.14970.129> 44154 guest / false false 0 0 0 0 0 0 false
<rabbit at ajtrafp064.1.8634.1251> 10.196.42.182:50012 -> 10.196.42.22:5672 (44155) <rabbit at ajtrafp064.1.14970.129> 44155 guest / false false 0 0 0 0 0 0 false
<rabbit at ajtrafp064.1.2184.1262> 10.196.42.128:38983 -> 10.196.42.22:5672 (62655) <rabbit at ajtrafp064.1.5604.131> 62655 guest / false false 0 0 0 0 0 0 false
<rabbit at ajtrafp064.1.2484.1262> 10.196.42.180:57261 -> 10.196.42.22:5672 (27264) <rabbit at ajtrafp064.1.13886.130> 27264 guest / false false 0 0 0 0 0 0 false
<rabbit at ajtrafp064.1.2981.1262> 10.196.42.179:41165 -> 10.196.42.22:5672 (7621) <rabbit at ajtrafp064.1.31824.131> 7621 guest / false false 0 0 0 0 0 0 false
Queues on /:
pid name durable auto_delete arguments owner_pid policy exclusive_consumer_pid exclusive_consumer_tag messages_ready messages_unacknowledged messages consumers active_consumers memory slave_pids synchronised_slave_pids backing_queue_status
<rabbit at ajtrafp063.3.626.0> Something true false [{"x-message-ttl",30000}] 0 0 0 0 0 21037632 [{q1,0}, {q2,0}, {delta,{delta,undefined,0,undefined}}, {q3,0}, {q4,0}, {len,0}, {pending_acks,0}, {target_ram_count,infinity}, {ram_msg_count,0}, {ram_ack_count,0}, {next_seq_id,72530278}, {persistent_count,0}, {avg_ingress_rate,0.0}, {avg_egress_rate,1946.202937736505}, {avg_ack_ingress_rate,0.0}, {avg_ack_egress_rate,0.0}]
<rabbit at ajtrafp063.3.4000.0> b2bserverajtrafd015.si.atl01.locn.s.nokia.com false true [{"x-message-ttl",45000}] <rabbit at ajtrafp063.3.3992.0> 0 0 0 1 1 23136 [{q1,0}, {q2,0}, {delta,{delta,undefined,0,undefined}}, {q3,0}, {q4,0}, {len,0}, {pending_acks,0}, {target_ram_count,infinity}, {ram_msg_count,0}, {ram_ack_count,0}, {next_seq_id,7114159}, {persistent_count,0}, {avg_ingress_rate,429.0251091151943}, {avg_egress_rate,429.0251091151943}, {avg_ack_ingress_rate,429.0251091151943}, {avg_ack_egress_rate,429.16893248398924}]
<rabbit at ajtrafp063.3.4261.0> b2bserverajtrafd016.si.atl01.locn.s.nokia.com false true [{"x-message-ttl",45000}] <rabbit at ajtrafp063.3.4254.0> 0 0 0 1 1 27568 [{q1,0}, {q2,0}, {delta,{delta,undefined,0,undefined}}, {q3,0}, {q4,0}, {len,0}, {pending_acks,0}, {target_ram_count,infinity}, {ram_msg_count,0}, {ram_ack_count,0}, {next_seq_id,5207687}, {persistent_count,0}, {avg_ingress_rate,349.00662555488776}, {avg_egress_rate,349.00662555488776}, {avg_ack_ingress_rate,349.00662555488776}, {avg_ack_egress_rate,349.1704784307351}]
<rabbit at ajtrafp063.3.20132.0> b2bserver-dicksMBPlocal.local false true [{"x-message-ttl",45000}] <rabbit at ajtrafp063.3.20125.0> 0 0 0 1 1 23136 [{q1,0}, {q2,0}, {delta,{delta,undefined,0,undefined}}, {q3,0}, {q4,0}, {len,0}, {pending_acks,0}, {target_ram_count,infinity}, {ram_msg_count,0}, {ram_ack_count,0}, {next_seq_id,107319}, {persistent_count,0}, {avg_ingress_rate,453.20681423716616}, {avg_egress_rate,453.20681423716616}, {avg_ack_ingress_rate,453.20681423716616}, {avg_ack_egress_rate,461.1071676868251}]
Exchanges on /:
name type durable auto_delete internal arguments policy
direct true false false []
amq.direct direct true false false []
amq.fanout fanout true false false []
amq.headers headers true false false []
amq.match headers true false false []
amq.rabbitmq.log topic true false false []
amq.rabbitmq.trace topic true false false []
amq.topic topic true false false []
clusterStatus topic true false false []
flowData topic true false false []
products topic true false false []
Bindings on /:
source_name source_kind destination_name destination_kind routing_key arguments
exchange Something queue Something []
exchange b2bserver-dicksMBPlocal.local queue b2bserver-dicksMBPlocal.local []
exchange b2bserverajtrafd015.si.atl01.locn.s.nokia.com queue b2bserverajtrafd015.si.atl01.locn.s.nokia.com []
exchange b2bserverajtrafd016.si.atl01.locn.s.nokia.com queue b2bserverajtrafd016.si.atl01.locn.s.nokia.com []
flowData exchange Something queue # []
flowData exchange b2bserver-dicksMBPlocal.local queue 1_0.107.* []
flowData exchange b2bserver-dicksMBPlocal.local queue 1_0.D01.* []
flowData exchange b2bserverajtrafd015.si.atl01.locn.s.nokia.com queue 1_0.D01.* []
flowData exchange b2bserverajtrafd016.si.atl01.locn.s.nokia.com queue 1_0.F32.* []
Consumers on /:
queue_name channel_pid consumer_tag ack_required
b2bserver-dicksMBPlocal.local <rabbit at ajtrafp063.3.20131.0> amq.ctag-7wgn3n8B8BpDFuw15dCNUQ true
b2bserverajtrafd015.si.atl01.locn.s.nokia.com <rabbit at ajtrafp063.3.3999.0> amq.ctag-cGLyM0l72ST1c05mwdn9eQ true
b2bserverajtrafd016.si.atl01.locn.s.nokia.com <rabbit at ajtrafp063.3.4260.0> amq.ctag-KBO1GEzW_LGGUBEXFu4Xag true
Permissions on /:
user configure write read
guest .* .* .*
Policies on /:
Parameters on /:
...done.
RabbitMQ Netstat
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 :::5672 :::* LISTEN
tcp 0 0 ::ffff:10.196.42.21:5672 ::ffff:10.196.42.119:48485 ESTABLISHED
tcp 0 0 ::ffff:10.196.42.21:5672 ::ffff:10.196.42.120:59485 ESTABLISHED
tcp 0 0 ::ffff:10.196.42.21:5672 ::ffff:147.243.130.14:34613 ESTABLISHED
TSA Client Netstat
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 246112 ::ffff:10.196.42.182:50012 ::ffff:10.196.42.22:5672 ESTABLISHED
Jstack:
"pool-6-thread-3" prio=10 tid=0x00007fc76050b800 nid=0x28b2 runnable [0x00007fc4aadeb000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketOutputStream.$$YJP$$socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite0(SocketOutputStream.java)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
- locked <0x000000063abd4638> (a java.io.BufferedOutputStream)
at java.io.DataOutputStream.flush(DataOutputStream.java:123)
at com.rabbitmq.client.impl.SocketFrameHandler.flush(SocketFrameHandler.java:142)
at com.rabbitmq.client.impl.AMQConnection.flush(AMQConnection.java:488)
at com.rabbitmq.client.impl.AMQCommand.transmit(AMQCommand.java:125)
at com.rabbitmq.client.impl.AMQChannel.quiescingTransmit(AMQChannel.java:316)
- locked <0x000000060f49ed88> (a java.lang.Object)
at com.rabbitmq.client.impl.AMQChannel.transmit(AMQChannel.java:292)
- locked <0x000000060f49ed88> (a java.lang.Object)
at com.rabbitmq.client.impl.ChannelN.basicPublish(ChannelN.java:634)
at sun.reflect.GeneratedMethodAccessor77.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory$CachedChannelInvocationHandler.invoke(CachingConnectionFactory.java:348)
- locked <0x000000060f49f2f8> (a java.lang.Object)
at $Proxy59.basicPublish(Unknown Source)
at org.springframework.amqp.rabbit.core.RabbitTemplate.doSend(RabbitTemplate.java:648)
at org.springframework.amqp.rabbit.core.RabbitTemplate$1.doInRabbit(RabbitTemplate.java:331)
at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:600)
at org.springframework.amqp.rabbit.core.RabbitTemplate.send(RabbitTemplate.java:329)
at org.springframework.amqp.rabbit.core.RabbitTemplate.send(RabbitTemplate.java:323)
at com.nokia.traffic.flowdata.transport.rabbitmq.RabbitMQSender.send(RabbitMQSender.java:53)
at com.nokia.traffic.ttbhm.mp.linearstate.TrafficStateAccumulatorMP.doSend_aroundBody4(TrafficStateAccumulatorMP.java:183)
at com.nokia.traffic.ttbhm.mp.linearstate.TrafficStateAccumulatorMP$AjcClosure5.run(TrafficStateAccumulatorMP.java:1)
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
at com.nokia.traffic.ttbhm.audit.MessageGroupingAspect.dispatchOutput(MessageGroupingAspect.java:70)
at com.nokia.traffic.ttbhm.mp.linearstate.TrafficStateAccumulatorMP.doSend(TrafficStateAccumulatorMP.java:182)
at com.nokia.traffic.ttbhm.mp.linearstate.TrafficStateAccumulatorMP.trySendingALinear(TrafficStateAccumulatorMP.java:140)
at com.nokia.traffic.ttbhm.mp.linearstate.TrafficStateAccumulatorMP.onSendLinear_aroundBody2(TrafficStateAccumulatorMP.java:115)
at com.nokia.traffic.ttbhm.mp.linearstate.TrafficStateAccumulatorMP$AjcClosure3.run(TrafficStateAccumulatorMP.java:1)
at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
at com.nokia.traffic.ttbhm.audit.MessageGroupingAspect.outputHandler(MessageGroupingAspect.java:231)
at com.nokia.traffic.ttbhm.mp.linearstate.TrafficStateAccumulatorMP.onSendLinear(TrafficStateAccumulatorMP.java:113)
at sun.reflect.GeneratedMethodAccessor70.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.nokia.dempsy.container.internal.LifecycleHelper.invokeOutput(LifecycleHelper.java:178)
at com.nokia.dempsy.container.MpContainer.invokeOperation(MpContainer.java:1160)
at com.nokia.dempsy.container.MpContainer.access$600(MpContainer.java:70)
at com.nokia.dempsy.container.MpContainer$6.run(MpContainer.java:924)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Thanks
Peter Konar
________________________________________
From: ext Matthias Radestock [matthias at rabbitmq.com]
Sent: Monday, February 11, 2013 3:58 PM
To: Konar Peter (Nokia-LC/Chicago)
Cc: Carroll James (Nokia-LC/Malvern); Discussions about RabbitMQ; Mccuiston Mitchell (Nokia-LC/Chicago); Lieber Dick (Nokia-LC/Chicago); Johnson Anthony (Nokia-LC/Alpharetta); Yu Jerry.3 (Nokia-LC/Alpharetta)
Subject: Re: [rabbitmq-discuss] RabbitMQ blocking issue
Peter,
On 11/02/13 21:44, Konar Peter (Nokia-LC/Chicago) wrote:
> I have upgraded client and server to 3.0.2 to get heartbeats.
Great, that will make it easier to track down any problem you may see.
More generally though note that heartbeats can be enabled in older
clients; all that changed in 3.0.x is that the server now by default
suggests that heartbeats should be enabled.
> I will get you a "netstat -an" output as soon as we can get this to
> fail.
A snapshot of
- client stacktrace
- server 'rabbitmqctl report' output
- client and server 'netstat -an' output
would be ideal.
Regards,
Matthias.
The information contained in this communication may be CONFIDENTIAL and is intended only for the use of the recipient(s) named above. If you are not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please notify the sender and delete/destroy the original message and any copy of it from your computer or paper files.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: jstack.13:13:52
Type: application/octet-stream
Size: 146933 bytes
Desc: jstack.13:13:52
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130212/d783c71e/attachment.obj>
More information about the rabbitmq-discuss
mailing list