<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; ">
<div style="font-family: Calibri, sans-serif; ">We are running RabbitMQ 3.1.5 and Erlang 16B01 x64 on Windows Server 2008R2 in a clustered configuration.</div>
<div style="font-family: Calibri, sans-serif; "><br>
</div>
<div style="font-family: Calibri, sans-serif; ">We are seeing nodes crash at random and don't understand what's going on. However, there is a common theme in the logs, of which a sample is here:</div>
<div style="font-family: Calibri, sans-serif; "><br>
</div>
<div style="font-family: Calibri, sans-serif; ">Does anyone have any insight into what might be happening? We see this in several different environments with similar cluster configurations and all running the same RabbitMQ/Erlang versions…</div>
<div style="font-family: Calibri, sans-serif; "><br>
</div>
<div>
<div><font face="Courier">=INFO REPORT==== 13-Sep-2013::15:10:10 ===</font></div>
<div><font face="Courier">accepting AMQP connection <0.5121.11> (10.100.183.10:63072 -> 10.100.183.75:5672)</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 13-Sep-2013::15:10:10 ===</font></div>
<div><font face="Courier">accepting AMQP connection <0.5149.11> (10.100.183.10:63075 -> 10.100.183.75:5672)</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:15 ===</font></div>
<div><font face="Courier">Mirrored-queue (queue 'ha-execute_sendtopracticefusionactivity' in vhost 'vortex'): Slave <rabbit@SJPERMQ05.2.416.0> saw deaths of mirrors <rabbit@SJPERMQ04.3.409.0> </font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:15 ===</font></div>
<div><font face="Courier">Mirrored-queue (queue 'ha-message_tracking_control_sjpeweb01' in vhost 'vortex'): Slave <rabbit@SJPERMQ05.2.532.0> saw deaths of mirrors <rabbit@SJPERMQ04.3.529.0> </font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:15 ===</font></div>
<div><font face="Courier">Mirrored-queue (queue 'ha-execute_lookupproviderinregistryactivity' in vhost 'vortex'): Slave <rabbit@SJPERMQ05.2.352.0> saw deaths of mirrors <rabbit@SJPERMQ04.3.349.0> </font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:15 ===</font></div>
<div><font face="Courier">Mirrored-queue (queue 'ha-compensate_sendtocloverleafactivity' in vhost 'vortex'): Slave <rabbit@SJPERMQ05.2.356.0> saw deaths of mirrors <rabbit@SJPERMQ04.3.357.0> </font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:15 ===</font></div>
<div><font face="Courier">Mirrored-queue (queue 'ha-subscriber_activity_service_control_sjpeweb02' in vhost 'vortex'): Slave <rabbit@SJPERMQ05.2.376.0> saw deaths of mirrors <rabbit@SJPERMQ04.3.373.0> </font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:15 ===</font></div>
<div><font face="Courier">Mirrored-queue (queue 'ha-execute_fakefaultedactivity' in vhost 'vortex'): Slave <rabbit@SJPERMQ05.2.348.0> saw deaths of mirrors <rabbit@SJPERMQ04.3.345.0> </font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:15 ===</font></div>
<div><font face="Courier">Mirrored-queue (queue 'ha-inbound_activity_service_control_sjpeweb04' in vhost 'vortex'): Slave <rabbit@SJPERMQ05.2.388.0> saw deaths of mirrors <rabbit@SJPERMQ04.3.385.0> </font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:15 ===</font></div>
<div><font face="Courier">Mirrored-queue (queue 'ha-execute_mongopersisteractivity' in vhost 'vortex'): Slave <rabbit@SJPERMQ05.2.320.0> saw deaths of mirrors <rabbit@SJPERMQ04.3.317.0> </font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:15 ===</font></div>
<div><font face="Courier">Mirrored-queue (queue 'ha-compensate_sendtorouteractivity' in vhost 'vortex'): Slave <rabbit@SJPERMQ05.2.344.0> saw deaths of mirrors <rabbit@SJPERMQ04.3.341.0> </font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:15 ===</font></div>
<div><font face="Courier">Mirrored-queue (queue 'ha-compensate_preparedocumentforroutingactivity' in vhost 'vortex'): Slave <rabbit@SJPERMQ05.2.384.0> saw deaths of mirrors <rabbit@SJPERMQ04.3.381.0> </font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:15 ===</font></div>
<div><font face="Courier">Mirrored-queue (queue 'ha-execute_unrecognizedmessageactivity' in vhost 'vortex'): Slave <rabbit@SJPERMQ05.2.316.0> saw deaths of mirrors <rabbit@SJPERMQ04.3.313.0> </font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:15 ===</font></div>
<div><font face="Courier">Mirrored-queue (queue 'ha-execute_sendtocloverleafactivity' in vhost 'vortex'): Slave <rabbit@SJPERMQ05.2.360.0> saw deaths of mirrors <rabbit@SJPERMQ04.3.353.0> </font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:15 ===</font></div>
<div><font face="Courier">Mirrored-queue (queue 'ha-inbound_activity_service_control_sjpeweb02' in vhost 'vortex'): Slave <rabbit@SJPERMQ05.2.372.0> saw deaths of mirrors <rabbit@SJPERMQ04.3.369.0> </font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:15 ===</font></div>
<div><font face="Courier">Mirrored-queue (queue 'ha-exchange_infrastructure_control_sjpeweb02' in vhost 'vortex'): Slave <rabbit@SJPERMQ05.2.304.0> saw deaths of mirrors <rabbit@SJPERMQ04.3.301.0> </font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=ERROR REPORT==== 15-Sep-2013::06:34:16 ===</font></div>
<div><font face="Courier">** Generic server <0.336.0> terminating</font></div>
<div><font face="Courier">** Last message in was {gm_deaths,[<5111.330.0>]}</font></div>
<div><font face="Courier">** When Server state == {state,</font></div>
<div><font face="Courier"> {amqqueue,</font></div>
<div><font face="Courier"> {resource,<<"vortex">>,queue,</font></div>
<div><font face="Courier"> <<"ha-execute_sendtorouteractivity">>},</font></div>
<div><font face="Courier"> true,false,none,</font></div>
<div><font face="Courier"> [{<<"x-ha-policy">>,longstr,<<"all">>}],</font></div>
<div><font face="Courier"> <5111.329.0>,</font></div>
<div><font face="Courier"> [<5111.329.0>,<7010.329.0>,<7011.333.0>],</font></div>
<div><font face="Courier"> [<7011.333.0>,<7010.329.0>,<5111.329.0>],</font></div>
<div><font face="Courier"> [{vhost,<<"vortex">>},</font></div>
<div><font face="Courier"> {name,<<"ha-policy">>},</font></div>
<div><font face="Courier"> {pattern,<<"^ha-.*">>},</font></div>
<div><font face="Courier"> {definition,</font></div>
<div><font face="Courier"> [{<<"ha-mode">>,<<"all">>},</font></div>
<div><font face="Courier"> {<<"ha-sync-mode">>,<<"automatic">>}]},</font></div>
<div><font face="Courier"> {priority,0}],</font></div>
<div><font face="Courier"> [{<7011.334.0>,<7011.333.0>},</font></div>
<div><font face="Courier"> {<7010.330.0>,<7010.329.0>},</font></div>
<div><font face="Courier"> {<5111.330.0>,<5111.329.0>},</font></div>
<div><font face="Courier"> {<7009.328.0>,<7009.327.0>}]},</font></div>
<div><font face="Courier"> <0.337.0>,rabbit_variable_queue,</font></div>
<div><font face="Courier"> {vqstate,</font></div>
<div><font face="Courier"> {0,{[],[]}},</font></div>
<div><font face="Courier"> {0,{[],[]}},</font></div>
<div><font face="Courier"> {delta,undefined,0,undefined},</font></div>
<div><font face="Courier"> {0,{[],[]}},</font></div>
<div><font face="Courier"> {0,{[],[]}},</font></div>
<div><font face="Courier"> 0,</font></div>
<div><font face="Courier"> {0,nil},</font></div>
<div><font face="Courier"> {0,nil},</font></div>
<div><font face="Courier"> {qistate,</font></div>
<div><font face="Courier"> "e:/RabbitMQ/db/rabbit-mnesia/queues/CZDXJN5ZJBZN98DWOHU2MZHL",</font></div>
<div><font face="Courier"> {{dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> []}}},</font></div>
<div><font face="Courier"> []},</font></div>
<div><font face="Courier"> undefined,0,65536,</font></div>
<div><font face="Courier"> #Fun<rabbit_variable_queue.2.81334491>,</font></div>
<div><font face="Courier"> {0,nil}},</font></div>
<div><font face="Courier"> {{client_msstate,msg_store_persistent,</font></div>
<div><font face="Courier"> <<127,33,92,106,192,40,110,10,76,157,95,226,1,93,</font></div>
<div><font face="Courier"> 82,226>>,</font></div>
<div><font face="Courier"> {dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> []}}},</font></div>
<div><font face="Courier"> {state,348236,</font></div>
<div><font face="Courier"> "e:/RabbitMQ/db/rabbit-mnesia/msg_store_persistent"},</font></div>
<div><font face="Courier"> rabbit_msg_store_ets_index,</font></div>
<div><font face="Courier"> "e:/RabbitMQ/db/rabbit-mnesia/msg_store_persistent",</font></div>
<div><font face="Courier"> <0.269.0>,352333,344139,356430,360527},</font></div>
<div><font face="Courier"> {client_msstate,msg_store_transient,</font></div>
<div><font face="Courier"> <<95,20,53,215,211,161,4,63,158,97,9,63,159,60,91,</font></div>
<div><font face="Courier"> 221>>,</font></div>
<div><font face="Courier"> {dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> []}}},</font></div>
<div><font face="Courier"> {state,327751,</font></div>
<div><font face="Courier"> "e:/RabbitMQ/db/rabbit-mnesia/msg_store_transient"},</font></div>
<div><font face="Courier"> rabbit_msg_store_ets_index,</font></div>
<div><font face="Courier"> "e:/RabbitMQ/db/rabbit-mnesia/msg_store_transient",</font></div>
<div><font face="Courier"> <0.264.0>,331848,323654,335945,340042}},</font></div>
<div><font face="Courier"> true,0,0,0,infinity,0,0,0,0,0,</font></div>
<div><font face="Courier"> {rates,</font></div>
<div><font face="Courier"> {{1379,109357,6220},0},</font></div>
<div><font face="Courier"> {{1379,109357,6220},0},</font></div>
<div><font face="Courier"> 0.0,0.0,</font></div>
<div><font face="Courier"> {1379,109829,181271}},</font></div>
<div><font face="Courier"> {0,nil},</font></div>
<div><font face="Courier"> {0,nil},</font></div>
<div><font face="Courier"> {0,nil},</font></div>
<div><font face="Courier"> {0,nil},</font></div>
<div><font face="Courier"> 0,0,</font></div>
<div><font face="Courier"> {rates,</font></div>
<div><font face="Courier"> {{1379,109357,6220},0},</font></div>
<div><font face="Courier"> {{1379,109357,6220},0},</font></div>
<div><font face="Courier"> 0.0,0.0,</font></div>
<div><font face="Courier"> {1379,109829,181271}}},</font></div>
<div><font face="Courier"> undefined,#Ref<0.0.55.58770>,</font></div>
<div><font face="Courier"> {dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},</font></div>
<div><font face="Courier"> {dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},</font></div>
<div><font face="Courier"> {dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},</font></div>
<div><font face="Courier"> {dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},</font></div>
<div><font face="Courier"> 0}</font></div>
<div><font face="Courier">** Reason for termination == </font></div>
<div><font face="Courier">** {{badmatch,{error,not_found}},</font></div>
<div><font face="Courier"> [{rabbit_amqqueue_process,i,2,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,'-infos/2-lc$^0/1-0-',2,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,'-infos/2-lc$^0/1-0-',2,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,emit_stats,2,[]},</font></div>
<div><font face="Courier"> {rabbit_event,if_enabled,3,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,'-terminate_delete/3-fun-1-',6,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,terminate_shutdown,2,[]},</font></div>
<div><font face="Courier"> {gen_server2,terminate,3,[]}]}</font></div>
<div><font face="Courier">** In 'terminate' callback with reason ==</font></div>
<div><font face="Courier">** normal</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:16 ===</font></div>
<div><font face="Courier">closing AMQP connection <0.4955.11> (10.100.183.10:62972 -> 10.100.183.75:5672)</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:16 ===</font></div>
<div><font face="Courier">closing AMQP connection <0.3177.11> (10.100.183.10:49728 -> 10.100.183.75:5672)</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:25 ===</font></div>
<div><font face="Courier">accepting AMQP connection <0.24239.16> (10.100.183.10:56699 -> 10.100.183.75:5672)</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:25 ===</font></div>
<div><font face="Courier">accepting AMQP connection <0.24248.16> (10.100.183.10:55592 -> 10</font></div>
</div>
<div style="font-family: Calibri, sans-serif; "><br>
</div>
<div style="font-family: Calibri, sans-serif; "><br>
</div>
<div style="font-family: Calibri, sans-serif; ">And a little later in the same log, here:</div>
<div style="font-family: Calibri, sans-serif; "><br>
</div>
<div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:25 ===</font></div>
<div><font face="Courier">Synchronising queue 'ha-compensate_fakecompensationfaultedactivity' in vhost 'vortex': all slaves already synced</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:25 ===</font></div>
<div><font face="Courier">Synchronising queue 'ha-compensate_fakecompensationfaultedactivity' in vhost 'vortex': 0 messages to synchronise</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:25 ===</font></div>
<div><font face="Courier">Synchronising queue 'ha-compensate_fakecompensationfaultedactivity' in vhost 'vortex': all slaves already synced</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:25 ===</font></div>
<div><font face="Courier">Synchronising queue 'ha-compensate_fakecompensationfaultedactivity' in vhost 'vortex': 0 messages to synchronise</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:25 ===</font></div>
<div><font face="Courier">Synchronising queue 'ha-compensate_fakecompensationfaultedactivity' in vhost 'vortex': all slaves already synced</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:45 ===</font></div>
<div><font face="Courier">accepting AMQP connection <0.24463.16> (10.100.183.10:55948 -> 10.100.183.75:5672)</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=ERROR REPORT==== 15-Sep-2013::06:34:45 ===</font></div>
<div><font face="Courier">** Generic server <0.353.0> terminating</font></div>
<div><font face="Courier">** Last message in was {'DOWN',#Ref<0.0.0.3812>,process,<5111.350.0>,</font></div>
<div><font face="Courier"> {function_clause,</font></div>
<div><font face="Courier"> [{orddict,fetch,</font></div>
<div><font face="Courier"> [{9,<5111.350.0>},[]],</font></div>
<div><font face="Courier"> [{file,"orddict.erl"},{line,72}]},</font></div>
<div><font face="Courier"> {gm,check_neighbours,1,[]},</font></div>
<div><font face="Courier"> {gm,handle_cast,2,[]},</font></div>
<div><font face="Courier"> {gen_server2,handle_msg,2,[]},</font></div>
<div><font face="Courier"> {proc_lib,wake_up,3,</font></div>
<div><font face="Courier"> [{file,"proc_lib.erl"},{line,249}]}]}}</font></div>
<div><font face="Courier">** When Server state == {state,</font></div>
<div><font face="Courier"> {18,<0.353.0>},</font></div>
<div><font face="Courier"> {{22,<0.24486.16>},#Ref<0.0.55.61004>},</font></div>
<div><font face="Courier"> {{9,<5111.350.0>},#Ref<0.0.0.3812>},</font></div>
<div><font face="Courier"> {resource,<<"vortex">>,queue,</font></div>
<div><font face="Courier"> <<"ha-execute_lookupproviderinregistryactivity">>},</font></div>
<div><font face="Courier"> rabbit_mirror_queue_slave,</font></div>
<div><font face="Courier"> {24,</font></div>
<div><font face="Courier"> [{{9,<5111.350.0>},</font></div>
<div><font face="Courier"> {view_member,</font></div>
<div><font face="Courier"> {9,<5111.350.0>},</font></div>
<div><font face="Courier"> [],</font></div>
<div><font face="Courier"> {18,<0.353.0>},</font></div>
<div><font face="Courier"> {12,<7010.350.0>}}},</font></div>
<div><font face="Courier"> {{12,<7010.350.0>},</font></div>
<div><font face="Courier"> {view_member,</font></div>
<div><font face="Courier"> {12,<7010.350.0>},</font></div>
<div><font face="Courier"> [],</font></div>
<div><font face="Courier"> {9,<5111.350.0>},</font></div>
<div><font face="Courier"> {22,<0.24486.16>}}},</font></div>
<div><font face="Courier"> {{18,<0.353.0>},</font></div>
<div><font face="Courier"> {view_member,</font></div>
<div><font face="Courier"> {18,<0.353.0>},</font></div>
<div><font face="Courier"> [],</font></div>
<div><font face="Courier"> {22,<0.24486.16>},</font></div>
<div><font face="Courier"> {9,<5111.350.0>}}},</font></div>
<div><font face="Courier"> {{22,<0.24486.16>},</font></div>
<div><font face="Courier"> {view_member,</font></div>
<div><font face="Courier"> {22,<0.24486.16>},</font></div>
<div><font face="Courier"> [],</font></div>
<div><font face="Courier"> {12,<7010.350.0>},</font></div>
<div><font face="Courier"> {18,<0.353.0>}}}]},</font></div>
<div><font face="Courier"> 0,</font></div>
<div><font face="Courier"> [{{9,<5111.350.0>},</font></div>
<div><font face="Courier"> {member,{[{2,process_death}],[]},2,1}},</font></div>
<div><font face="Courier"> {{12,<7010.350.0>},</font></div>
<div><font face="Courier"> {member,{[{2,process_death}],[]},2,1}},</font></div>
<div><font face="Courier"> {{18,<0.353.0>},{member,{[],[]},0,0}},</font></div>
<div><font face="Courier"> {{22,<0.24486.16>},{member,{[],[]},-1,-1}}],</font></div>
<div><font face="Courier"> [<0.352.0>],</font></div>
<div><font face="Courier"> {[],[]},</font></div>
<div><font face="Courier"> [],undefined,</font></div>
<div><font face="Courier"> #Fun<rabbit_misc.execute_mnesia_transaction.1>}</font></div>
<div><font face="Courier">** Reason for termination == </font></div>
<div><font face="Courier">** {function_clause,[{orddict,fetch,</font></div>
<div><font face="Courier"> [{18,<0.353.0>},[]],</font></div>
<div><font face="Courier"> [{file,"orddict.erl"},{line,72}]},</font></div>
<div><font face="Courier"> {gm,check_neighbours,1,[]},</font></div>
<div><font face="Courier"> {gm,handle_info,2,[]},</font></div>
<div><font face="Courier"> {gen_server2,handle_msg,2,[]},</font></div>
<div><font face="Courier"> {proc_lib,wake_up,3,[{file,"proc_lib.erl"},{line,249}]}]}</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=ERROR REPORT==== 15-Sep-2013::06:34:45 ===</font></div>
<div><font face="Courier">** Generic server <0.24486.16> terminating</font></div>
<div><font face="Courier">** Last message in was {'DOWN',#Ref<0.0.55.60998>,process,<0.353.0>,</font></div>
<div><font face="Courier"> {function_clause,</font></div>
<div><font face="Courier"> [{orddict,fetch,</font></div>
<div><font face="Courier"> [{18,<0.353.0>},[]],</font></div>
<div><font face="Courier"> [{file,"orddict.erl"},{line,72}]},</font></div>
<div><font face="Courier"> {gm,check_neighbours,1,[]},</font></div>
<div><font face="Courier"> {gm,handle_info,2,[]},</font></div>
<div><font face="Courier"> {gen_server2,handle_msg,2,[]},</font></div>
<div><font face="Courier"> {proc_lib,wake_up,3,</font></div>
<div><font face="Courier"> [{file,"proc_lib.erl"},{line,249}]}]}}</font></div>
<div><font face="Courier">** When Server state == {state,</font></div>
<div><font face="Courier"> {22,<0.24486.16>},</font></div>
<div><font face="Courier"> {{12,<7010.350.0>},#Ref<0.0.55.60997>},</font></div>
<div><font face="Courier"> {{18,<0.353.0>},#Ref<0.0.55.60998>},</font></div>
<div><font face="Courier"> {resource,<<"vortex">>,queue,</font></div>
<div><font face="Courier"> <<"ha-execute_lookupproviderinregistryactivity">>},</font></div>
<div><font face="Courier"> rabbit_mirror_queue_coordinator,</font></div>
<div><font face="Courier"> {23,</font></div>
<div><font face="Courier"> [{{9,<5111.350.0>},</font></div>
<div><font face="Courier"> {view_member,</font></div>
<div><font face="Courier"> {9,<5111.350.0>},</font></div>
<div><font face="Courier"> [],</font></div>
<div><font face="Courier"> {18,<0.353.0>},</font></div>
<div><font face="Courier"> {12,<7010.350.0>}}},</font></div>
<div><font face="Courier"> {{12,<7010.350.0>},</font></div>
<div><font face="Courier"> {view_member,</font></div>
<div><font face="Courier"> {12,<7010.350.0>},</font></div>
<div><font face="Courier"> [],</font></div>
<div><font face="Courier"> {9,<5111.350.0>},</font></div>
<div><font face="Courier"> {22,<0.24486.16>}}},</font></div>
<div><font face="Courier"> {{18,<0.353.0>},</font></div>
<div><font face="Courier"> {view_member,</font></div>
<div><font face="Courier"> {18,<0.353.0>},</font></div>
<div><font face="Courier"> [{6,<7009.348.0>}],</font></div>
<div><font face="Courier"> {22,<0.24486.16>},</font></div>
<div><font face="Courier"> {9,<5111.350.0>}}},</font></div>
<div><font face="Courier"> {{22,<0.24486.16>},</font></div>
<div><font face="Courier"> {view_member,</font></div>
<div><font face="Courier"> {22,<0.24486.16>},</font></div>
<div><font face="Courier"> [],</font></div>
<div><font face="Courier"> {12,<7010.350.0>},</font></div>
<div><font face="Courier"> {18,<0.353.0>}}}]},</font></div>
<div><font face="Courier"> -1,</font></div>
<div><font face="Courier"> [{{6,<7009.348.0>},{member,{[],[]},15,15}},</font></div>
<div><font face="Courier"> {{9,<5111.350.0>},</font></div>
<div><font face="Courier"> {member,{[{2,process_death}],[]},2,1}},</font></div>
<div><font face="Courier"> {{12,<7010.350.0>},</font></div>
<div><font face="Courier"> {member,{[{2,process_death}],[]},2,1}},</font></div>
<div><font face="Courier"> {{18,<0.353.0>},{member,{[],[]},0,0}},</font></div>
<div><font face="Courier"> {{22,<0.24486.16>},{member,{[],[]},-1,-1}}],</font></div>
<div><font face="Courier"> [<0.24485.16>],</font></div>
<div><font face="Courier"> {[],[]},</font></div>
<div><font face="Courier"> [],undefined,</font></div>
<div><font face="Courier"> #Fun<rabbit_misc.execute_mnesia_transaction.1>}</font></div>
<div><font face="Courier">** Reason for termination == </font></div>
<div><font face="Courier">** {function_clause,[{orddict,fetch,</font></div>
<div><font face="Courier"> [{22,<0.24486.16>},[]],</font></div>
<div><font face="Courier"> [{file,"orddict.erl"},{line,72}]},</font></div>
<div><font face="Courier"> {gm,check_neighbours,1,[]},</font></div>
<div><font face="Courier"> {gm,handle_info,2,[]},</font></div>
<div><font face="Courier"> {gen_server2,handle_msg,2,[]},</font></div>
<div><font face="Courier"> {proc_lib,init_p_do_apply,3,</font></div>
<div><font face="Courier"> [{file,"proc_lib.erl"},{line,239}]}]}</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=ERROR REPORT==== 15-Sep-2013::06:34:45 ===</font></div>
<div><font face="Courier">** Generic server <0.352.0> terminating</font></div>
<div><font face="Courier">** Last message in was {gm_deaths,[<0.24486.16>,<0.353.0>,<7010.350.0>,</font></div>
<div><font face="Courier"> <5111.350.0>]}</font></div>
<div><font face="Courier">** When Server state == {state,</font></div>
<div><font face="Courier"> {amqqueue,</font></div>
<div><font face="Courier"> {resource,<<"vortex">>,queue,</font></div>
<div><font face="Courier"> <<"ha-execute_lookupproviderinregistryactivity">>},</font></div>
<div><font face="Courier"> true,false,none,</font></div>
<div><font face="Courier"> [{<<"x-ha-policy">>,longstr,<<"all">>}],</font></div>
<div><font face="Courier"> <0.24484.16>,</font></div>
<div><font face="Courier"> [<5111.349.0>,<7010.349.0>,<7011.349.0>],</font></div>
<div><font face="Courier"> [<7011.349.0>,<7010.349.0>,<5111.349.0>],</font></div>
<div><font face="Courier"> [{vhost,<<"vortex">>},</font></div>
<div><font face="Courier"> {name,<<"ha-policy">>},</font></div>
<div><font face="Courier"> {pattern,<<"^ha-.*">>},</font></div>
<div><font face="Courier"> {definition,</font></div>
<div><font face="Courier"> [{<<"ha-mode">>,<<"all">>},</font></div>
<div><font face="Courier"> {<<"ha-sync-mode">>,<<"automatic">>}]},</font></div>
<div><font face="Courier"> {priority,0}],</font></div>
<div><font face="Courier"> [{<7011.350.0>,<7011.349.0>},</font></div>
<div><font face="Courier"> {<7010.350.0>,<7010.349.0>},</font></div>
<div><font face="Courier"> {<5111.350.0>,<5111.349.0>},</font></div>
<div><font face="Courier"> {<7009.348.0>,<7009.347.0>}]},</font></div>
<div><font face="Courier"> <0.353.0>,rabbit_variable_queue,</font></div>
<div><font face="Courier"> {vqstate,</font></div>
<div><font face="Courier"> {0,{[],[]}},</font></div>
<div><font face="Courier"> {0,{[],[]}},</font></div>
<div><font face="Courier"> {delta,undefined,0,undefined},</font></div>
<div><font face="Courier"> {0,{[],[]}},</font></div>
<div><font face="Courier"> {0,{[],[]}},</font></div>
<div><font face="Courier"> 0,</font></div>
<div><font face="Courier"> {0,nil},</font></div>
<div><font face="Courier"> {0,nil},</font></div>
<div><font face="Courier"> {qistate,</font></div>
<div><font face="Courier"> "e:/RabbitMQ/db/rabbit-mnesia/queues/5MD5HQ7PCUA3C2UP3Z9HUANOV",</font></div>
<div><font face="Courier"> {{dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> []}}},</font></div>
<div><font face="Courier"> []},</font></div>
<div><font face="Courier"> undefined,0,65536,</font></div>
<div><font face="Courier"> #Fun<rabbit_variable_queue.2.81334491>,</font></div>
<div><font face="Courier"> {0,nil}},</font></div>
<div><font face="Courier"> {{client_msstate,msg_store_persistent,</font></div>
<div><font face="Courier"> <<139,43,92,149,89,56,222,175,208,46,119,138,91,</font></div>
<div><font face="Courier"> 209,199,19>>,</font></div>
<div><font face="Courier"> {dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> []}}},</font></div>
<div><font face="Courier"> {state,348236,</font></div>
<div><font face="Courier"> "e:/RabbitMQ/db/rabbit-mnesia/msg_store_persistent"},</font></div>
<div><font face="Courier"> rabbit_msg_store_ets_index,</font></div>
<div><font face="Courier"> "e:/RabbitMQ/db/rabbit-mnesia/msg_store_persistent",</font></div>
<div><font face="Courier"> <0.269.0>,352333,344139,356430,360527},</font></div>
<div><font face="Courier"> {client_msstate,msg_store_transient,</font></div>
<div><font face="Courier"> <<241,205,183,204,120,219,30,233,243,36,174,112,</font></div>
<div><font face="Courier"> 168,245,105,99>>,</font></div>
<div><font face="Courier"> {dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> []}}},</font></div>
<div><font face="Courier"> {state,327751,</font></div>
<div><font face="Courier"> "e:/RabbitMQ/db/rabbit-mnesia/msg_store_transient"},</font></div>
<div><font face="Courier"> rabbit_msg_store_ets_index,</font></div>
<div><font face="Courier"> "e:/RabbitMQ/db/rabbit-mnesia/msg_store_transient",</font></div>
<div><font face="Courier"> <0.264.0>,331848,323654,335945,340042}},</font></div>
<div><font face="Courier"> true,0,0,0,infinity,0,0,0,0,0,</font></div>
<div><font face="Courier"> {rates,</font></div>
<div><font face="Courier"> {{1379,109827,41270},0},</font></div>
<div><font face="Courier"> {{1379,109827,41270},0},</font></div>
<div><font face="Courier"> 0.0,0.0,</font></div>
<div><font face="Courier"> {1379,252056,729281}},</font></div>
<div><font face="Courier"> {0,nil},</font></div>
<div><font face="Courier"> {0,nil},</font></div>
<div><font face="Courier"> {0,nil},</font></div>
<div><font face="Courier"> {0,nil},</font></div>
<div><font face="Courier"> 0,0,</font></div>
<div><font face="Courier"> {rates,</font></div>
<div><font face="Courier"> {{1379,109827,41270},0},</font></div>
<div><font face="Courier"> {{1379,109827,41270},0},</font></div>
<div><font face="Courier"> 0.0,0.0,</font></div>
<div><font face="Courier"> {1379,252056,729281}}},</font></div>
<div><font face="Courier"> undefined,#Ref<0.0.55.61023>,</font></div>
<div><font face="Courier"> {dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},</font></div>
<div><font face="Courier"> {dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},</font></div>
<div><font face="Courier"> {dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},</font></div>
<div><font face="Courier"> {dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},</font></div>
<div><font face="Courier"> 0}</font></div>
<div><font face="Courier">** Reason for termination == </font></div>
<div><font face="Courier">** {{badmatch,{error,not_found}},</font></div>
<div><font face="Courier"> [{rabbit_amqqueue_process,i,2,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,'-infos/2-lc$^0/1-0-',2,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,'-infos/2-lc$^0/1-0-',2,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,emit_stats,2,[]},</font></div>
<div><font face="Courier"> {rabbit_event,if_enabled,3,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,'-terminate_delete/3-fun-1-',6,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,terminate_shutdown,2,[]},</font></div>
<div><font face="Courier"> {gen_server2,terminate,3,[]}]}</font></div>
<div><font face="Courier">** In 'terminate' callback with reason ==</font></div>
<div><font face="Courier">** normal</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=ERROR REPORT==== 15-Sep-2013::06:34:45 ===</font></div>
<div><font face="Courier">** Generic server <0.24484.16> terminating</font></div>
<div><font face="Courier">** Last message in was {init,new}</font></div>
<div><font face="Courier">** When Server state == {q,{amqqueue,</font></div>
<div><font face="Courier"> {resource,<<"vortex">>,queue,</font></div>
<div><font face="Courier"> <<"ha-execute_lookupproviderinregistryactivity">>},</font></div>
<div><font face="Courier"> true,false,none,</font></div>
<div><font face="Courier"> [{<<"x-ha-policy">>,longstr,<<"all">>}],</font></div>
<div><font face="Courier"> <0.24484.16>,[],[],</font></div>
<div><font face="Courier"> [{vhost,<<"vortex">>},</font></div>
<div><font face="Courier"> {name,<<"ha-policy">>},</font></div>
<div><font face="Courier"> {pattern,<<"^ha-.*">>},</font></div>
<div><font face="Courier"> {definition,</font></div>
<div><font face="Courier"> [{<<"ha-mode">>,<<"all">>},</font></div>
<div><font face="Courier"> {<<"ha-sync-mode">>,<<"automatic">>}]},</font></div>
<div><font face="Courier"> {priority,0}],</font></div>
<div><font face="Courier"> []},</font></div>
<div><font face="Courier"> none,false,undefined,undefined,</font></div>
<div><font face="Courier"> {[],[]},</font></div>
<div><font face="Courier"> undefined,undefined,undefined,undefined,</font></div>
<div><font face="Courier"> {state,fine,5000,undefined},</font></div>
<div><font face="Courier"> {0,nil},</font></div>
<div><font face="Courier"> undefined,undefined,undefined,</font></div>
<div><font face="Courier"> {dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> []},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> []}}},</font></div>
<div><font face="Courier"> undefined,undefined,undefined,running}</font></div>
<div><font face="Courier">** Reason for termination == </font></div>
<div><font face="Courier">** {{badmatch,{error,not_found}},</font></div>
<div><font face="Courier"> [{rabbit_amqqueue_process,i,2,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,'-infos/2-lc$^0/1-0-',2,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,'-infos/2-lc$^0/1-0-',2,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,emit_stats,2,[]},</font></div>
<div><font face="Courier"> {rabbit_event,if_enabled,3,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,declare,3,[]},</font></div>
<div><font face="Courier"> {gen_server2,handle_msg,2,[]},</font></div>
<div><font face="Courier"> {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=ERROR REPORT==== 15-Sep-2013::06:34:45 ===</font></div>
<div><font face="Courier">** Generic server <0.24482.16> terminating</font></div>
<div><font face="Courier">** Last message in was {'$gen_cast',</font></div>
<div><font face="Courier"> {method,</font></div>
<div><font face="Courier"> {'basic.consume',0,</font></div>
<div><font face="Courier"> <<"ha-execute_lookupproviderinregistryactivity">>,</font></div>
<div><font face="Courier"> <<>>,false,false,false,false,[]},</font></div>
<div><font face="Courier"> none,noflow}}</font></div>
<div><font face="Courier">** When Server state == {ch,running,rabbit_framing_amqp_0_9_1,2,<0.24463.16>,</font></div>
<div><font face="Courier"> <0.24480.16>,<0.24463.16>,</font></div>
<div><font face="Courier"> <<"10.100.183.10:55948 -> 10.100.183.75:5672">>,</font></div>
<div><font face="Courier"> {lstate,<0.24481.16>,true,false},</font></div>
<div><font face="Courier"> none,1,</font></div>
<div><font face="Courier"> {[],[]},</font></div>
<div><font face="Courier"> {user,<<"SJPERMQVORTEXWRITE">>,[],</font></div>
<div><font face="Courier"> rabbit_auth_backend_ldap,</font></div>
<div><font face="Courier"> {impl,</font></div>
<div><font face="Courier"> "CN=SJPERMQ VORTEXWRITE,OU=PERMQ,OU=PERFORMANCE,OU=Environments,OU=Servers,DC=RHB,DC=AD",</font></div>
<div><font face="Courier"> <<"5k!Kz_T86Xa.W~">>}},</font></div>
<div><font face="Courier"> <<"vortex">>,</font></div>
<div><font face="Courier"> <<"ha-execute_lookupproviderinregistryactivity">>,</font></div>
<div><font face="Courier"> {dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> []},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> [],[]}}},</font></div>
<div><font face="Courier"> {dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> []},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> [],[]}}},</font></div>
<div><font face="Courier"> {dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> []},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> [],[]}}},</font></div>
<div><font face="Courier"> {set,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> []},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> [],[]}}},</font></div>
<div><font face="Courier"> {dict,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> []},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> [],[]}}},</font></div>
<div><font face="Courier"> {set,0,16,16,8,80,48,</font></div>
<div><font face="Courier"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> []},</font></div>
<div><font face="Courier"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="Courier"> [],[]}}},</font></div>
<div><font face="Courier"> <0.24461.16>,</font></div>
<div><font face="Courier"> {state,fine,5000,#Ref<0.0.55.60968>},</font></div>
<div><font face="Courier"> false,1,</font></div>
<div><font face="Courier"> {{0,nil},{0,nil}},</font></div>
<div><font face="Courier"> [],</font></div>
<div><font face="Courier"> [{<<"publisher_confirms">>,bool,true},</font></div>
<div><font face="Courier"> {<<"exchange_exchange_bindings">>,bool,true},</font></div>
<div><font face="Courier"> {<<"consumer_cancel_notify">>,bool,true},</font></div>
<div><font face="Courier"> {<<"basic.nack">>,bool,true}],</font></div>
<div><font face="Courier"> none}</font></div>
<div><font face="Courier">** Reason for termination == </font></div>
<div><font face="Courier">** {{{{badmatch,{error,not_found}},</font></div>
<div><font face="Courier"> [{rabbit_amqqueue_process,i,2,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,'-infos/2-lc$^0/1-0-',2,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,'-infos/2-lc$^0/1-0-',2,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,emit_stats,2,[]},</font></div>
<div><font face="Courier"> {rabbit_event,if_enabled,3,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,declare,3,[]},</font></div>
<div><font face="Courier"> {gen_server2,handle_msg,2,[]},</font></div>
<div><font face="Courier"> {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]},</font></div>
<div><font face="Courier"> {gen_server2,call,</font></div>
<div><font face="Courier"> [<0.24484.16>,</font></div>
<div><font face="Courier"> {basic_consume,false,<0.24482.16>,<0.24481.16>,true,</font></div>
<div><font face="Courier"> <<"amq.ctag-sPY5Ih3rfTx5gxibnPoXkw">>,false,none,</font></div>
<div><font face="Courier"> {'basic.consume_ok',<<"amq.ctag-sPY5Ih3rfTx5gxibnPoXkw">>}},</font></div>
<div><font face="Courier"> infinity]}},</font></div>
<div><font face="Courier"> [{gen_server2,call,3,[]},</font></div>
<div><font face="Courier"> {rabbit_channel,'-handle_method/3-fun-1-',7,[]},</font></div>
<div><font face="Courier"> {rabbit_misc,with_exit_handler,2,[]},</font></div>
<div><font face="Courier"> {rabbit_channel,handle_method,3,[]},</font></div>
<div><font face="Courier"> {rabbit_channel,handle_cast,2,[]},</font></div>
<div><font face="Courier"> {gen_server2,handle_msg,2,[]},</font></div>
<div><font face="Courier"> {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=ERROR REPORT==== 15-Sep-2013::06:34:45 ===</font></div>
<div><font face="Courier">AMQP connection <0.24463.16> (running), channel 2 - error:</font></div>
<div><font face="Courier">{{{{badmatch,{error,not_found}},</font></div>
<div><font face="Courier"> [{rabbit_amqqueue_process,i,2,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,'-infos/2-lc$^0/1-0-',2,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,'-infos/2-lc$^0/1-0-',2,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,emit_stats,2,[]},</font></div>
<div><font face="Courier"> {rabbit_event,if_enabled,3,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,declare,3,[]},</font></div>
<div><font face="Courier"> {gen_server2,handle_msg,2,[]},</font></div>
<div><font face="Courier"> {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]},</font></div>
<div><font face="Courier"> {gen_server2,call,</font></div>
<div><font face="Courier"> [<0.24484.16>,</font></div>
<div><font face="Courier"> {basic_consume,false,<0.24482.16>,<0.24481.16>,true,</font></div>
<div><font face="Courier"> <<"amq.ctag-sPY5Ih3rfTx5gxibnPoXkw">>,false,none,</font></div>
<div><font face="Courier"> {'basic.consume_ok',<<"amq.ctag-sPY5Ih3rfTx5gxibnPoXkw">>}},</font></div>
<div><font face="Courier"> infinity]}},</font></div>
<div><font face="Courier"> [{gen_server2,call,3,[]},</font></div>
<div><font face="Courier"> {rabbit_channel,'-handle_method/3-fun-1-',7,[]},</font></div>
<div><font face="Courier"> {rabbit_misc,with_exit_handler,2,[]},</font></div>
<div><font face="Courier"> {rabbit_channel,handle_method,3,[]},</font></div>
<div><font face="Courier"> {rabbit_channel,handle_cast,2,[]},</font></div>
<div><font face="Courier"> {gen_server2,handle_msg,2,[]},</font></div>
<div><font face="Courier"> {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,239}]}]}</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=WARNING REPORT==== 15-Sep-2013::06:34:45 ===</font></div>
<div><font face="Courier">Non-AMQP exit reason '{{{{badmatch,{error,not_found}},</font></div>
<div><font face="Courier"> [{rabbit_amqqueue_process,i,2,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,'-infos/2-lc$^0/1-0-',2,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,'-infos/2-lc$^0/1-0-',2,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,emit_stats,2,[]},</font></div>
<div><font face="Courier"> {rabbit_event,if_enabled,3,[]},</font></div>
<div><font face="Courier"> {rabbit_amqqueue_process,declare,3,[]},</font></div>
<div><font face="Courier"> {gen_server2,handle_msg,2,[]},</font></div>
<div><font face="Courier"> {proc_lib,init_p_do_apply,3,</font></div>
<div><font face="Courier"> [{file,"proc_lib.erl"},{line,239}]}]},</font></div>
<div><font face="Courier"> {gen_server2,call,</font></div>
<div><font face="Courier"> [<0.24484.16>,</font></div>
<div><font face="Courier"> {basic_consume,false,<0.24482.16>,<0.24481.16>,</font></div>
<div><font face="Courier"> true,<<"amq.ctag-sPY5Ih3rfTx5gxibnPoXkw">>,</font></div>
<div><font face="Courier"> false,none,</font></div>
<div><font face="Courier"> {'basic.consume_ok',</font></div>
<div><font face="Courier"> <<"amq.ctag-sPY5Ih3rfTx5gxibnPoXkw">>}},</font></div>
<div><font face="Courier"> infinity]}},</font></div>
<div><font face="Courier"> [{gen_server2,call,3,[]},</font></div>
<div><font face="Courier"> {rabbit_channel,'-handle_method/3-fun-1-',7,[]},</font></div>
<div><font face="Courier"> {rabbit_misc,with_exit_handler,2,[]},</font></div>
<div><font face="Courier"> {rabbit_channel,handle_method,3,[]},</font></div>
<div><font face="Courier"> {rabbit_channel,handle_cast,2,[]},</font></div>
<div><font face="Courier"> {gen_server2,handle_msg,2,[]},</font></div>
<div><font face="Courier"> {proc_lib,init_p_do_apply,3,</font></div>
<div><font face="Courier"> [{file,"proc_lib.erl"},{line,239}]}]}'</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=INFO REPORT==== 15-Sep-2013::06:34:45 ===</font></div>
<div><font face="Courier">closing AMQP connection <0.24463.16> (10.100.183.10:55948 -> 10.100.183.75:5672)</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">=WARNING REPORT==== 15-Sep-2013::06:34:47 ===</font></div>
<div><font face="Courier">memory resource limit alarm set on node rabbit@SJPERMQ04.</font></div>
<div><font face="Courier"><br>
</font></div>
<div><font face="Courier">**********************************************************</font></div>
<div><font face="Courier">*** Publishers will be blocked until this alarm clears ***</font></div>
<div><font face="Courier">**********************************************************</font></div>
<div style="font-family: Calibri, sans-serif; "><br>
</div>
</div>
<div style="font-family: Calibri, sans-serif; "><br>
</div>
<div style="font-family: Calibri, sans-serif; "><br>
</div>
<div style="font-family: Calibri, sans-serif; "><br>
</div>
<div style="font-family: Calibri, sans-serif; "><br>
</div>
<div style="font-family: Calibri, sans-serif; "><br>
</div>
<div style="font-family: Calibri, sans-serif; "><br>
</div>
<div style="font-family: Calibri, sans-serif; "><br>
</div>
</body>
</html>