<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Matt,<br>
<br>
This is a problem we're aware of (i.e., we have an open bug which
we're working on!) and are looking at ways of handling it. What is
basically happening here is that mnesia gets into an inconsistent
state due to a network partition and the mnesia transaction manager
sends unanticipated (and unhandled) messages to certain Erlang
processes causing a crash.<br>
<br>
Whilst this falls into the category of "Rabbit clusters do not
handle network partitions" obviously we want to prevent this sort of
thing happening and as I say, we're looking at various ways of
responding in a somewhat more user friendly manner.<br>
<br>
In the meanwhile, I'd be very interested to see if the queue
information has actually been deleted from mnesia when this happens.
Are these HA queues durable and do they all "come back to life"
after restarting the node(s)?<br>
<br>
Cheers,<br>
Tim<br>
<br>
On 10/12/2012 09:01 PM, Matt Pietrek wrote:
<blockquote
cite="mid:CACLE7iyF8XFc2hGmy0YyaUJmGkV05dDZ6v5dXOsTJeq_6+KcKg@mail.gmail.com"
type="cite">I'll be the first to admit that this is a somewhat
unusual/complicated circumstance, but I'm posting this on the
chance there's something unusual. This is RabbitMQ is <b>2.8.7 </b>on
Erlang R15B01.<br>
<br>
We have two brokers (mq1, mq2) running clustered with all HA
queues. A VIP is in front of them. <br>
<br>
At some point there was a network event, and afterwards, a queue
vanished. (Still trying to figure out exactly what the network
hiccup was, but it's fair to say there was a communication
disruption between the nodes.) My guess from looking at the logs
is that each node had a moment where it couldn't see the other, so
took actions that were ultimately incompatible when both nodes
were visible to each other.<br>
<br>
The actual error as seen by our code is a missing queue named
"charon". You'll see it near the end of the <a class="moz-txt-link-abbreviated" href="mailto:rabbit@mq2.log">rabbit@mq2.log</a>
(below):<br>
<br>
<b><span style="color:rgb(255,0,0)">{amqp_error,not_found,"no
queue 'charon' in vhost '/'",'queue.declare'}</span></b><br>
<br>
Prior to the network event, our client was successfully reading
from the charon queue.<br>
<br>
<br>
<b>Here's the relevant parts of the <a class="moz-txt-link-abbreviated" href="mailto:rabbit@mq1.log">rabbit@mq1.log</a> file<br>
====================================================</b><br>
=INFO REPORT==== 11-Oct-2012::14:01:58 ===<br>
rabbit on node rabbit@mq2 down<br>
<br>
=ERROR REPORT==== 11-Oct-2012::14:02:03 ===<br>
Mnesia(rabbit@mq1): ** ERROR ** mnesia_event got
{inconsistent_database, running_partitioned_network, rabbit@mq2}<br>
<br>
=ERROR REPORT==== 11-Oct-2012::14:02:03 ===<br>
** Generic server <0.547.0> terminating<br>
** Last message in was {mnesia_locker,rabbit@mq2,granted}<br>
** When Server state == {state,<br>
{0,<0.547.0>},<br>
{{0,<0.547.0>},undefined},<br>
{{0,<0.547.0>},undefined},<br>
{resource,<<"/">>,queue,<<"charon">>},<br>
rabbit_mirror_queue_coordinator,<br>
{3,<br>
[{{0,<0.547.0>},<br>
{view_member,<br>
{0,<0.547.0>},<br>
[],<br>
{0,<0.547.0>},<br>
{0,<0.547.0>}}}]},<br>
1,[],<br>
[<0.546.0>],<br>
{[],[]},<br>
[],undefined}<br>
** Reason for termination == <br>
** {function_clause,<br>
[{gm,handle_info,<br>
[{mnesia_locker,rabbit@mq2,granted},<br>
{state,<br>
{0,<0.547.0>},<br>
{{0,<0.547.0>},undefined},<br>
{{0,<0.547.0>},undefined},<br>
{resource,<<"/">>,queue,<<"charon">>},<br>
rabbit_mirror_queue_coordinator,<br>
{3,<br>
[{{0,<0.547.0>},<br>
{view_member,<br>
{0,<0.547.0>},<br>
[],<br>
{0,<0.547.0>},<br>
{0,<0.547.0>}}}]},<br>
1,[],<br>
[<0.546.0>],<br>
{[],[]},<br>
[],undefined}],<br>
[{file,"src/gm.erl"},{line,665}]},<br>
{gen_server2,handle_msg,2,[{file,"src/gen_server2.erl"},{line,912}]},<br>
{proc_lib,wake_up,3,[{file,"proc_lib.erl"},{line,237}]}]}<br>
<br>
=ERROR REPORT==== 11-Oct-2012::14:02:03 ===<br>
** Generic server <0.545.0> terminating<br>
** Last message in was {'EXIT',<0.546.0>,<br>
{function_clause,<br>
[{gm,handle_info,<br>
[{mnesia_locker,rabbit@mq2,granted},<br>
{state,<br>
{0,<0.547.0>},<br>
{{0,<0.547.0>},undefined},<br>
{{0,<0.547.0>},undefined},<br>
{resource,<<"/">>,queue,<<"charon">>},<br>
rabbit_mirror_queue_coordinator,<br>
{3,<br>
[{{0,<0.547.0>},<br>
{view_member,<br>
{0,<0.547.0>},<br>
[],<br>
{0,<0.547.0>},<br>
{0,<0.547.0>}}}]},<br>
1,[],<br>
[<0.546.0>],<br>
{[],[]},<br>
[],undefined}],<br>
[{file,"src/gm.erl"},{line,665}]},<br>
{gen_server2,handle_msg,2,<br>
[{file,"src/gen_server2.erl"},{line,912}]},<br>
{proc_lib,wake_up,3,<br>
[{file,"proc_lib.erl"},{line,237}]}]}}<br>
** When Server state == {q,<br>
{amqqueue,<br>
{resource,<<"/">>,queue,<<"charon">>},<br>
true,false,none,<br>
[{<<"x-ha-policy">>,longstr,<<"all">>}],<br>
<0.545.0>,[],all},<br>
none,false,rabbit_mirror_queue_master,<br>
{state,<0.547.0>,<0.546.0>,rabbit_variable_queue,<br>
{vqstate,<br>
{0,{[],[]}},<br>
{0,{[],[]}},<br>
{delta,undefined,0,undefined},<br>
{0,{[],[]}},<br>
{0,{[],[]}},<br>
0,<br>
{0,nil},<br>
undefined,<br>
{0,nil},<br>
{qistate,<br>
"/highland/var/lib/rabbit@mq1/queues/4L2SY4TVRPM8HP85JDVO85BPL",<br>
{{dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]}}},<br>
[]},<br>
undefined,0,262144,<br>
#Fun<rabbit_variable_queue.2.121500735>,<br>
{0,nil}},<br>
{{client_msstate,msg_store_persistent,<br>
<<112,32,84,92,54,73,67,43,231,35,164,79,106,8,27,<br>
132>>,<br>
{dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]}}},<br>
{state,237637,<br>
"/highland/var/lib/rabbit@mq1/msg_store_persistent"},<br>
rabbit_msg_store_ets_index,<br>
"/highland/var/lib/rabbit@mq1/msg_store_persistent",<br>
<0.289.0>,241734,233540,245831,249928},<br>
{client_msstate,msg_store_transient,<br>
<<204,18,155,40,29,120,124,76,144,83,195,135,250,<br>
91,216,3>>,<br>
{dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]}}},<br>
{state,217152,<br>
"/highland/var/lib/rabbit@mq1/msg_store_transient"},<br>
rabbit_msg_store_ets_index,<br>
"/highland/var/lib/rabbit@mq1/msg_store_transient",<br>
<0.284.0>,221249,213055,225346,229443}},<br>
true,0,#Fun<rabbit_amqqueue_process.5.24780734>,0,<br>
0,infinity,0,0,0,0,0,<br>
{rates,<br>
{{1349,988602,967125},0},<br>
{{1349,988602,967125},0},<br>
0.0,0.0,<br>
{1349,988606,141019}},<br>
{0,nil},<br>
{0,nil},<br>
{0,nil},<br>
{0,nil},<br>
0,0,<br>
{rates,<br>
{{1349,988602,967125},0},<br>
{{1349,988602,967125},0},<br>
0.0,0.0,<br>
{1349,988606,141019}}},<br>
0,<br>
{dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]}}},<br>
[],<br>
{dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]}}},<br>
{set,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]}}}},<br>
{[],[]},<br>
undefined,undefined,undefined,undefined,<br>
{state,fine,5000,undefined},<br>
{0,nil},<br>
undefined,undefined,undefined,<br>
{dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},<br>
1,<br>
{{0,nil},{0,nil}},<br>
undefined,<br>
{dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},<br>
undefined,undefined}<br>
** Reason for termination == <br>
** {noproc,{gen_server2,call,[<0.547.0>,info,infinity]}}<br>
** In 'terminate' callback with reason ==<br>
** {function_clause,<br>
[{gm,handle_info,<br>
[{mnesia_locker,rabbit@mq2,granted},<br>
{state,<br>
{0,<0.547.0>},<br>
{{0,<0.547.0>},undefined},<br>
{{0,<0.547.0>},undefined},<br>
{resource,<<"/">>,queue,<<"charon">>},<br>
rabbit_mirror_queue_coordinator,<br>
{3,<br>
[{{0,<0.547.0>},<br>
{view_member,<br>
{0,<0.547.0>},<br>
[],<br>
{0,<0.547.0>},<br>
{0,<0.547.0>}}}]},<br>
1,[],<br>
[<0.546.0>],<br>
{[],[]},<br>
[],undefined}],<br>
[{file,"src/gm.erl"},{line,665}]},<br>
{gen_server2,handle_msg,2,[{file,"src/gen_server2.erl"},{line,912}]},<br>
{proc_lib,wake_up,3,[{file,"proc_lib.erl"},{line,237}]}]}<br>
<br>
=INFO REPORT==== 11-Oct-2012::14:02:03 ===<br>
Mirrored-queue (queue 'cmcmd' in vhost '/'): Master
<a class="moz-txt-link-rfc2396E" href="mailto:rabbit@mq1.2.665.0"><rabbit@mq1.2.665.0></a> saw deaths of mirrors
<a class="moz-txt-link-rfc2396E" href="mailto:rabbit@mq2.1.383.0"><rabbit@mq2.1.383.0></a> <br>
<br>
=INFO REPORT==== 11-Oct-2012::14:02:03 ===<br>
Mirrored-queue (queue 'ConfigurationManager' in vhost '/'): Master
<a class="moz-txt-link-rfc2396E" href="mailto:rabbit@mq1.2.605.0"><rabbit@mq1.2.605.0></a> saw deaths of mirrors
<a class="moz-txt-link-rfc2396E" href="mailto:rabbit@mq2.1.377.0"><rabbit@mq2.1.377.0></a> <br>
<br>
=INFO REPORT==== 11-Oct-2012::14:02:03 ===<br>
global: Name conflict terminating
{rabbit_mgmt_db,<3198.32532.0>}<br>
<br>
=INFO REPORT==== 11-Oct-2012::14:26:40 ===<br>
accepting AMQP connection <0.6507.1> (<a
moz-do-not-send="true" href="http://10.72.0.50:46742">10.72.0.50:46742</a>
-> <a moz-do-not-send="true" href="http://10.72.0.47:5672">10.72.0.47:5672</a>)<br>
<br>
=INFO REPORT==== 11-Oct-2012::14:26:40 ===<br>
closing AMQP connection <0.6507.1> (<a
moz-do-not-send="true" href="http://10.72.0.50:46742">10.72.0.50:46742</a>
-> <a moz-do-not-send="true" href="http://10.72.0.47:5672">10.72.0.47:5672</a>)<br>
<br>
<br>
<br>
<br>
<b>And here's the relevant parts of the <a class="moz-txt-link-abbreviated" href="mailto:rabbit@m2.log">rabbit@m2.log</a> file<br>
====================================================</b><br>
=ERROR REPORT==== 11-Oct-2012::14:01:37 ===<br>
** Node rabbit@mq1 not responding **<br>
** Removing (timedout) connection **<br>
<br>
=INFO REPORT==== 11-Oct-2012::14:01:37 ===<br>
rabbit on node rabbit@mq1 down<br>
<br>
=INFO REPORT==== 11-Oct-2012::14:01:40 ===<br>
Statistics database started.<br>
<br>
=INFO REPORT==== 11-Oct-2012::14:01:43 ===<br>
Mirrored-queue (queue 'cmcmd' in vhost '/'): Slave
<a class="moz-txt-link-rfc2396E" href="mailto:rabbit@mq2.1.383.0"><rabbit@mq2.1.383.0></a> saw deaths of mirrors
<a class="moz-txt-link-rfc2396E" href="mailto:rabbit@mq1.2.665.0"><rabbit@mq1.2.665.0></a> <br>
<br>
=INFO REPORT==== 11-Oct-2012::14:01:43 ===<br>
Mirrored-queue (queue 'cmcmd' in vhost '/'): Promoting slave
<a class="moz-txt-link-rfc2396E" href="mailto:rabbit@mq2.1.383.0"><rabbit@mq2.1.383.0></a> to master<br>
<br>
=INFO REPORT==== 11-Oct-2012::14:01:43 ===<br>
Mirrored-queue (queue 'charon' in vhost '/'): Slave
<a class="moz-txt-link-rfc2396E" href="mailto:rabbit@mq2.1.371.0"><rabbit@mq2.1.371.0></a> saw deaths of mirrors
<a class="moz-txt-link-rfc2396E" href="mailto:rabbit@mq1.2.545.0"><rabbit@mq1.2.545.0></a> <br>
<br>
=INFO REPORT==== 11-Oct-2012::14:01:43 ===<br>
Mirrored-queue (queue 'charon' in vhost '/'): Promoting slave
<a class="moz-txt-link-rfc2396E" href="mailto:rabbit@mq2.1.371.0"><rabbit@mq2.1.371.0></a> to master<br>
<br>
=INFO REPORT==== 11-Oct-2012::14:01:43 ===<br>
Mirrored-queue (queue 'ConfigurationManager' in vhost '/'): Slave
<a class="moz-txt-link-rfc2396E" href="mailto:rabbit@mq2.1.377.0"><rabbit@mq2.1.377.0></a> saw deaths of mirrors
<a class="moz-txt-link-rfc2396E" href="mailto:rabbit@mq1.2.605.0"><rabbit@mq1.2.605.0></a> <br>
<br>
=INFO REPORT==== 11-Oct-2012::14:01:43 ===<br>
Mirrored-queue (queue 'ConfigurationManager' in vhost '/'):
Promoting slave <a class="moz-txt-link-rfc2396E" href="mailto:rabbit@mq2.1.377.0"><rabbit@mq2.1.377.0></a> to master<br>
<br>
=ERROR REPORT==== 11-Oct-2012::14:05:03 ===<br>
Mnesia(rabbit@mq2): ** ERROR ** mnesia_event got
{inconsistent_database, running_partitioned_network, rabbit@mq1}<br>
<br>
=INFO REPORT==== 11-Oct-2012::14:05:03 ===<br>
Statistics database started.<br>
<br>
=ERROR REPORT==== 11-Oct-2012::14:29:39 ===<br>
** Generic server <0.32583.0> terminating<br>
** Last message in was
{'$gen_cast',{gm_deaths,[<0.5504.1>]}}<br>
** When Server state == {state,<br>
{amqqueue,<br>
{resource,<<"/">>,queue,<<"charon">>},<br>
true,false,none,<br>
[{<<"x-ha-policy">>,longstr,<<"all">>}],<br>
<0.371.0>,[],all},<br>
<0.372.0>,<br>
{dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[],[]}}},<br>
#Fun<rabbit_mirror_queue_master.1.32244930>,<br>
#Fun<rabbit_mirror_queue_master.2.32244930>}<br>
** Reason for termination == <br>
** {bad_return_value,<br>
{error,<br>
{{badmatch,[]},<br>
[{rabbit_mirror_queue_misc,'-remove_from_queue/2-fun-0-',2,<br>
[{file,"src/rabbit_mirror_queue_misc.erl"},{line,63}]},<br>
{mnesia_tm,apply_fun,3,[{file,"mnesia_tm.erl"},{line,829}]},<br>
{mnesia_tm,execute_transaction,5,<br>
[{file,"mnesia_tm.erl"},{line,809}]},<br>
{rabbit_misc,'-execute_mnesia_transaction/1-fun-0-',1,<br>
[{file,"src/rabbit_misc.erl"},{line,440}]},<br>
{worker_pool_worker,handle_call,3,<br>
[{file,"src/worker_pool_worker.erl"},{line,80}]},<br>
{gen_server2,handle_msg,2,<br>
[{file,"src/gen_server2.erl"},{line,879}]},<br>
{proc_lib,wake_up,3,[{file,"proc_lib.erl"},{line,237}]}]}}}<br>
<br>
=ERROR REPORT==== 11-Oct-2012::14:29:39 ===<br>
** Generic server <0.371.0> terminating<br>
** Last message in was {'EXIT',<0.32583.0>,<br>
{bad_return_value,<br>
{error,<br>
{{badmatch,[]},<br>
[{rabbit_mirror_queue_misc,<br>
'-remove_from_queue/2-fun-0-',2,<br>
[{file,"src/rabbit_mirror_queue_misc.erl"},<br>
{line,63}]},<br>
{mnesia_tm,apply_fun,3,<br>
[{file,"mnesia_tm.erl"},{line,829}]},<br>
{mnesia_tm,execute_transaction,5,<br>
[{file,"mnesia_tm.erl"},{line,809}]},<br>
{rabbit_misc,<br>
'-execute_mnesia_transaction/1-fun-0-',1,<br>
[{file,"src/rabbit_misc.erl"},{line,440}]},<br>
{worker_pool_worker,handle_call,3,<br>
[{file,"src/worker_pool_worker.erl"},{line,80}]},<br>
{gen_server2,handle_msg,2,<br>
[{file,"src/gen_server2.erl"},{line,879}]},<br>
{proc_lib,wake_up,3,<br>
[{file,"proc_lib.erl"},{line,237}]}]}}}}<br>
** When Server state == {q,<br>
{amqqueue,<br>
{resource,<<"/">>,queue,<<"charon">>},<br>
true,false,none,<br>
[{<<"x-ha-policy">>,longstr,<<"all">>}],<br>
<0.371.0>,[],all},<br>
none,false,rabbit_mirror_queue_master,<br>
{state,<0.372.0>,<0.32583.0>,rabbit_variable_queue,<br>
{vqstate,<br>
{0,{[],[]}},<br>
{0,{[],[]}},<br>
{delta,undefined,0,undefined},<br>
{0,{[],[]}},<br>
{0,{[],[]}},<br>
0,<br>
{0,nil},<br>
undefined,<br>
{0,nil},<br>
{qistate,<br>
"/highland/var/lib/rabbit@mq2/queues/4L2SY4TVRPM8HP85JDVO85BPL",<br>
{{dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]}}},<br>
[]},<br>
undefined,0,262144,<br>
#Fun<rabbit_variable_queue.2.121500735>,<br>
{0,nil}},<br>
{{client_msstate,msg_store_persistent,<br>
<<23,245,132,154,3,96,225,184,108,75,232,224,184,<br>
114,108,141>>,<br>
{dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]}}},<br>
{state,282693,<br>
"/highland/var/lib/rabbit@mq2/msg_store_persistent"},<br>
rabbit_msg_store_ets_index,<br>
"/highland/var/lib/rabbit@mq2/msg_store_persistent",<br>
<0.285.0>,286790,278596,290887,294984},<br>
{client_msstate,msg_store_transient,<br>
<<17,62,75,125,126,21,66,37,170,44,198,72,18,94,<br>
170,197>>,<br>
{dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]}}},<br>
{state,262208,<br>
"/highland/var/lib/rabbit@mq2/msg_store_transient"},<br>
rabbit_msg_store_ets_index,<br>
"/highland/var/lib/rabbit@mq2/msg_store_transient",<br>
<0.280.0>,266305,258111,270402,274499}},<br>
true,0,#Fun<rabbit_mirror_queue_slave.1.69615989>,<br>
0,0,infinity,0,0,0,0,0,<br>
{rates,<br>
{{1349,989325,677031},0},<br>
{{1349,989325,677031},0},<br>
0.0,0.0,<br>
{1349,989505,348090}},<br>
{0,nil},<br>
{0,nil},<br>
{0,nil},<br>
{0,nil},<br>
0,0,<br>
{rates,<br>
{{1349,989325,677031},0},<br>
{{1349,989325,677031},0},<br>
0.0,0.0,<br>
{1349,989505,348090}}},<br>
0,<br>
{dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]}}},<br>
[],<br>
{dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]}}},<br>
{set,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]}}}},<br>
{[],[]},<br>
undefined,undefined,undefined,undefined,<br>
{state,fine,5000,undefined},<br>
{0,nil},<br>
undefined,undefined,undefined,<br>
{dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},<br>
1,<br>
{{0,nil},{0,nil}},<br>
undefined,<br>
{dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},<br>
undefined,undefined}<br>
** Reason for termination == <br>
** {noproc,{gen_server2,call,[<0.372.0>,info,infinity]}}<br>
** In 'terminate' callback with reason ==<br>
** {bad_return_value,<br>
{error,<br>
{{badmatch,[]},<br>
[{rabbit_mirror_queue_misc,'-remove_from_queue/2-fun-0-',2,<br>
[{file,"src/rabbit_mirror_queue_misc.erl"},{line,63}]},<br>
{mnesia_tm,apply_fun,3,[{file,"mnesia_tm.erl"},{line,829}]},<br>
{mnesia_tm,execute_transaction,5,<br>
[{file,"mnesia_tm.erl"},{line,809}]},<br>
{rabbit_misc,'-execute_mnesia_transaction/1-fun-0-',1,<br>
[{file,"src/rabbit_misc.erl"},{line,440}]},<br>
{worker_pool_worker,handle_call,3,<br>
[{file,"src/worker_pool_worker.erl"},{line,80}]},<br>
{gen_server2,handle_msg,2,<br>
[{file,"src/gen_server2.erl"},{line,879}]},<br>
{proc_lib,wake_up,3,[{file,"proc_lib.erl"},{line,237}]}]}}}<br>
<br>
=INFO REPORT==== 11-Oct-2012::14:29:43 ===<br>
accepting AMQP connection <0.5524.1> (<a
moz-do-not-send="true" href="http://10.72.0.23:36056">10.72.0.23:36056</a>
-> <a moz-do-not-send="true" href="http://10.72.0.56:5672">10.72.0.56:5672</a>)<br>
<br>
=INFO REPORT==== 11-Oct-2012::14:29:44 ===<br>
accepting AMQP connection <0.5574.1> (<a
moz-do-not-send="true" href="http://10.72.0.23:36064">10.72.0.23:36064</a>
-> <a moz-do-not-send="true" href="http://10.72.0.56:5672">10.72.0.56:5672</a>)<br>
<br>
=INFO REPORT==== 11-Oct-2012::17:56:16 ===<br>
accepting AMQP connection <0.23868.1> (<a
moz-do-not-send="true" href="http://10.72.0.53:57122">10.72.0.53:57122</a>
-> <a moz-do-not-send="true" href="http://10.72.0.56:5672">10.72.0.56:5672</a>)<br>
<br>
=ERROR REPORT==== 11-Oct-2012::17:56:16 ===<br>
connection <0.23868.1>, channel 1 - error:<br>
{amqp_error,not_found,"no queue 'charon' in vhost
'/'",'queue.declare'}<br>
<br>
=WARNING REPORT==== 11-Oct-2012::17:56:16 ===<br>
closing AMQP connection <0.23868.1> (<a
moz-do-not-send="true" href="http://10.72.0.53:57122">10.72.0.53:57122</a>
-> <a moz-do-not-send="true" href="http://10.72.0.56:5672">10.72.0.56:5672</a>):<br>
connection_closed_abruptly<br>
<br>
=INFO REPORT==== 11-Oct-2012::17:56:16 ===<br>
accepting AMQP connection <0.23878.1> (<a
moz-do-not-send="true" href="http://10.72.0.52:56025">10.72.0.52:56025</a>
-> <a moz-do-not-send="true" href="http://10.72.0.56:5672">10.72.0.56:5672</a>)<br>
<br>
=ERROR REPORT==== 11-Oct-2012::17:56:16 ===<br>
connection <0.23878.1>, channel 1 - error:<br>
{amqp_error,not_found,"no queue 'charon' in vhost
'/'",'queue.declare'}<br>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
rabbitmq-discuss mailing list
<a class="moz-txt-link-abbreviated" href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a>
<a class="moz-txt-link-freetext" href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a>
</pre>
</blockquote>
<br>
</body>
</html>