<div class="gmail_extra">Ok, I got it again. <br>3 nodes. Stop slave, restart app(client), bring slave back online, stop master, restart app, bring master back online, BOOM!<br>So here are logs from both nodes, the one which I shutdown (dev-2) and the one which failed unexpectedly (dev-3)<br>
<br>dev-2 was the master node and when I stopped it, it brought dev-3 node down with it.<br>Did everything started because of this?<br>> Reason: reached_max_restart_intensity<br><br>Vadim<br><br>dev-3 log :<br>===============================================<br>
=INFO REPORT==== 26-Apr-2012::16:50:38 ===<br>Management agent started.<br><br>=INFO REPORT==== 26-Apr-2012::16:50:38 ===<br>Management plugin started. Port: 55672, path: /<br><br>=INFO REPORT==== 26-Apr-2012::16:51:07 ===<br>
application: rabbitmq_management<br> exited: shutdown<br> type: permanent<br><br>=INFO REPORT==== 26-Apr-2012::16:51:08 ===<br>stopped TCP Listener on <a href="http://0.0.0.0:5672">0.0.0.0:5672</a><br>===============================================<br>
<br>dev-3-sasl.log<br>===============================================<br>=SUPERVISOR REPORT==== 26-Apr-2012::16:51:07 ===<br> Supervisor: {local,rabbit_mgmt_sup}<br> Context: child_terminated<br> Reason: shutdown<br>
Offender: [{pid,<0.396.0>},<br> {name,mirroring},<br> {mfa,<br> {mirrored_supervisor,start_internal,<br> [rabbit_mgmt_sup,<br> [{rabbit_mgmt_db,<br>
{rabbit_mgmt_db,start_link,[]},<br> permanent,4294967295,worker,<br> [rabbit_mgmt_db]}]]}},<br> {restart_type,permanent},<br>
{shutdown,4294967295},<br> {child_type,worker}]<br><br><br>=SUPERVISOR REPORT==== 26-Apr-2012::16:51:07 ===<br> Supervisor: {local,rabbit_mgmt_sup}<br> Context: shutdown<br> Reason: reached_max_restart_intensity<br>
Offender: [{pid,<0.396.0>},<br> {name,mirroring},<br> {mfa,<br> {mirrored_supervisor,start_internal,<br> [rabbit_mgmt_sup,<br> [{rabbit_mgmt_db,<br>
{rabbit_mgmt_db,start_link,[]},<br> permanent,4294967295,worker,<br> [rabbit_mgmt_db]}]]}},<br> {restart_type,permanent},<br>
{shutdown,4294967295},<br> {child_type,worker}]<br><br>===============================================<br><br>dev-2 log<br>===============================================<br>=INFO REPORT==== 26-Apr-2012::16:51:07 ===<br>
Stopping Rabbit<br><br>=INFO REPORT==== 26-Apr-2012::16:51:07 ===<br> application: rabbitmq_management_visualiser<br> exited: stopped<br> type: permanent<br><br>=INFO REPORT==== 26-Apr-2012::16:51:07 ===<br> application: rabbitmq_management<br>
exited: stopped<br> type: permanent<br><br>=INFO REPORT==== 26-Apr-2012::16:51:07 ===<br> application: rabbitmq_management_agent<br> exited: stopped<br> type: permanent<br><br>=INFO REPORT==== 26-Apr-2012::16:51:07 ===<br>
stopped TCP Listener on <a href="http://0.0.0.0:5672">0.0.0.0:5672</a><br><br>=INFO REPORT==== 26-Apr-2012::16:51:08 ===<br>rabbit on node 'rabbit@rabbitmq-dev-3' down<br><br>=INFO REPORT==== 26-Apr-2012::16:51:08 ===<br>
Mirrored-queue (queue 'Advertiser-ONYX' in vhost '/'): Slave <'rabbit@rabbitmq-dev-2'.1.265.0> saw deaths of mirrors <'rabbit@rabbitmq-dev-3'.2.290.0> <br><br>=INFO REPORT==== 26-Apr-2012::16:51:08 ===<br>
Mirrored-queue (queue 'Search-JROSE6420' in vhost '/'): Slave <'rabbit@rabbitmq-dev-2'.1.276.0> saw deaths of mirrors <'rabbit@rabbitmq-dev-3'.2.328.0> <br><br>=INFO REPORT==== 26-Apr-2012::16:51:08 ===<br>
Mirrored-queue (queue 'Advertiser-DHOHMEYERGX280' in vhost '/'): Slave <'rabbit@rabbitmq-dev-2'.1.268.0> saw deaths of mirrors <'rabbit@rabbitmq-dev-3'.2.288.0> <br><br>=INFO REPORT==== 26-Apr-2012::16:51:08 ===<br>
Mirrored-queue (queue 'CustomerAdmin-DHOHMEYERGX280' in vhost '/'): Slave <'rabbit@rabbitmq-dev-2'.1.270.0> saw deaths of mirrors <'rabbit@rabbitmq-dev-3'.2.294.0> <br><br>=INFO REPORT==== 26-Apr-2012::16:51:08 ===<br>
Mirrored-queue (queue 'Tracking-JROSE6420' in vhost '/'): Slave <'rabbit@rabbitmq-dev-2'.1.250.0> saw deaths of mirrors <'rabbit@rabbitmq-dev-3'.2.247.0> <br><br>=ERROR REPORT==== 26-Apr-2012::16:51:08 ===<br>
** Generic server rabbit_mirror_queue_slave_sup terminating <br>** Last message in was {'EXIT',<0.127.0>,shutdown}<br>** When Server state == {state,<br> {local,rabbit_mirror_queue_slave_sup},<br>
simple_one_for_one_terminate,<br> [{child,undefined,rabbit_mirror_queue_slave,<br> {rabbit_mirror_queue_slave,start_link,[]},<br> temporary,4294967295,worker,<br>
[rabbit_mirror_queue_slave]}],<br> {dict,11,16,16,8,80,48,<br> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br> {{[],[],<br>
[[<0.265.0>,<br> {amqqueue,<br> {resource,<<"/">>,queue,<<"Advertiser-ONYX">>},<br> false,false,none,<br>
[{<<"x-ha-policy">>,longstr,<<"all">>},<br> {<<"x-message-ttl">>,signedint,1200000}],<br> <3067.289.0>,<br>
[<3065.334.0>],<br> all}]],<br> [],<br> [[undefined,<br> {amqqueue,<br>
{resource,<<"/">>,queue,<br> <<"Advertiser-heartbeat-DHOHMEYERGX280">>},<br> false,true,<3065.24765.0>,<br>
[{<<"x-ha-policy">>,longstr,<<"all">>}],<br> <3065.24784.0>,<br> [<3067.12370.2>],<br>
all}],<br> [<0.6730.0>,<br> {amqqueue,<br> {resource,<<"/">>,queue,<br> <<"Advertiser-heartbeat-DHOHMEYERGX280">>},<br>
false,true,<3065.14349.1>,<br> [{<<"x-ha-policy">>,longstr,<<"all">>}],<br> <3065.14370.1>,[],all}]],<br>
[[<0.268.0>,<br> {amqqueue,<br> {resource,<<"/">>,queue,<br> <<"Advertiser-DHOHMEYERGX280">>},<br>
false,false,none,<br> [{<<"x-ha-policy">>,longstr,<<"all">>},<br> {<<"x-message-ttl">>,signedint,1200000}],<br>
<3067.287.0>,<br> [<3065.332.0>],<br> all}]],<br> [],<br> [[<0.270.0>,<br>
{amqqueue,<br> {resource,<<"/">>,queue,<br> <<"CustomerAdmin-DHOHMEYERGX280">>},<br> false,false,none,<br>
[{<<"x-ha-policy">>,longstr,<<"all">>},<br> {<<"x-message-ttl">>,signedint,1200000}],<br> <3067.295.0>,<br>
[<3065.338.0>],<br> all}]],<br> [],<br> [[<0.272.0>,<br> {amqqueue,<br>
{resource,<<"/">>,queue,<br> <<"Sales-DHOHMEYERGX280">>},<br> false,false,none,<br> [{<<"x-ha-policy">>,longstr,<<"all">>},<br>
{<<"x-message-ttl">>,signedint,1200000}],<br> <3067.317.0>,<br> [<3065.356.0>],<br> all}]],<br>
[[<0.250.0>,<br> {amqqueue,<br> {resource,<<"/">>,queue,<br> <<"Tracking-JROSE6420">>},<br>
false,false,none,<br> [{<<"x-ha-policy">>,longstr,<<"all">>},<br> {<<"x-message-ttl">>,signedint,1200000}],<br>
<3067.247.0>,<br> [<3065.249.0>],<br> all}]],<br> [[<0.274.0>,<br> {amqqueue,<br>
{resource,<<"/">>,queue,<br> <<"RelatedLinks-ROGERMOORE">>},<br> false,false,none,<br>
[{<<"x-ha-policy">>,longstr,<<"all">>},<br> {<<"x-message-ttl">>,signedint,1200000}],<br> <3067.331.0>,<br>
[<3065.370.0>],<br> all}]],<br> [[<0.252.0>,<br> {amqqueue,<br> {resource,<<"ActivityLogging">>,queue,<br>
<<"vsw.activitylogging.conversion.pendingconversion">>},<br> true,false,none,<br> [{<<"x-ha-policy">>,longstr,<<"all">>}],<br>
<3067.253.0>,<br> [<3065.257.0>],<br> all}]],<br> [[<0.276.0>,<br> {amqqueue,<br>
{resource,<<"/">>,queue,<<"Search-JROSE6420">>},<br> false,false,none,<br> [{<<"x-ha-policy">>,longstr,<<"all">>},<br>
{<<"x-message-ttl">>,signedint,1200000}],<br> <3067.26259.1>,<br> [<3065.4161.0>],<br> all}]],<br>
[[<0.261.0>,<br> {amqqueue,<br> {resource,<<"/">>,queue,<br> <<"RelatedLinks-ONYX">>},<br>
false,false,none,<br> [{<<"x-ha-policy">>,longstr,<<"all">>},<br> {<<"x-message-ttl">>,signedint,1200000}],<br>
<3067.277.0>,<br> [<3065.322.0>],<br> all}]],<br> []}}},<br> 10,10,[],rabbit_mirror_queue_slave_sup,[]}<br>
** Reason for termination == <br>** {badarg,[{erlang,unlink,[undefined]},<br> {supervisor2,'-terminate_simple_children/3-fun-0-',4},<br> {dict,fold_bucket,3},<br> {dict,fold_seg,4},<br>
{dict,fold_segs,4},<br> {supervisor2,terminate_simple_children,3},<br> {supervisor2,terminate,2},<br> {gen_server,terminate,6}]}<br><br>=INFO REPORT==== 26-Apr-2012::16:51:08 ===<br>
Mirrored-queue (queue 'vsw.activitylogging.conversion.pendingconversion' in vhost 'ActivityLogging'): Slave <'rabbit@rabbitmq-dev-2'.1.252.0> saw deaths of mirrors <'rabbit@rabbitmq-dev-3'.2.253.0> <br>
<br>=ERROR REPORT==== 26-Apr-2012::16:51:08 ===<br>** Generic server <0.265.0> terminating<br>** Last message in was {'EXIT',<0.249.0>,<br> {badarg,<br> [{erlang,unlink,[undefined]},<br>
{supervisor2,<br> '-terminate_simple_children/3-fun-0-',4},<br> {dict,fold_bucket,3},<br> {dict,fold_seg,4},<br>
{dict,fold_segs,4},<br> {supervisor2,terminate_simple_children,3},<br> {supervisor2,terminate,2},<br> {gen_server,terminate,6}]}}<br>
** When Server state == {state,<br> {amqqueue,<br> {resource,<<"/">>,queue,<<"Advertiser-ONYX">>},<br> false,false,none,<br>
[{<<"x-ha-policy">>,longstr,<<"all">>},<br> {<<"x-message-ttl">>,signedint,1200000}],<br> <3067.289.0>,<br>
[<3065.334.0>],<br> all},<br> <0.267.0>,<3065.334.0>,rabbit_variable_queue,<br> {vqstate,<br> {0,{[],[]}},<br>
{0,{[],[]}},<br> {delta,undefined,0,undefined},<br> {0,{[],[]}},<br> {0,{[],[]}},<br> 0,<br> {0,nil},<br>
undefined,<br> {0,nil},<br> {qistate,<br> "/var/lib/rabbitmq/mnesia/rabbit@rabbitmq-dev-2/queues/1ON03RTBZAE64B67H41L4ICDE",<br>
{{dict,0,16,16,8,80,48,<br> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br> []}}},<br>
[]},<br> undefined,0,262144,<br> #Fun<rabbit_variable_queue.2.91259159>,<br> {0,nil}},<br> {undefined,<br>
{client_msstate,msg_store_transient,<br> <<228,99,153,84,177,114,138,215,76,30,0,48,128,77,<br> 68,245>>,<br> {dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br> []}}},<br> {state,368709,<br>
"/var/lib/rabbitmq/mnesia/rabbit@rabbitmq-dev-2/msg_store_transient"},<br> rabbit_msg_store_ets_index,<br> "/var/lib/rabbitmq/mnesia/rabbit@rabbitmq-dev-2/msg_store_transient",<br>
<0.239.0>,372806,364585,376903,381000}},<br> false,0,#Fun<rabbit_mirror_queue_slave.3.29941254>,<br> 0,0,infinity,0,0,0,0,0,<br> {rates,<br>
{{1335,483942,985021},0},<br> {{1335,483942,985021},0},<br> 0.0,0.0,<br> {1335,484240,226498}},<br> {0,nil},<br>
{0,nil},<br> {0,nil},<br> {0,nil},<br> 0,0,<br> {rates,<br> {{1335,483942,985021},0},<br>
{{1335,483942,985021},0},<br> 0.0,0.0,<br> {1335,484240,226498}}},<br> undefined,#Ref<0.0.9.158766>,<br> {dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},<br> {dict,0,16,16,8,80,48,<br> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},<br> 0,<br> {dict,0,16,16,8,80,48,<br> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},<br> {dict,0,16,16,8,80,48,<br> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},<br>
true}<br>** Reason for termination == <br>** {badarg,[{erlang,unlink,[undefined]},<br> {supervisor2,'-terminate_simple_children/3-fun-0-',4},<br> {dict,fold_bucket,3},<br>
{dict,fold_seg,4},<br> {dict,fold_segs,4},<br> {supervisor2,terminate_simple_children,3},<br> {supervisor2,terminate,2},<br> {gen_server,terminate,6}]}<br><br>=ERROR REPORT==== 26-Apr-2012::16:51:08 ===<br>
** Generic server <0.6730.0> terminating<br>** Last message in was {'EXIT',<0.249.0>,<br> {badarg,<br> [{erlang,unlink,[undefined]},<br> {supervisor2,<br>
'-terminate_simple_children/3-fun-0-',4},<br> {dict,fold_bucket,3},<br> {dict,fold_seg,4},<br> {dict,fold_segs,4},<br>
{supervisor2,terminate_simple_children,3},<br> {supervisor2,terminate,2},<br> {gen_server,terminate,6}]}}<br>** When Server state == {state,<br>
{amqqueue,<br> {resource,<<"/">>,queue,<br> <<"Advertiser-heartbeat-DHOHMEYERGX280">>},<br> false,true,<3065.14349.1>,<br>
[{<<"x-ha-policy">>,longstr,<<"all">>}],<br> <3065.14370.1>,[],all},<br> <0.6731.0>,<3065.14370.1>,rabbit_variable_queue,<br>
{vqstate,<br> {0,{[],[]}},<br> {0,{[],[]}},<br> {delta,undefined,0,undefined},<br> {0,{[],[]}},<br>
{0,{[],[]}},<br> 0,<br> {0,nil},<br> undefined,<br> {0,nil},<br> {qistate,<br>
"/var/lib/rabbitmq/mnesia/rabbit@rabbitmq-dev-2/queues/5R0GE9UEB8S0EMSJORBUKHBTE",<br> {{dict,0,16,16,8,80,48,<br> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br> []}}},<br> []},<br> undefined,0,262144,<br> #Fun<rabbit_variable_queue.2.91259159>,<br>
{0,nil}},<br> {undefined,<br> {client_msstate,msg_store_transient,<br> <<182,86,2,135,69,87,14,108,163,188,15,159,127,119,<br>
55,241>>,<br> {dict,0,16,16,8,80,48,<br> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]}}},<br> {state,368709,<br> "/var/lib/rabbitmq/mnesia/rabbit@rabbitmq-dev-2/msg_store_transient"},<br> rabbit_msg_store_ets_index,<br>
"/var/lib/rabbitmq/mnesia/rabbit@rabbitmq-dev-2/msg_store_transient",<br> <0.239.0>,372806,364585,376903,381000}},<br> false,0,#Fun<rabbit_mirror_queue_slave.3.29941254>,<br>
0,0,infinity,0,0,0,0,0,<br> {rates,<br> {{1335,484239,805553},0},<br> {{1335,484239,805553},0},<br> 0.0,0.0,<br>
{1335,484255,991021}},<br> {0,nil},<br> {0,nil},<br> {0,nil},<br> {0,nil},<br> 0,0,<br>
{rates,<br> {{1335,484239,805553},0},<br> {{1335,484239,805553},0},<br> 0.0,0.0,<br> {1335,484255,991021}}},<br>
undefined,#Ref<0.0.9.158602>,<br> {dict,0,16,16,8,80,48,<br> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},<br>
{dict,0,16,16,8,80,48,<br> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},<br> 0,<br>
{dict,0,16,16,8,80,48,<br> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},<br> {dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},<br> true}<br>** Reason for termination == <br>
** {badarg,[{erlang,unlink,[undefined]},<br> {supervisor2,'-terminate_simple_children/3-fun-0-',4},<br> {dict,fold_bucket,3},<br> {dict,fold_seg,4},<br> {dict,fold_segs,4},<br>
{supervisor2,terminate_simple_children,3},<br> {supervisor2,terminate,2},<br> {gen_server,terminate,6}]}<br><br>=ERROR REPORT==== 26-Apr-2012::16:51:08 ===<br>** Generic server <0.252.0> terminating<br>
** Last message in was {'EXIT',<0.249.0>,shutdown}<br>** When Server state == {state,<br> {amqqueue,<br> {resource,<<"ActivityLogging">>,queue,<br>
<<"vsw.activitylogging.conversion.pendingconversion">>},<br> true,false,none,<br> [{<<"x-ha-policy">>,longstr,<<"all">>}],<br>
<3067.253.0>,<br> [<3065.257.0>],<br> all},<br> <0.253.0>,<3065.257.0>,rabbit_variable_queue,<br>
{vqstate,<br> {0,{[],[]}},<br> {0,{[],[]}},<br> {delta,undefined,0,undefined},<br> {0,{[],[]}},<br>
{0,{[],[]}},<br> 0,<br> {0,nil},<br> undefined,<br> {0,nil},<br> {qistate,<br>
"/var/lib/rabbitmq/mnesia/rabbit@rabbitmq-dev-2/queues/9KDJJKR11HTT40SVOPWG0AWDM",<br> {{dict,0,16,16,8,80,48,<br> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br>
{{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br> []}}},<br> []},<br> undefined,0,262144,<br> #Fun<rabbit_variable_queue.2.91259159>,<br>
{0,nil}},<br> {{client_msstate,msg_store_persistent,<br> <<58,135,182,192,228,241,29,217,255,38,4,53,94,213,<br> 95,102>>,<br>
{dict,0,16,16,8,80,48,<br> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br> []}}},<br>
{state,389194,<br> "/var/lib/rabbitmq/mnesia/rabbit@rabbitmq-dev-2/msg_store_persistent"},<br> rabbit_msg_store_ets_index,<br> "/var/lib/rabbitmq/mnesia/rabbit@rabbitmq-dev-2/msg_store_persistent",<br>
<0.244.0>,393291,385097,397388,401485},<br> {client_msstate,msg_store_transient,<br> <<63,170,201,248,36,125,208,20,133,142,139,71,219,<br>
91,212,33>>,<br> {dict,0,16,16,8,80,48,<br> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],<br>
[]}}},<br> {state,368709,<br> "/var/lib/rabbitmq/mnesia/rabbit@rabbitmq-dev-2/msg_store_transient"},<br> rabbit_msg_store_ets_index,<br>
"/var/lib/rabbitmq/mnesia/rabbit@rabbitmq-dev-2/msg_store_transient",<br> <0.239.0>,372806,364585,376903,381000}},<br> true,0,#Fun<rabbit_mirror_queue_slave.3.29941254>,0,<br>
0,infinity,0,0,0,0,0,<br> {rates,<br> {{1335,483943,551021},0},<br> {{1335,483943,551021},0},<br> 0.0,0.0,<br>
{1335,484240,51118}},<br> {0,nil},<br> {0,nil},<br> {0,nil},<br> {0,nil},<br> 0,0,<br>
{rates,<br> {{1335,483943,551021},0},<br> {{1335,483943,551021},0},<br> 0.0,0.0,<br> {1335,484240,51118}}},<br>
undefined,#Ref<0.0.9.158618>,<br> {dict,0,16,16,8,80,48,<br> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},<br>
{dict,0,16,16,8,80,48,<br> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},<br> 0,<br>
{dict,0,16,16,8,80,48,<br> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},<br> {dict,0,16,16,8,80,48,<br>
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},<br> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},<br> true}<br>** Reason for termination == <br>
** {shutdown,<br> {gen_server2,call,<br> [msg_store_persistent,<br> {client_terminate,<br> <<58,135,182,192,228,241,29,217,255,38,4,53,94,213,95,102>>},<br> infinity]}}<br>
** In 'terminate' callback with reason ==<br>** shutdown<br><br>=INFO REPORT==== 26-Apr-2012::16:51:08 ===<br> application: rabbit<br> exited: stopped<br> type: permanent<br><br>=INFO REPORT==== 26-Apr-2012::16:51:08 ===<br>
application: os_mon<br> exited: stopped<br> type: permanent<br><br>=INFO REPORT==== 26-Apr-2012::16:51:08 ===<br> application: mnesia<br> exited: stopped<br> type: permanent<br><br>=INFO REPORT==== 26-Apr-2012::16:51:08 ===<br>
Halting Erlang VM<br><br>===============================================<br><br>dev-2-sasl.log<br>===============================================<br>=CRASH REPORT==== 26-Apr-2012::16:51:08 ===<br> crasher:<br> initial call: supervisor2:init/1<br>
pid: <0.249.0><br> registered_name: rabbit_mirror_queue_slave_sup<br> exception exit: {badarg,<br> [{erlang,unlink,[undefined]},<br> {supervisor2,'-terminate_simple_children/3-fun-0-',4},<br>
{dict,fold_bucket,3},<br> {dict,fold_seg,4},<br> {dict,fold_segs,4},<br> {supervisor2,terminate_simple_children,3},<br> {supervisor2,terminate,2},<br>
{gen_server,terminate,6}]}<br> in function gen_server:terminate/6<br> ancestors: [rabbit_sup,<0.126.0>]<br> messages: [{'DOWN',#Ref<0.0.9.159703>,process,<br> {undefined,'rabbit@rabbitmq-dev-2'},<br>
noproc}]<br> links: [<0.265.0>,<0.6730.0>]<br> dictionary: []<br> trap_exit: true<br> status: running<br> heap_size: 987<br> stack_size: 24<br> reductions: 5064<br>
neighbours:<br><br>=SUPERVISOR REPORT==== 26-Apr-2012::16:51:08 ===<br> Supervisor: {local,rabbit_sup}<br> Context: shutdown_error<br> Reason: {badarg,<br> [{erlang,unlink,[undefined]},<br>
{supervisor2,'-terminate_simple_children/3-fun-0-',4},<br> {dict,fold_bucket,3},<br> {dict,fold_seg,4},<br> {dict,fold_segs,4},<br>
{supervisor2,terminate_simple_children,3},<br> {supervisor2,terminate,2},<br> {gen_server,terminate,6}]}<br> Offender: [{pid,<0.249.0>},<br> {name,rabbit_mirror_queue_slave_sup},<br>
{mfargs,{rabbit_mirror_queue_slave_sup,start_link,[]}},<br> {restart_type,transient},<br> {shutdown,infinity},<br> {child_type,supervisor}]<br><br><br>=CRASH REPORT==== 26-Apr-2012::16:51:08 ===<br>
crasher:<br> initial call: gen:init_it/6<br> pid: <0.265.0><br> registered_name: []<br> exception exit: {badarg,<br> [{erlang,unlink,[undefined]},<br> {supervisor2,'-terminate_simple_children/3-fun-0-',4},<br>
{dict,fold_bucket,3},<br> {dict,fold_seg,4},<br> {dict,fold_segs,4},<br> {supervisor2,terminate_simple_children,3},<br> {supervisor2,terminate,2},<br>
{gen_server,terminate,6}]}<br> in function gen_server2:terminate/3<br> ancestors: [rabbit_mirror_queue_slave_sup,rabbit_sup,<0.126.0>]<br> messages: [{'EXIT',<0.267.0>,normal},<br>
{'DOWN',#Ref<0.0.3.44831>,process,<3065.334.0>,normal}]<br> links: []<br> dictionary: [{{xtype_to_module,direct},rabbit_exchange_type_direct},<br> {{xtype_to_module,topic},rabbit_exchange_type_topic},<br>
{guid,{{3831732564,2977073879,1277034544,2152547573},0}}]<br> trap_exit: true<br> status: running<br> heap_size: 2584<br> stack_size: 24<br> reductions: 4916<br> neighbours:<br><br>=CRASH REPORT==== 26-Apr-2012::16:51:08 ===<br>
crasher:<br> initial call: gen:init_it/6<br> pid: <0.6730.0><br> registered_name: []<br> exception exit: {badarg,<br> [{erlang,unlink,[undefined]},<br> {supervisor2,'-terminate_simple_children/3-fun-0-',4},<br>
{dict,fold_bucket,3},<br> {dict,fold_seg,4},<br> {dict,fold_segs,4},<br> {supervisor2,terminate_simple_children,3},<br> {supervisor2,terminate,2},<br>
{gen_server,terminate,6}]}<br> in function gen_server2:terminate/3<br> ancestors: [rabbit_mirror_queue_slave_sup,rabbit_sup,<0.126.0>]<br> messages: [{'EXIT',<0.6731.0>,normal},<br>
{'DOWN',#Ref<0.0.4.191216>,process,<3065.14370.1>,normal}]<br> links: []<br> dictionary: [{credit_blocked,[]},<br> {{xtype_to_module,direct},rabbit_exchange_type_direct},<br>
{{xtype_to_module,fanout},rabbit_exchange_type_fanout},<br> {guid,{{3059090055,1163333228,2747010975,2138519537},0}}]<br> trap_exit: true<br> status: running<br> heap_size: 1597<br>
stack_size: 24<br> reductions: 8290<br> neighbours:<br>===============================================<br><br><br><br>
</div>