<div dir="ltr">Hi all,<div><br></div><div>I am currently running a 2 node cluster with Rabbit 3.1.5. We have policies setup to ensure that certain queues are mirrored on all the nodes in a cluster. </div><div><br></div><div>
Operations was configuring a new cluster and put the hipe_compile flag to true (normally set to false). When they did this, queue creation was failing for queues that needed to be mirrored. The error is below</div><div><br>
</div><div><div><font face="courier new, monospace">=INFO REPORT==== 10-Jun-2014::19:19:52 ===</font></div><div><font face="courier new, monospace">Adding mirror of queue 'title-server-test' in vhost '/test' on node 'rabbit@udbamq-cmc-02': <14717.5271.0></font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">=ERROR REPORT==== 10-Jun-2014::19:19:53 ===</font></div><div><font face="courier new, monospace">** Generic server <0.5257.0> terminating</font></div>
<div><font face="courier new, monospace">** Last message in was pre_hibernate</font></div><div><font face="courier new, monospace">** When Server state == {q,</font></div><div><font face="courier new, monospace"> {amqqueue,</font></div>
<div><font face="courier new, monospace"> {resource,<<"/test">>,queue,</font></div><div><font face="courier new, monospace"> <<"title-server-test">>},</font></div>
<div><font face="courier new, monospace"> true,false,none,[],<0.5257.0>,[],[],</font></div><div><font face="courier new, monospace"> [{vhost,<<"/test">>},</font></div>
<div><font face="courier new, monospace"> {name,<<"unu-ha-all">>},</font></div><div><font face="courier new, monospace"> {pattern,</font></div><div>
<font face="courier new, monospace"> <<"menu-item-server.*|title-server.*|unu-server.*">>},</font></div><div><font face="courier new, monospace"> {definition,[{<<"ha-mode">>,<<"all">>}]},</font></div>
<div><font face="courier new, monospace"> {priority,0}],</font></div><div><font face="courier new, monospace"> []},</font></div><div><font face="courier new, monospace"> none,false,rabbit_mirror_queue_master,</font></div>
<div><font face="courier new, monospace"> {state,</font></div><div><font face="courier new, monospace"> {resource,<<"/test">>,queue,</font></div><div>
<font face="courier new, monospace"> <<"title-server-test">>},</font></div><div><font face="courier new, monospace"> <0.5259.0>,<0.5258.0>,rabbit_variable_queue,</font></div>
<div><font face="courier new, monospace"> {vqstate,</font></div><div><font face="courier new, monospace"> {0,{[],[]}},</font></div><div><font face="courier new, monospace"> {0,{[],[]}},</font></div>
<div><font face="courier new, monospace"> {delta,undefined,0,undefined},</font></div><div><font face="courier new, monospace"> {0,{[],[]}},</font></div><div><font face="courier new, monospace"> {0,{[],[]}},</font></div>
<div><font face="courier new, monospace"> 0,</font></div><div><font face="courier new, monospace"> {0,nil},</font></div><div><font face="courier new, monospace"> {0,nil},</font></div>
<div><font face="courier new, monospace"> {qistate,</font></div><div><font face="courier new, monospace"> "/var/lib/rabbitmq/mnesia/rabbit@udbamq-cmc-01/queues/C3QO8FHBOV9KWRA6JK601BOPG",</font></div>
<div><font face="courier new, monospace"> {{dict,0,16,16,8,80,48,</font></div><div><font face="courier new, monospace"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="courier new, monospace"> []},</font></div><div><font face="courier new, monospace"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div><div>
<font face="courier new, monospace"> []}}},</font></div><div><font face="courier new, monospace"> []},</font></div><div><font face="courier new, monospace"> undefined,0,65536,</font></div>
<div><font face="courier new, monospace"> #Fun<rabbit_variable_queue.2.81334491>,</font></div><div><font face="courier new, monospace"> {0,nil}},</font></div><div>
<font face="courier new, monospace"> {{client_msstate,msg_store_persistent,</font></div><div><font face="courier new, monospace"> <<155,86,72,191,145,120,119,132,195,47,38,124,147,</font></div>
<div><font face="courier new, monospace"> 109,237,88>>,</font></div><div><font face="courier new, monospace"> {dict,0,16,16,8,80,48,</font></div><div><font face="courier new, monospace"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="courier new, monospace"> []},</font></div><div><font face="courier new, monospace"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div><div>
<font face="courier new, monospace"> []}}},</font></div><div><font face="courier new, monospace"> {state,35409996,</font></div><div><font face="courier new, monospace"> "/var/lib/rabbitmq/mnesia/rabbit@udbamq-cmc-01/msg_store_persistent"},</font></div>
<div><font face="courier new, monospace"> rabbit_msg_store_ets_index,</font></div><div><font face="courier new, monospace"> "/var/lib/rabbitmq/mnesia/rabbit@udbamq-cmc-01/msg_store_persistent",</font></div>
<div><font face="courier new, monospace"> <0.4760.0>,35414093,35405899,35418190,35422287},</font></div><div><font face="courier new, monospace"> {client_msstate,msg_store_transient,</font></div>
<div><font face="courier new, monospace"> <<167,204,117,67,245,155,36,187,165,217,239,159,</font></div><div><font face="courier new, monospace"> 54,180,2,199>>,</font></div>
<div><font face="courier new, monospace"> {dict,0,16,16,8,80,48,</font></div><div><font face="courier new, monospace"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div>
<div><font face="courier new, monospace"> []},</font></div><div><font face="courier new, monospace"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div><div>
<font face="courier new, monospace"> []}}},</font></div><div><font face="courier new, monospace"> {state,35389511,</font></div><div><font face="courier new, monospace"> "/var/lib/rabbitmq/mnesia/rabbit@udbamq-cmc-01/msg_store_transient"},</font></div>
<div><font face="courier new, monospace"> rabbit_msg_store_ets_index,</font></div><div><font face="courier new, monospace"> "/var/lib/rabbitmq/mnesia/rabbit@udbamq-cmc-01/msg_store_transient",</font></div>
<div><font face="courier new, monospace"> <0.4753.0>,35393608,35385408,35397705,35401802}},</font></div><div><font face="courier new, monospace"> true,0,0,0,infinity,0,0,0,0,0,</font></div>
<div><font face="courier new, monospace"> {rates,</font></div><div><font face="courier new, monospace"> {{1402,427992,812678},0},</font></div><div><font face="courier new, monospace"> {{1402,427992,812678},0},</font></div>
<div><font face="courier new, monospace"> 0.0,0.0,</font></div><div><font face="courier new, monospace"> {1402,427992,812678}},</font></div><div><font face="courier new, monospace"> {0,nil},</font></div>
<div><font face="courier new, monospace"> {0,nil},</font></div><div><font face="courier new, monospace"> {0,nil},</font></div><div><font face="courier new, monospace"> {0,nil},</font></div>
<div><font face="courier new, monospace"> 0,0,</font></div><div><font face="courier new, monospace"> {rates,</font></div><div><font face="courier new, monospace"> {{1402,427992,812678},0},</font></div>
<div><font face="courier new, monospace"> {{1402,427992,812678},0},</font></div><div><font face="courier new, monospace"> 0.0,0.0,</font></div><div><font face="courier new, monospace"> {1402,427992,812678}}},</font></div>
</div><div><div><font face="courier new, monospace"> {dict,0,16,16,8,80,48,</font></div><div><font face="courier new, monospace"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},</font></div>
<div><font face="courier new, monospace"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div><div><font face="courier new, monospace"> []}}},</font></div><div>
<font face="courier new, monospace"> [],</font></div><div><font face="courier new, monospace"> {set,0,16,16,8,80,48,</font></div><div><font face="courier new, monospace"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},</font></div>
<div><font face="courier new, monospace"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],</font></div><div><font face="courier new, monospace"> []}}}},</font></div><div>
<font face="courier new, monospace"> {[],[]},</font></div><div><font face="courier new, monospace"> undefined,undefined,#Ref<0.0.0.146892>,undefined,</font></div><div>
<font face="courier new, monospace"> {state,fine,5000,#Ref<0.0.0.146893>},</font></div><div><font face="courier new, monospace"> {0,nil},</font></div><div><font face="courier new, monospace"> undefined,undefined,undefined,</font></div>
<div><font face="courier new, monospace"> {dict,0,16,16,8,80,48,</font></div><div><font face="courier new, monospace"> {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},</font></div>
<div><font face="courier new, monospace"> {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]}}},</font></div><div><font face="courier new, monospace"> undefined,undefined,undefined,running}</font></div>
<div><font face="courier new, monospace">** Reason for termination ==</font></div><div><font face="courier new, monospace">** {badarith,[{rabbit_variable_queue,ram_duration,1},</font></div><div><font face="courier new, monospace"> {rabbit_mnesia,node_info,0},</font></div>
<div><font face="courier new, monospace"> {rabbit_amqqueue_process,handle_pre_hibernate,1},</font></div><div><font face="courier new, monospace"> {gen_server2,pre_hibernate,1},</font></div><div><font face="courier new, monospace"> {rabbit_mnesia,node_info,0}]}</font></div>
</div><div><br></div><div>From the sasl log</div><div><font face="courier new, monospace"><br></font></div><div><div><font face="courier new, monospace">=CRASH REPORT==== 10-Jun-2014::19:19:53 ===</font></div><div><font face="courier new, monospace"> crasher:</font></div>
<div><font face="courier new, monospace"> initial call: gen:init_it/6</font></div><div><font face="courier new, monospace"> pid: <0.5257.0></font></div><div><font face="courier new, monospace"> registered_name: []</font></div>
<div><font face="courier new, monospace"> exception exit: {badarith,</font></div><div><font face="courier new, monospace"> [{rabbit_variable_queue,ram_duration,1},</font></div><div><font face="courier new, monospace"> {rabbit_mnesia,node_info,0},</font></div>
<div><font face="courier new, monospace"> {rabbit_amqqueue_process,handle_pre_hibernate,1},</font></div><div><font face="courier new, monospace"> {gen_server2,pre_hibernate,1},</font></div>
<div><font face="courier new, monospace"> {rabbit_mnesia,node_info,0}]}</font></div><div><font face="courier new, monospace"> in function gen_server2:terminate/3</font></div><div><font face="courier new, monospace"> in call from rabbit_mnesia:node_info/0</font></div>
<div><font face="courier new, monospace"> ancestors: [rabbit_amqqueue_sup,rabbit_sup,<0.4622.0>]</font></div><div><font face="courier new, monospace"> messages: [{'EXIT',<0.5258.0>,{shutdown,ring_shutdown}}]</font></div>
<div><font face="courier new, monospace"> links: [<0.4761.0>]</font></div><div><font face="courier new, monospace"> dictionary: [{{xtype_to_module,direct},rabbit_exchange_type_direct},</font></div><div><font face="courier new, monospace"> {guid,{{2815194435,4120585403,2782523295,917766855},1}}]</font></div>
<div><font face="courier new, monospace"> trap_exit: true</font></div><div><font face="courier new, monospace"> status: running</font></div><div><font face="courier new, monospace"> heap_size: 2584</font></div><div>
<font face="courier new, monospace"> stack_size: 24</font></div><div><font face="courier new, monospace"> reductions: 2122</font></div><div><font face="courier new, monospace"> neighbours:</font></div><div><font face="courier new, monospace"><br>
</font></div><div><font face="courier new, monospace">=SUPERVISOR REPORT==== 10-Jun-2014::19:19:53 ===</font></div><div><font face="courier new, monospace"> Supervisor: {local,rabbit_amqqueue_sup}</font></div><div><font face="courier new, monospace"> Context: child_terminated</font></div>
<div><font face="courier new, monospace"> Reason: {badarith,[{rabbit_variable_queue,ram_duration,1},</font></div><div><font face="courier new, monospace"> {rabbit_mnesia,node_info,0},</font></div>
<div><font face="courier new, monospace"> {rabbit_amqqueue_process,handle_pre_hibernate,1},</font></div><div><font face="courier new, monospace"> {gen_server2,pre_hibernate,1},</font></div>
<div><font face="courier new, monospace"> {rabbit_mnesia,node_info,0}]}</font></div><div><font face="courier new, monospace"> Offender: [{pid,<0.5257.0>},</font></div><div><font face="courier new, monospace"> {name,rabbit_amqqueue},</font></div>
<div><font face="courier new, monospace"> {mfa,</font></div><div><font face="courier new, monospace"> {rabbit_amqqueue_process,start_link,</font></div><div><font face="courier new, monospace"> [{amqqueue,</font></div>
<div><font face="courier new, monospace"> {resource,<<"/unu-broker">>,queue,</font></div><div><font face="courier new, monospace"> <<"title-server-date">>},</font></div>
<div><font face="courier new, monospace"> true,false,none,[],none,[],[],</font></div><div><font face="courier new, monospace"> [{vhost,<<"/unu-broker">>},</font></div>
<div><font face="courier new, monospace"> {name,<<"unu-ha-all">>},</font></div><div><font face="courier new, monospace"> {pattern,</font></div>
<div><font face="courier new, monospace"> <<"menu-item-server.*|title-server.*|unu-server.*">>},</font></div><div><font face="courier new, monospace"> {definition,[{<<"ha-mode">>,<<"all">>}]},</font></div>
<div><font face="courier new, monospace"> {priority,0}],</font></div><div><font face="courier new, monospace"> []}]}},</font></div><div><font face="courier new, monospace"> {restart_type,temporary},</font></div>
<div><font face="courier new, monospace"> {shutdown,4294967295},</font></div><div><font face="courier new, monospace"> {child_type,worker}]</font></div><div><font face="courier new, monospace"><br>
</font></div><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">=CRASH REPORT==== 10-Jun-2014::19:21:33 ===</font></div><div><font face="courier new, monospace"> crasher:</font></div>
<div><font face="courier new, monospace"> initial call: mochiweb_acceptor:init/3</font></div><div><font face="courier new, monospace"> pid: <0.4789.0></font></div><div><font face="courier new, monospace"> registered_name: []</font></div>
<div><font face="courier new, monospace"> exception exit: {error,accept_failed}</font></div><div><font face="courier new, monospace"> in function mochiweb_acceptor:init/3</font></div><div><font face="courier new, monospace"> ancestors: [rabbit_web_dispatch_sup_55672,rabbit_web_dispatch_sup,</font></div>
<div><font face="courier new, monospace"> <0.4613.0>]</font></div><div><font face="courier new, monospace"> messages: []</font></div><div><font face="courier new, monospace"> links: []</font></div>
<div><font face="courier new, monospace"> dictionary: []</font></div><div><font face="courier new, monospace"> trap_exit: false</font></div><div><font face="courier new, monospace"> status: running</font></div><div>
<font face="courier new, monospace"> heap_size: 377</font></div><div><font face="courier new, monospace"> stack_size: 24</font></div><div><font face="courier new, monospace"> reductions: 2378</font></div><div><font face="courier new, monospace"> neighbours:</font></div>
<div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace">=CRASH REPORT==== 10-Jun-2014::19:21:33 ===</font></div><div><font face="courier new, monospace"> crasher:</font></div><div>
<font face="courier new, monospace"> initial call: mochiweb_acceptor:init/3</font></div><div><font face="courier new, monospace"> pid: <0.4787.0></font></div><div><font face="courier new, monospace"> registered_name: []</font></div>
<div><font face="courier new, monospace"> exception exit: {error,accept_failed}</font></div><div><font face="courier new, monospace"> in function mochiweb_acceptor:init/3</font></div><div><font face="courier new, monospace"> ancestors: [rabbit_web_dispatch_sup_55672,rabbit_web_dispatch_sup,</font></div>
<div><font face="courier new, monospace"> <0.4613.0>]</font></div><div><font face="courier new, monospace"> messages: []</font></div><div><font face="courier new, monospace"> links: []</font></div>
<div><font face="courier new, monospace"> dictionary: []</font></div><div><font face="courier new, monospace"> trap_exit: false</font></div><div><font face="courier new, monospace"> status: running</font></div><div>
<font face="courier new, monospace"> heap_size: 377</font></div><div><font face="courier new, monospace"> stack_size: 24</font></div><div><font face="courier new, monospace"> reductions: 2383</font></div><div><font face="courier new, monospace"> neighbours:</font></div>
</div><div><br></div><div>We can run with hipe_compile=false, but wanted to see if anyone else has seen this.</div><div><br></div><div>thanks</div><div><br></div><div>dave</div><div><br></div><div><br></div></div>