<div dir="ltr"><div>Hi,</div><div><br></div><div>I'm running two RabbitMQ nodes in a cluster on Ubuntu 12.04. I have a virtual host there and a ha policy for a queue. Both nodes are disc nodes. During the last 24 hours RabbitMQ crashed twice. Producers are able to continue publishing messages but they don't end up routed or something. rabbitmqctl list_queues does not list the queue. In the HTTP management console the queue is listed but the page only shows:</div>
<div><span style="color:rgb(72,72,72);font-family:Verdana,sans-serif;font-size:12px"><br></span></div><div><span style="color:rgb(72,72,72);font-family:Verdana,sans-serif;font-size:12px">TypeError: Cannot read property 'ram_msg_count' of undefined</span><br>
</div><div><span style="color:rgb(72,72,72);font-family:Verdana,sans-serif;font-size:12px">TypeError: Cannot read property 'ram_msg_count' of undefined</span>�</div><div><br></div><div>There should not anything special going on on the servers. Also, they are not under heavy load or something.</div>
<div><br></div><div>After the last crash the log file on both nodes were empty. Unfortunately I didn't backup the logs after the first crashes. I was able to find the following information in xxx-sasl.log:</div><div><br>
</div><div>=CRASH REPORT==== 19-Nov-2013::04:17:22 ===</div><div>� crasher:</div><div>� � initial call: gen:init_it/6</div><div>� � pid: <0.259.0></div><div>� � registered_name: []</div><div>� � exception exit: {{badmatch,{error,not_found}},</div>
<div>� � � � � � � � � � �[{rabbit_mirror_queue_master,stop_all_slaves,2},</div><div>� � � � � � � � � � � {rabbit_mirror_queue_master,delete_and_terminate,2},</div><div>� � � � � � � � � � � {rabbit_amqqueue_process,'-terminate_delete/3-fun-1-',6},</div>
<div>� � � � � � � � � � � {rabbit_amqqueue_process,terminate_shutdown,2},</div><div>� � � � � � � � � � � {gen_server2,terminate,3},</div><div>� � � � � � � � � � � {proc_lib,wake_up,3}]}</div><div>� � � in function �gen_server2:terminate/3</div>
<div>� � ancestors: [rabbit_amqqueue_sup,rabbit_sup,<0.192.0>]</div><div>� � messages: []</div><div>� � links: [<0.257.0>,<0.452.0>,#Port<0.13309>]</div><div>� � dictionary: [{delegate,delegate_15},</div>
<div>� � � � � � � � � {{credit_to,<0.788.0>},20},</div><div>� � � � � � � � � {{#Ref<0.0.0.1271>,fhc_handle},</div><div>� � � � � � � � � �{handle,{file_descriptor,prim_file,{#Port<0.13309>,20}},</div><div>
� � � � � � � � � � � � � �774680,false,0,infinity,[],true,</div><div>� � � � � � � � � � � � � �"/var/lib/rabbitmq/mnesia/rabbit@mq1-api-test/queues/B1433I2VYA7YIS4WJY5I6NO4R/journal.jif",</div><div>� � � � � � � � � � � � � �[write,binary,raw,read],</div>
<div>� � � � � � � � � � � � � �[{write_buffer,infinity}],</div><div>� � � � � � � � � � � � � �true,true,</div><div>� � � � � � � � � � � � � �{1384,827437,174496}}},</div><div>� � � � � � � � � {{credit_to,<0.563.0>},1},</div>
<div>� � � � � � � � � {fhc_age_tree,{1,</div><div>� � � � � � � � � � � � � � � � �{{1384,827437,174496},</div><div>� � � � � � � � � � � � � � � � � #Ref<0.0.0.1271>,nil,nil}}},</div><div>� � � � � � � � � {{"/var/lib/rabbitmq/mnesia/rabbit@mq1-api-test/queues/B1433I2VYA7YIS4WJY5I6NO4R/journal.jif",</div>
<div>� � � � � � � � � � fhc_file},</div><div><div>� � � � � � � � � �{file,1,true}},</div><div>� � � � � � � � � {credit_blocked,[]},</div><div>� � � � � � � � � {{ch,<0.8244.0>},</div><div>� � � � � � � � � �{cr,<0.8244.0>,#Ref<0.0.10.78984>,</div>
<div>� � � � � � � � � � � �{[],[]},</div><div>� � � � � � � � � � � �1,</div><div>� � � � � � � � � � � �{queue,[],[],0},</div><div>� � � � � � � � � � � �{qstate,<0.8243.0>,active,{0,nil}},</div><div>� � � � � � � � � � � �25}},</div>
<div>� � � � � � � � � {{ch,<0.8274.0>},</div><div>� � � � � � � � � �{cr,<0.8274.0>,#Ref<0.0.10.91831>,</div><div>� � � � � � � � � � � �{[],[]},</div><div>� � � � � � � � � � � �1,</div><div>� � � � � � � � � � � �{queue,[],[],0},</div>
<div>� � � � � � � � � � � �{qstate,<0.8273.0>,active,{0,nil}},</div><div>� � � � � � � � � � � �49}},</div><div>� � � � � � � � � {{credit_to,<0.13668.0>},41},</div><div>� � � � � � � � � {guid,{{1330803274,3383937130,2004047782,2434038047},1}},</div>
<div>� � � � � � � � � {{credit_from,<0.251.0>},1826}]</div><div>� � trap_exit: true</div><div>� � status: running</div><div>� � heap_size: 10946</div><div>� � stack_size: 24</div><div>� � reductions: 1514324983</div>
<div>� neighbours:</div><div>� � neighbour: [{pid,<0.453.0>},</div><div>� � � � � � � � � {registered_name,[]},</div><div>� � � � � � � � � {initial_call,{gen,init_it,</div><div>� � � � � � � � � � � � � � � � � � �['Argument__1','Argument__2',</div>
<div>� � � � � � � � � � � � � � � � � � � 'Argument__3','Argument__4',</div><div>� � � � � � � � � � � � � � � � � � � 'Argument__5','Argument__6']}},</div><div>� � � � � � � � � {current_function,{gen_server2,process_next_msg,1}},</div>
<div>� � � � � � � � � {ancestors,[<0.452.0>,<0.259.0>,rabbit_amqqueue_sup,</div><div>� � � � � � � � � � � � � � � rabbit_sup,<0.192.0>]},</div><div>� � � � � � � � � {messages,[]},</div><div>� � � � � � � � � {links,[<0.452.0>]},</div>
</div><div><div>� � � � � � � � � {dictionary,[{random_seed,{24781,9822,29460}}]},</div><div>� � � � � � � � � {trap_exit,false},</div><div>� � � � � � � � � {status,waiting},</div><div>� � � � � � � � � {heap_size,610},</div>
<div>� � � � � � � � � {stack_size,7},</div><div>� � � � � � � � � {reductions,660407699}]</div><div>� � neighbour: [{pid,<0.452.0>},</div><div>� � � � � � � � � {registered_name,[]},</div><div>� � � � � � � � � {initial_call,{gen,init_it,</div>
<div>� � � � � � � � � � � � � � � � � � �['Argument__1','Argument__2',</div><div>� � � � � � � � � � � � � � � � � � � 'Argument__3','Argument__4',</div><div>� � � � � � � � � � � � � � � � � � � 'Argument__5','Argument__6']}},</div>
<div>� � � � � � � � � {current_function,{gen_server2,process_next_msg,1}},</div><div>� � � � � � � � � {ancestors,[<0.259.0>,rabbit_amqqueue_sup,rabbit_sup,</div><div>� � � � � � � � � � � � � � � <0.192.0>]},</div>
<div>� � � � � � � � � {messages,[]},</div><div>� � � � � � � � � {links,[<0.259.0>,<0.453.0>]},</div><div>� � � � � � � � � {dictionary,[]},</div><div>� � � � � � � � � {trap_exit,false},</div><div>� � � � � � � � � {status,waiting},</div>
<div>� � � � � � � � � {heap_size,2584},</div><div>� � � � � � � � � {stack_size,7},</div><div>� � � � � � � � � {reductions,59821}]</div></div><div><br></div><div>=SUPERVISOR REPORT==== 19-Nov-2013::04:17:22 ===</div><div>
� � �Supervisor: {local,rabbit_amqqueue_sup}</div><div>� � �Context: � �child_terminated</div><div>� � �Reason: � � {{badmatch,{error,not_found}},</div><div>� � � � � � � � � [{rabbit_mirror_queue_master,stop_all_slaves,2},</div>
<div>� � � � � � � � � �{rabbit_mirror_queue_master,delete_and_terminate,2},</div><div>� � � � � � � � � �{rabbit_amqqueue_process,'-terminate_delete/3-fun-1-',6},</div><div>� � � � � � � � � �{rabbit_amqqueue_process,terminate_shutdown,2},</div>
<div>� � � � � � � � � �{gen_server2,terminate,3},</div><div>� � � � � � � � � �{proc_lib,wake_up,3}]}</div><div>� � �Offender: � [{pid,<0.259.0>},</div><div>� � � � � � � � � {name,rabbit_amqqueue},</div><div>� � � � � � � � � {mfargs,{rabbit_amqqueue_process,start_link,undefined}},</div>
<div>� � � � � � � � � {restart_type,temporary},</div><div>� � � � � � � � � {shutdown,4294967295},</div><div>� � � � � � � � � {child_type,worker}]</div><div><br></div><div><br></div><div>=CRASH REPORT==== 19-Nov-2013::04:30:20 ===</div>
<div>� crasher:</div><div>� � initial call: gen:init_it/6</div><div>� � pid: <0.13842.0></div><div>� � registered_name: []</div><div>� � exception exit: {{badmatch,true},</div><div>� � � � � � � � � � �[{rabbit_queue_index,init,2},</div>
<div>� � � � � � � � � � � {rabbit_variable_queue,init,5},</div><div>� � � � � � � � � � � {rabbit_mirror_queue_master,init,3},</div><div>� � � � � � � � � � � {rabbit_amqqueue_process,declare,3},</div><div>� � � � � � � � � � � {gen_server2,handle_msg,2},</div>
<div>� � � � � � � � � � � {proc_lib,init_p_do_apply,3}]}</div><div>� � � in function �gen_server2:terminate/3</div><div>� � ancestors: [rabbit_amqqueue_sup,rabbit_sup,<0.192.0>]</div><div>� � messages: []</div><div>
� � links: [<0.257.0>]</div><div>� � dictionary: [{{xtype_to_module,direct},rabbit_exchange_type_direct}]</div><div>� � trap_exit: true</div><div>� � status: running</div><div>� � heap_size: 1597</div><div>� � stack_size: 24</div>
<div>� � reductions: 1224</div><div>� neighbours:</div><div><br></div><div>=SUPERVISOR REPORT==== 19-Nov-2013::04:30:20 ===</div><div>� � �Supervisor: {local,rabbit_amqqueue_sup}</div><div>� � �Context: � �child_terminated</div>
<div>� � �Reason: � � {{badmatch,true},</div><div>� � � � � � � � � [{rabbit_queue_index,init,2},</div><div>� � � � � � � � � �{rabbit_variable_queue,init,5},</div><div>� � � � � � � � � �{rabbit_mirror_queue_master,init,3},</div>
<div>� � � � � � � � � �{rabbit_amqqueue_process,declare,3},</div><div>� � � � � � � � � �{gen_server2,handle_msg,2},</div><div>� � � � � � � � � �{proc_lib,init_p_do_apply,3}]}</div><div>� � �Offender: � [{pid,<0.13842.0>},</div>
<div>� � � � � � � � � {name,rabbit_amqqueue},</div><div>� � � � � � � � � {mfargs,{rabbit_amqqueue_process,start_link,undefined}},</div><div>� � � � � � � � � {restart_type,temporary},</div><div>� � � � � � � � � {shutdown,4294967295},</div>
<div>� � � � � � � � � {child_type,worker}]</div><div><br></div><div><br></div><div><br></div><div>I have no clue what's going on here. Can anyone give me a hint?</div><div><br></div><div>Best Regards,</div><div>Lauri</div>
</div>