[rabbitmq-discuss] Crash on badmatch

[Chris Larsen]

Hi folks, we had a problem today where the MQ server kept crashing and we
think it was due to some client pushing in a bad message as we were able to
keep the server up only after removing access to some users. We're still
trying to track down the exact client that was causing the issues. However
the cause of the crash was listed in the logs as bellow. I couldn't find any
references to "badmatch" errors on the list before so I was wondering if
anyone has seen it?

 

We're running Rabbitmq 2.8.5 on erlang 1:15.b.1-1~ubuntu~lucid. Thanks!

 

=ERROR REPORT==== 20-Sep-2012::12:59:37 ===

** Generic server <0.229.0> terminating

** Last message in was {'$gen_cast',{combine,1001,999}}

** When Server state == {state,

                            {dict,0,16,16,8,80,48,

 
{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],

                                 []},

                                {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],

                                  [],[]}}},

                            [#Fun<rabbit_msg_store.15.106110853>],

                            {gc_state,

 
"/mnt/drbd0/rabbit at localhost/msg_store_persistent",

                                rabbit_msg_store_ets_index,

                                {state,458823,

 
"/mnt/drbd0/rabbit at localhost/msg_store_persistent"},

                                454726,462920,<0.226.0>}}

** Reason for termination == 

**
{{badmatch,{[{msg_location,<<119,180,79,244,242,22,59,63,219,227,203,29,171,

                                138,191,149>>,

                              1,999,2301,298},

 
{msg_location,<<15,166,249,210,74,141,92,125,164,106,201,179,

                                247,29,128,81>>,

                              1,999,4643,297},

 
{msg_location,<<61,32,10,20,15,47,161,44,66,91,98,119,129,10,

                                230,135>>,

                              1,999,7239,297},

 
{msg_location,<<242,222,190,244,108,107,42,177,98,113,130,255,

                                211,78,44,242>>,

                              1,999,7791,297},

 
{msg_location,<<107,45,44,203,166,43,87,116,132,109,19,63,113,

                                184,191,143>>,

                              1,999,9877,297},

..... { THOUSANDS OF LINES } .....

 
{msg_location,<<167,233,20,236,134,250,109,211,72,107,129,39,

                                57,87,109,67>>,

                              1,999,1580282,254}],

               378139}},

    [{rabbit_msg_store,combine_files,3,[]},

     {rabbit_msg_store_gc,attempt_action,3,[]},

     {rabbit_msg_store_gc,handle_cast,2,[]},

     {gen_server2,handle_msg,2,[]},

     {proc_lib,wake_up,3,[{file,"proc_lib.erl"},{line,237}]}]}

 

=ERROR REPORT==== 20-Sep-2012::13:31:15 ===

Error description:

   {shutdown,

       {gen_server,call,

           [{rabbit_amqqueue_sup,rabbit at localhost},

            {start_child,

                [{amqqueue,

                     {resource,<<"/exchange/queu">>,queue,

                         <<"/queue/hostname:1983">>},

                     true,false,none,[],<0.2434.0>,[],undefined}]},

            infinity]}}

 

Log files (may contain more information):

   /var/log/rabbitmq/rabbit at localhost.log

   /var/log/rabbitmq/rabbit at localhost-sasl.log

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120920/6cd0d752/attachment.htm>