[rabbitmq-discuss] clearing a queue error

Fri Sep 27 17:01:33 BST 2013

Hi, I've got a durable, high availability queue across a cluster of 2 servers that was working fine, but I'm now getting an error related to it when rabbitmq starts:

=ERROR REPORT==== 27-Sep-2013::09:02:35 ===
** Generic server <0.249.0> terminating
** Last message in was {init,<0.126.0>}
** When Server state == {q,{amqqueue,
                               {resource,<<"xsede_public">>,queue,
                                   <<"carnac.sysinfo">>},
                               true,false,none,[],<0.249.0>,[],[],
                               [{vhost,<<"xsede_public">>},
                                {name,<<"ha-carnac">>},
                                {pattern,<<"^carnac\\.">>},
                                {definition,[{<<"ha-mode">>,<<"all">>}]},
                                {priority,0}],
                               []},
                           none,false,undefined,undefined,
                           {[],[]},
                           undefined,undefined,undefined,undefined,
                           {state,fine,5000,undefined},
                           {0,nil},
                           undefined,undefined,undefined,
                           {dict,0,16,16,8,80,48,
                               {[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                []},
                               {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                 []}}},
                           undefined,undefined,undefined,running}
** Reason for termination == 
** {function_clause,[{rabbit_variable_queue,d,[{delta,49152,15059,49152}],[]},
                     {rabbit_variable_queue,maybe_deltas_to_betas,1,[]},
                     {rabbit_variable_queue,init,6,[]},
                     {rabbit_mirror_queue_master,init,3,[]},
                     {rabbit_amqqueue_process,declare,3,[]},
                     {gen_server2,handle_msg,2,[]},
                     {proc_lib,init_p_do_apply,3,
                               [{file,"proc_lib.erl"},{line,239}]}]}

The error started after the rabbitmq servers were each up and down several times over the course of an hour or two.

The high availability policy is:

$ rabbitmqctl -p xsede_public list_policies
Listing policies ...
xsede_public     ha-carnac      ^carnac\\.      {"ha-mode":"all"}       0
...done.

I don't see the queue when I do a:

$ rabbitmqctl -p xsede_public list_queues

In an effort to clean out the problem, I deleted the high availability policy, but that didn't fix things - I got the same error as above, just without the policy information in it. I also tried to just delete the queue by name using amqp, but no luck. Rabbitmqctl sync_queue on that queue also just hung for me.

There may be a bug here for you to look in to, but do you have a suggestion for how I can clear this error?

I'm also getting a 500 error in the management web interface that I assume is caused by the above:

Internal Server Error

The server encountered an error while processing this request:
{error,
    {error,
        {badmatch,false},
        [{rabbit_mgmt_wm_overview,version,1,
             [{file,"rabbitmq-management/src/rabbit_mgmt_wm_overview.erl"},
              {line,73}]},
         {rabbit_mgmt_wm_overview,to_json,2,
             [{file,"rabbitmq-management/src/rabbit_mgmt_wm_overview.erl"},
              {line,39}]},
         {webmachine_resource,resource_call,3,[]},
         {webmachine_resource,do,3,[]},
         {webmachine_decision_core,resource_call,1,[]},
         {webmachine_decision_core,decision,1,[]},
         {webmachine_decision_core,handle_request,2,[]},
         {rabbit_webmachine,'-makeloop/1-fun-0-',2,
             [{file,"rabbitmq-web-dispatch/src/rabbit_webmachine.erl"},
              {line,71}]}]}}

Thanks,

Warren