[rabbitmq-discuss] Mgmt API stopped reporting a lot of stuff

Fri Apr 19 01:13:52 BST 2013

When I "kill -9" tiger02, tiger01 reported this in the sasl:
=CRASH REPORT==== 18-Apr-2013::23:51:35 ===
  crasher:
    initial call: gen:init_it/6
    pid: <0.29816.660>
    registered_name: []
    exception exit: {badarith,[{rabbit_mgmt_db,rate,5},
                               {rabbit_mgmt_db,'-rates/5-lc$^0/1-0-',5},
                               {rabbit_mgmt_db,rates,5},
                               {rabbit_mgmt_db,handle_stats,6},
                               {rabbit_mgmt_db,handle_cast,2},
                               {gen_server2,handle_msg,2},
                               {proc_lib,init_p_do_apply,3}]}
      in function  gen_server2:terminate/3
    ancestors: [<0.5880.0>,rabbit_mgmt_sup,<0.5843.0>]
    messages: []
    links: [<0.5880.0>]
    dictionary: []
    trap_exit: false
    status: running
    heap_size: 832040
    stack_size: 24
    reductions: 3582745480
  neighbours:

and stopped working. Any calls to /api/overview or "rabbitmqctl status" just
hangs, as does "rabbitmqctl stop", this time the only output in the log is:

=INFO REPORT==== 18-Apr-2013::23:55:51 ===
Stopping RabbitMQ

I killed tiger01 and started it again, the sasl log then contained a couple
of these:

=SUPERVISOR REPORT==== 18-Apr-2013::23:57:27 ===
     Supervisor: {local,rabbit_amqqueue_sup}
     Context:    child_terminated
     Reason:     {undef,
                     [{undefined,init,
                          [{amqqueue,
                               {resource,<<"vhost3">>,queue,
                                   <<"tmp_topic-0.4612013988662511">>},
                               true,true,<0.11266.6402>,[],<0.817.0>,[],[],
                               undefined,[]},
                           true,#Fun<rabbit_amqqueue_process.5.64830354>]},
                      {rabbit_amqqueue_process,handle_call,3},
                      {gen_server2,handle_msg,2},
                      {proc_lib,wake_up,3}]}
     Offender:   [{pid,<0.817.0>},
                  {name,rabbit_amqqueue},
                  {mfa,
                      {rabbit_amqqueue_process,start_link,
                          [{amqqueue,
                               {resource,<<"vhost3">>,queue,
                                   <<"tmp_topic-0.4612013988662511">>},

true,true,<0.11266.6402>,[],<0.11437.6402>,[],
                               [],undefined,[]}]}},
                  {restart_type,temporary},
                  {shutdown,4294967295},
                  {child_type,worker}]

And even though all queues are mirrored, tiger01 still produces a lot of
these, while tiger03 hasn't been started yet: 

=ERROR REPORT==== 19-Apr-2013::00:07:49 ===
connection <0.28262.92>, channel 1 - soft error:
{amqp_error,not_found,
            "home node 'rabbit at tiger03' of durable queue 'celery' in vhost
'vhost2' is down or inaccessible",
            'queue.declare'}

--
View this message in context: http://rabbitmq.1065348.n5.nabble.com/Mgmt-API-stopped-reporting-a-lot-of-stuff-tp26065p26094.html
Sent from the RabbitMQ mailing list archive at Nabble.com.