[rabbitmq-discuss] rabbit_mgmt_db crashed

[Simon MacMullen]

Hi. I remember you posted a similar stack trace six months ago:

http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2012-May/019990.html

I stared at the code involved for some time but was completely unable to 
see how that could occur. I don't suppose you have any way to replicate 
this?

Apart from that, could you post the output of:

rabbitmqctl.bat eval
"lists:sublist(lists:reverse(lists:sort([{process_info(Pid, memory),
Pid, process_info(Pid)} || Pid <- processes()])), 30)."

(all on one line)

and

rabbitmqctl.bat report

somewhere please?

Cheers, Simon

On 31/10/12 00:09, Chen, Bin wrote:
> Hello,
>
> I just noticed a crash report in sasl log of one of our RMQ server.
>
> It seems the management plug-in crashed. The management web page cannot
> be opened any more.
>
> I had this problem before with an old version. Now I am using 2.8.6
> 64-bit version on Windows 2008 Server.
>
> The RMQ server is still running fine, but the memory utilization is much
> higher than other similar servers without such crash.
>
> This “bad” server uses about 8 GB memory now, while other normal servers
> use only about 500 MB memory.
>
> It may eventually hit the high memory water mark soon at 9.6 GB.
>
> Here is the complete crash report.
>
> =CRASH REPORT==== 29-Oct-2012::23:01:08 ===
>
>    crasher:
>
>      initial call: rabbit_mgmt_db:init/1
>
>      pid: <0.284.0>
>
>      registered_name: []
>
>      exception exit: {badarith,
>
>                          [{rabbit_mgmt_db,rate,5,[]},
>
>                           {rabbit_mgmt_db,'-rates/5-lc$^0/1-0-',5,[]},
>
>                           {rabbit_mgmt_db,'-rates/5-lc$^1/1-1-',6,[]},
>
>                           {rabbit_mgmt_db,rates,5,[]},
>
>                           {rabbit_mgmt_db,handle_fine_stat,7,[]},
>
>
> {rabbit_mgmt_db,'-handle_fine_stats/4-lc$^1/1-1-',4,
>
>                               []},
>
>
>         {rabbit_mgmt_db,'-handle_event/2-lc$^1/1-0-',4,[]},
>
>                           {rabbit_mgmt_db,handle_event,2,[]}]}
>
>        in function  gen_server:terminate/6 (gen_server.erl, line 747)
>
>      ancestors: [<0.281.0>,rabbit_mgmt_sup,<0.262.0>]
>
>      messages: [{'$gen_cast',
>
>                        {event,
>
>                            {event,channel_stats,
>
>                                [{pid,<0.12756.396>},
>
>                                 {transactional,false},
>
>                                 {confirm,false},
>
>                                 {consumer_count,0},
>
>                                 {messages_unacknowledged,0},
>
>                                 {messages_unconfirmed,0},
>
>                                 {messages_uncommitted,0},
>
>                                 {acks_uncommitted,0},
>
>                                 {prefetch_count,1},
>
>                                 {client_flow_blocked,false},
>
>                                 {channel_queue_stats,[]},
>
>                                 {channel_exchange_stats,[]},
>
>                                 {channel_queue_exchange_stats,[]}],
>
>                                {1351,576868,812003}}}}]
>
>      links: [<0.281.0>]
>
>      dictionary: []
>
>      trap_exit: false
>
>      status: running
>
>      heap_size: 6765
>
>      stack_size: 24
>
>      reductions: 3730467164
>
>    neighbours:
>
> Here is the output of “rabbitmqctl.bat status”
>
> Status of node rabbit at RACK4UNIT001 ...
>
> [{pid,4708},
>
> {running_applications,
>
>       [{rabbitmq_management,"RabbitMQ Management Console","2.8.6"},
>
>        {rabbitmq_management_agent,"RabbitMQ Management Agent","2.8.6"},
>
>        {amqp_client,"RabbitMQ AMQP Client","2.8.6"},
>
>        {rabbit,"RabbitMQ","2.8.6"},
>
>        {os_mon,"CPO  CXC 138 46","2.2.10"},
>
>        {sasl,"SASL  CXC 138 11","2.2.1"},
>
>        {rabbitmq_mochiweb,"RabbitMQ Mochiweb Embedding","2.8.6"},
>
>        {webmachine,"webmachine","1.9.1-rmq2.8.6-git52e62bc"},
>
>        {mochiweb,"MochiMedia Web Server","2.3.1-rmq2.8.6-gitd541e9a"},
>
>        {xmerl,"XML parser","1.3.2"},
>
>        {inets,"INETS  CXC 138 49","5.9.1"},
>
>        {mnesia,"MNESIA  CXC 138 12","4.7.1"},
>
>        {stdlib,"ERTS  CXC 138 10","1.18.2"},
>
>        {kernel,"ERTS  CXC 138 10","2.15.2"}]},
>
> {os,{win32,nt}},
>
> {erlang_version,
>
>       "Erlang R15B02 (erts-5.9.2) [64-bit] [smp:12:12]
> [async-threads:30]\n"},
>
> {memory,
>
>       [{total,8512001360},
>
>        {processes,7401925315},
>
>        {processes_used,7401754972},
>
>        {system,1110076045},
>
>        {atom,662409},
>
>        {atom_used,652416},
>
>        {binary,857726336},
>
>        {code,17969722},
>
>        {ets,4550280}]},
>
> {vm_memory_high_watermark,0.4},
>
> {vm_memory_limit,10304190873},
>
> {disk_free_limit,1000000000},
>
> {disk_free,39365828608},
>
> {file_descriptors,
>
>       [{total_limit,16284},
>
>        {total_used,142},
>
>        {sockets_limit,14653},
>
>        {sockets_used,113}]},
>
> {processes,[{limit,1048576},{used,5986}]},
>
> {run_queue,0},
>
> {uptime,104350}]
>
> ...done.
>
> Best regards,
>
> --Bin Chen
>
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>


-- 
Simon MacMullen
RabbitMQ, VMware