[rabbitmq-discuss] rabbit_mgmt_db crashed
Chen, Bin
Bin.Chen at kla-tencor.com
Wed Oct 31 18:48:43 GMT 2012
Hi Simon,
I found the similar crash reports on another RMQ server.
It had happened a few times.
However, the memory utilization didn't went up greatly as another
server.
Two of the attached zip files are the eval output and report output.
The other one is the sasl log file containing the crash reports.
Thanks,
--Bin Chen
-----Original Message-----
From: Simon MacMullen [mailto:simon at rabbitmq.com]
Sent: Wednesday, October 31, 2012 4:07 AM
To: Discussions about RabbitMQ
Cc: Chen, Bin
Subject: Re: [rabbitmq-discuss] rabbit_mgmt_db crashed
Hi. I remember you posted a similar stack trace six months ago:
http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2012-May/019990.htm
l
I stared at the code involved for some time but was completely unable to
see how that could occur. I don't suppose you have any way to replicate
this?
Apart from that, could you post the output of:
rabbitmqctl.bat eval
"lists:sublist(lists:reverse(lists:sort([{process_info(Pid, memory),
Pid, process_info(Pid)} || Pid <- processes()])), 30)."
(all on one line)
and
rabbitmqctl.bat report
somewhere please?
Cheers, Simon
On 31/10/12 00:09, Chen, Bin wrote:
> Hello,
>
> I just noticed a crash report in sasl log of one of our RMQ server.
>
> It seems the management plug-in crashed. The management web page
> cannot be opened any more.
>
> I had this problem before with an old version. Now I am using 2.8.6
> 64-bit version on Windows 2008 Server.
>
> The RMQ server is still running fine, but the memory utilization is
> much higher than other similar servers without such crash.
>
> This "bad" server uses about 8 GB memory now, while other normal
> servers use only about 500 MB memory.
>
> It may eventually hit the high memory water mark soon at 9.6 GB.
>
> Here is the complete crash report.
>
> =CRASH REPORT==== 29-Oct-2012::23:01:08 ===
>
> crasher:
>
> initial call: rabbit_mgmt_db:init/1
>
> pid: <0.284.0>
>
> registered_name: []
>
> exception exit: {badarith,
>
> [{rabbit_mgmt_db,rate,5,[]},
>
> {rabbit_mgmt_db,'-rates/5-lc$^0/1-0-',5,[]},
>
> {rabbit_mgmt_db,'-rates/5-lc$^1/1-1-',6,[]},
>
> {rabbit_mgmt_db,rates,5,[]},
>
> {rabbit_mgmt_db,handle_fine_stat,7,[]},
>
>
> {rabbit_mgmt_db,'-handle_fine_stats/4-lc$^1/1-1-',4,
>
> []},
>
>
> {rabbit_mgmt_db,'-handle_event/2-lc$^1/1-0-',4,[]},
>
> {rabbit_mgmt_db,handle_event,2,[]}]}
>
> in function gen_server:terminate/6 (gen_server.erl, line 747)
>
> ancestors: [<0.281.0>,rabbit_mgmt_sup,<0.262.0>]
>
> messages: [{'$gen_cast',
>
> {event,
>
> {event,channel_stats,
>
> [{pid,<0.12756.396>},
>
> {transactional,false},
>
> {confirm,false},
>
> {consumer_count,0},
>
> {messages_unacknowledged,0},
>
> {messages_unconfirmed,0},
>
> {messages_uncommitted,0},
>
> {acks_uncommitted,0},
>
> {prefetch_count,1},
>
> {client_flow_blocked,false},
>
> {channel_queue_stats,[]},
>
> {channel_exchange_stats,[]},
>
> {channel_queue_exchange_stats,[]}],
>
> {1351,576868,812003}}}}]
>
> links: [<0.281.0>]
>
> dictionary: []
>
> trap_exit: false
>
> status: running
>
> heap_size: 6765
>
> stack_size: 24
>
> reductions: 3730467164
>
> neighbours:
>
> Here is the output of "rabbitmqctl.bat status"
>
> Status of node rabbit at RACK4UNIT001 ...
>
> [{pid,4708},
>
> {running_applications,
>
> [{rabbitmq_management,"RabbitMQ Management Console","2.8.6"},
>
> {rabbitmq_management_agent,"RabbitMQ Management
> Agent","2.8.6"},
>
> {amqp_client,"RabbitMQ AMQP Client","2.8.6"},
>
> {rabbit,"RabbitMQ","2.8.6"},
>
> {os_mon,"CPO CXC 138 46","2.2.10"},
>
> {sasl,"SASL CXC 138 11","2.2.1"},
>
> {rabbitmq_mochiweb,"RabbitMQ Mochiweb Embedding","2.8.6"},
>
> {webmachine,"webmachine","1.9.1-rmq2.8.6-git52e62bc"},
>
> {mochiweb,"MochiMedia Web Server","2.3.1-rmq2.8.6-gitd541e9a"},
>
> {xmerl,"XML parser","1.3.2"},
>
> {inets,"INETS CXC 138 49","5.9.1"},
>
> {mnesia,"MNESIA CXC 138 12","4.7.1"},
>
> {stdlib,"ERTS CXC 138 10","1.18.2"},
>
> {kernel,"ERTS CXC 138 10","2.15.2"}]},
>
> {os,{win32,nt}},
>
> {erlang_version,
>
> "Erlang R15B02 (erts-5.9.2) [64-bit] [smp:12:12]
> [async-threads:30]\n"},
>
> {memory,
>
> [{total,8512001360},
>
> {processes,7401925315},
>
> {processes_used,7401754972},
>
> {system,1110076045},
>
> {atom,662409},
>
> {atom_used,652416},
>
> {binary,857726336},
>
> {code,17969722},
>
> {ets,4550280}]},
>
> {vm_memory_high_watermark,0.4},
>
> {vm_memory_limit,10304190873},
>
> {disk_free_limit,1000000000},
>
> {disk_free,39365828608},
>
> {file_descriptors,
>
> [{total_limit,16284},
>
> {total_used,142},
>
> {sockets_limit,14653},
>
> {sockets_used,113}]},
>
> {processes,[{limit,1048576},{used,5986}]},
>
> {run_queue,0},
>
> {uptime,104350}]
>
> ...done.
>
> Best regards,
>
> --Bin Chen
>
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
--
Simon MacMullen
RabbitMQ, VMware
-------------- next part --------------
A non-text attachment was scrubbed...
Name: report-output.zip
Type: application/x-zip-compressed
Size: 51941 bytes
Desc: report-output.zip
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20121031/1876792b/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: eval-output.zip
Type: application/x-zip-compressed
Size: 492573 bytes
Desc: eval-output.zip
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20121031/1876792b/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rabbit at RACK3UNIT001-sasl.log.2012.10.31-10.00.00.zip
Type: application/x-zip-compressed
Size: 2130 bytes
Desc: rabbit at RACK3UNIT001-sasl.log.2012.10.31-10.00.00.zip
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20121031/1876792b/attachment-0002.bin>
More information about the rabbitmq-discuss
mailing list