[rabbitmq-discuss] Crash Report

[Simon MacMullen]

Hmm. So the logs make it fairly clear that Mnesia shut down due to an 
error. Mnesia is the distributed database RabbitMQ uses to store 
metadata, so when it is down all sorts of things will fail. 
Unfortunately it's not very clear why it shut down. This is, needless to 
say, an unusual occurrence - I found one similar report from 2009 and 
that's it.

So I'm not sure what to recommend. It may be worth upgrading your Erlang 
installation (the debs from 
https://www.erlang-solutions.com/downloads/download-erlang-otp should 
help here). If this happens again then the output of

  rabbitmqctl eval 'ets:tab2list(mnesia_gvar).'

might prove useful.

Sorry I can't give you something more concrete.

Cheers, Simon

On 17/12/12 23:09, Matt Long wrote:
> Using RabbitMQ v2.8.5 on Ubuntu 11.10. Yes, I'm aware that version 3.0+
> is now available...I'm just a bit behind in staying up to date :-)
>
> RabbitMQ suddenly became unreachable by my clients. In particular, the
> amqp Python package raised ConnectionError: 541: (INTERNAL_ERROR, (0,
> 0), None). First thing I did was attempting to list the queues:
>
> $ sudo rabbitmqctl list_queues
> Listing queues ...
> Error: {aborted,{no_exists,[rabbit_queue,
>                              {amqqueue,{resource,<<"/">>,queue,'_'},
>                                        '_','_','_','_','_','_','_'}]}}
>
> This was fairly cryptic to me since I'm not too familiar with the
> innards of RabbitMQ. Next I checked the status:
>
> $ sudo service rabbitmq-server status
> Status of node 'rabbit at ip-10-100-105-118' ...
> [{pid,879},
>   {running_applications,[{rabbit,"RabbitMQ","2.8.5"},
>                          {os_mon,"CPO  CXC 138 46","2.2.5"},
>                          {sasl,"SASL  CXC 138 11","2.1.9.3"},
>                          {stdlib,"ERTS  CXC 138 10","1.17.3"},
>                          {kernel,"ERTS  CXC 138 10","2.14.3"}]},
>   {os,{unix,linux}},
>   {erlang_version,"Erlang R14B02 (erts-5.8.3) [source] [64-bit]
> [smp:8:8] [rq:8] [async-threads:30] [kernel-poll:true]\n"},
>   {memory,[{total,128970944},
>            {processes,49035088},
>            {processes_used,36245608},
>            {system,79935856},
>            {atom,1182377},
>            {atom_used,1157674},
>            {binary,17554600},
>            {code,11329658},
>            {ets,47162232}]},
>   {vm_memory_high_watermark,0.4},
>   {vm_memory_limit,2920456192},
>   {disk_free_limit,1000000000},
>   {disk_free,6324633600},
>   {file_descriptors,[{total_limit,924},
>                      {total_used,186},
>                      {sockets_limit,829},
>                      {sockets_used,128}]},
>   {processes,[{limit,1048576},{used,1235}]},
>   {run_queue,0},
>   {uptime,4829636}]
> ...done.
>
> The error logs contain quite a bit of information related to the crash.
> Here are links to excerpts from the main RabbitMQ log and the sasl log:
>
> rabbit at myhostname.log: http://pastebin.com/1CBjmbGb
> rabbit at myhostname-sasl.log: http://pastebin.com/8qZas7hA
>
> Stopping and then starting the rabbitmq-server daemon luckily fixed the
> problem and seemed to maintain at least most of Rabbit's state. Can
> someone help me interpret what happened and what I can do to prevent
> this in the future?
>
> Many thanks in advance!
>
> Cheers,
> Matt
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>


-- 
Simon MacMullen
RabbitMQ, VMware