[rabbitmq-discuss] Memory watermark alert not resetting.

Thu Oct 4 16:38:46 BST 2012

Upgraded the other node (there are two) and it fails to start.
Here is the startup log:

+---+   +---+
|   |   |   |
|   |   |   |
|   |   |   |
|   +---+   +-------+
|                   |
| RabbitMQ  +---+   |
|           |   |   |
|   v2.8.7  +---+   |
|                   |
+-------------------+
AMQP 0-9-1 / 0-9 / 0-8
Copyright (C) 2007-2012 VMware, Inc.
Licensed under the MPL.  See http://www.rabbitmq.com/

node           : rabbit at mongo-qa2
app descriptor :
/usr/lib/rabbitmq/lib/rabbitmq_server-2.8.7/sbin/../ebin/rabbit.app
home dir       : /var/lib/rabbitmq
config file(s) : (none)
cookie hash    : pu0BlS9+G2N9yLfd51TkmA==
log            : /var/log/rabbitmq/rabbit at mongo-qa2.log
sasl log       : /var/log/rabbitmq/rabbit at mongo-qa2-sasl.log
database dir   : /var/lib/rabbitmq/mnesia/rabbit at mongo-qa2
erlang version : 5.8.4

-- rabbit boot start
starting file handle cache server
...done
starting worker pool
 ...done
starting database                                                     ...

BOOT FAILED
===========

Error description:
   {error,{failed_to_cluster_with,['rabbit at mongo-qa1'],
                                  "Mnesia could not connect to any disc
nodes."}}

Log files (may contain more information):
   /var/log/rabbitmq/rabbit at mongo-qa2.log
   /var/log/rabbitmq/rabbit at mongo-qa2-sasl.log

Stack trace:
   [{rabbit_mnesia,init_db,3},
    {rabbit_mnesia,init,0},
    {rabbit,'-run_boot_step/1-lc$^1/1-1-',1},
    {rabbit,run_boot_step,1},
    {rabbit,'-start/2-lc$^0/1-0-',1},
    {rabbit,start,2},
    {application_master,start_it_old,4}]

{"Kernel pid
terminated",application_controller,"{application_start_failure,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot}}}}}"}

Thanks
*--*Raviv
*
*

On Thu, Oct 4, 2012 at 5:34 PM, Simon MacMullen <simon at rabbitmq.com> wrote:

> Hard to believe that's to do with the server not starting - can you post a
> more complete log?
>
>
> On 04/10/12 16:32, Raviv Pavel wrote:
>
>> After upgrading the server won't start. from the logs:
>>
>> =ERROR REPORT==== 4-Oct-2012::17:11:59 ===
>> ** Generic server <0.341.0> terminating
>> ** Last message in was {'$gen_cast',
>>                             {method,
>>
>>   {'queue.declare',0,<<"es1">>,**false,true,false,
>>                                     false,false,
>>
>>   [{<<"x-ha-policy">>,longstr,<<**"all">>}]},
>>                                 none,noflow}}
>>
>>
>> Thanks
>> /--/Raviv
>> /
>>
>> /
>>
>>
>>
>> On Thu, Oct 4, 2012 at 5:05 PM, Raviv Pavel <raviv at gigya-inc.com
>> <mailto:raviv at gigya-inc.com>> wrote:
>>
>>     All queues are empty and "rabbitmqctl list_queues name memory" shows
>>     they use about 3K each.
>>     VM memory stats are much lower then the one shown in the overview.
>>     We're using 2.8.1 - I'll try upgrading to 2.8.7
>>
>>     Thanks
>>     /--/Raviv
>>     /
>>     /
>>
>>
>>
>>     On Thu, Oct 4, 2012 <tel:2012> at 4:57 PM, Matthias Radestock
>>
>>     <matthias at rabbitmq.com <mailto:matthias at rabbitmq.com>**> wrote:
>>
>>         On 04/10/12 15:53, Simon MacMullen wrote:
>>
>>             On 04/10/12 13:01, Raviv Pavel wrote:
>>
>>                 Based on the management UI, memory usage doesn't drop
>>                 and publishers are
>>                 still blocked.
>>
>>
>>             Hmm. In that case my first guess would be that the queue you
>>             deleted /
>>             purged wasn't the queue which was using all the memory.
>>             Check the memory
>>             use of other queues - this can be found on the queue details
>>             page, or
>>             with "rabbitmqctl list_queues name memory". Also, check the
>>             VM-wide
>>             memory statistics (on the node details page, under
>>             "advanced". This
>>             might give a clue as to where the memory is going.
>>
>>
>>         ...and please make sure you are running the latest version of
>>         RabbitMQ (2.8.7, atm), since we have fixed a few memory leaks in
>>         the in recent versions.
>>
>>         Matthias.
>>
>>
>>
>>
>
> --
> Simon MacMullen
> RabbitMQ, VMware
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20121004/6aa90c75/attachment.htm>