[rabbitmq-discuss] Strange start error issue

Fri Sep 21 03:46:58 BST 2012

Hi Simon,

liubida is my colleague. So we are facing the same problem. We know these
each other's mail at the next morning :)

Thank you very much for you kindly answer.

Samuel

On Fri, Sep 21, 2012 at 1:05 AM, Simon MacMullen <simon at rabbitmq.com> wrote:

> Hi.
>
> I think you must be from the same organisation as "liubida", the log file
> is identical, so see my answer there:
>
> http://lists.rabbitmq.com/**pipermail/rabbitmq-discuss/**
> 2012-September/022560.html<http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2012-September/022560.html>
>
> Cheers, Simon
>
>
> On 19/09/12 17:10, Samuel Chen wrote:
>
>> Hi all
>>
>> I'm new to RabbitMQ. We are facing a strange issue against
>> rabbit_mgmt_db. I could not find any similar issues by searching on
>> google or stackoverflow. So I wonder if anyone could help to diagnose
>> this problem. Thanks in advance.
>>
>> We have a 2-node clustered RabbitMQ integrated with Celery. It worked
>> well for several months.
>> The issue occurred the first time on Jul 4th. After restarted, it worked
>> for about 2 months. Yesterday the issue occurred twice (one is after
>> restarted).
>> The stat was that a child (rabbit_mgmt_db??) was killed automatically.
>>   By some failures of restarting automatically, it reached the max
>> restart intensity. Eventually it was shutdown.
>> (Anther situation is that we deployed to 2-node cluster from 1 node
>> server at the end of June. Note sure if it caused this issue.)
>>
>> The hosts are virtual servers with 8/12G ram and 30G disk.
>> One node is disc node and the other is ram.
>> The load balance was very low (around 100M ram, few tasks) . Disk has
>> 2.5G free space.
>> Log as below.
>>
>> Thanks for any help.
>>
>>         SUPERVISOR REPORT==== 18-Sep-2012::14:53:13 ===
>>
>>               Supervisor: {<0.28802.1777>,mirrored_**supervisor}
>>
>>               Context:    child_terminated
>>
>>               Reason:     killed
>>
>>               Offender:   [{pid,<0.28804.1777>},
>>
>>                            {name,rabbit_mgmt_db},
>>
>>                            {mfa,{rabbit_mgmt_db,start_**link,[]}},
>>
>>                            {restart_type,permanent},
>>
>>                            {shutdown,4294967295},
>>
>>                            {child_type,worker}]
>>
>>
>>
>>         =SUPERVISOR REPORT==== 18-Sep-2012::14:53:14 ===
>>
>>               Supervisor: {<0.28802.1777>,mirrored_**supervisor}
>>
>>               Context:    start_error
>>
>>               Reason:     {already_started,<19704.13906.**2335>}
>>
>>               Offender:   [{pid,<0.28804.1777>},
>>
>>                            {name,rabbit_mgmt_db},
>>
>>                            {mfa,{rabbit_mgmt_db,start_**link,[]}},
>>
>>                            {restart_type,permanent},
>>
>>                            {shutdown,4294967295},
>>
>>                            {child_type,worker}]
>>
>>
>>
>>         =SUPERVISOR REPORT==== 18-Sep-2012::14:53:14 ===
>>
>>               Supervisor: {<0.28802.1777>,mirrored_**supervisor}
>>
>>               Context:    start_error
>>
>>               Reason:     {already_started,<19704.13906.**2335>}
>>
>>               Offender:   [{pid,<0.28804.1777>},
>>
>>                            {name,rabbit_mgmt_db},
>>
>>                            {mfa,{rabbit_mgmt_db,start_**link,[]}},
>>
>>                            {restart_type,permanent},
>>
>>                            {shutdown,4294967295},
>>
>>                            {child_type,worker}]
>>
>>
>>
>>         =SUPERVISOR REPORT==== 18-Sep-2012::14:53:14 ===
>>
>>               Supervisor: {<0.28802.1777>,mirrored_**supervisor}
>>
>>               Context:    start_error
>>
>>               Reason:     {already_started,<19704.13906.**2335>}
>>
>>               Offender:   [{pid,<0.28804.1777>},
>>
>>                            {name,rabbit_mgmt_db},
>>
>>                            {mfa,{rabbit_mgmt_db,start_**link,[]}},
>>
>>                            {restart_type,permanent},
>>
>>                            {shutdown,4294967295},
>>
>>                            {child_type,worker}]
>>
>>
>>
>>         =SUPERVISOR REPORT==== 18-Sep-2012::14:53:14 ===
>>
>>               Supervisor: {<0.28802.1777>,mirrored_**supervisor}
>>
>>               Context:    start_error
>>
>>               Reason:     {already_started,<19704.13906.**2335>}
>>
>>               Offender:   [{pid,<0.28804.1777>},
>>
>>                            {name,rabbit_mgmt_db},
>>
>>                            {mfa,{rabbit_mgmt_db,start_**link,[]}},
>>
>>                            {restart_type,permanent},
>>
>>                            {shutdown,4294967295},
>>
>>                            {child_type,worker}]
>>
>>
>>
>>         =SUPERVISOR REPORT==== 18-Sep-2012::14:53:14 ===
>>
>>               Supervisor: {<0.28802.1777>,mirrored_**supervisor}
>>
>>               Context:    start_error
>>
>>               Reason:     {already_started,<19704.13906.**2335>}
>>
>>               Offender:   [{pid,<0.28804.1777>},
>>
>>                            {name,rabbit_mgmt_db},
>>
>>                            {mfa,{rabbit_mgmt_db,start_**link,[]}},
>>
>>                            {restart_type,permanent},
>>
>>                            {shutdown,4294967295},
>>
>>                            {child_type,worker}]
>>
>>
>>
>>         =SUPERVISOR REPORT==== 18-Sep-2012::14:53:14 ===
>>
>>               Supervisor: {local,rabbit_mgmt_sup}
>>
>>               Context:    shutdown
>>
>>               Reason:     reached_max_restart_intensity
>>
>>               Offender:   [{pid,<0.28803.1777>},
>>
>>                            {name,mirroring},
>>
>>                            {mfa,
>>
>>                                {mirrored_supervisor,start_**internal,
>>
>>                                    [rabbit_mgmt_sup,
>>
>>                                     [{rabbit_mgmt_db,
>>
>>                                          {rabbit_mgmt_db,start_link,[]}**
>> ,
>>
>>                                          permanent,4294967295,worker,
>>
>>                                          [rabbit_mgmt_db]}]]}},
>>
>>                            {restart_type,permanent},
>>
>>                            {shutdown,4294967295},
>>
>>                            {child_type,worker}]
>>
>>
>>
>> - Sam
>>
>>
>>
>> ______________________________**_________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.**rabbitmq.com<rabbitmq-discuss at lists.rabbitmq.com>
>> https://lists.rabbitmq.com/**cgi-bin/mailman/listinfo/**rabbitmq-discuss<https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss>
>>
>
>
> --
> Simon MacMullen
> RabbitMQ, VMware
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120921/42d90a50/attachment.htm>