[rabbitmq-discuss] Active-active crash report

[Vadim Chekan]

Hi all,

I'm testing my active-active setup (2.8.1, linux 64) and I am randomly
running into some crashes when I'm stopping a node. I can stop one node
abut another one fails along with it. Below is a crash log.

Is "rabbitmq_management" the UI plugin? Could it bring down the whone node,
because I keep UI open in browser while doing my testing.

Vadim.

=INFO REPORT==== 26-Apr-2012::12:15:58 ===
Adding mirror of queue 'test_33' in vhost '/' on node 'rabbit at rabbitmq-dev-2':
<3066.4773.0>

=INFO REPORT==== 26-Apr-2012::12:15:58 ===
Adding mirror of queue 'test_34' in vhost '/' on node 'rabbit at rabbitmq-dev-2':
<3066.4777.0>

=INFO REPORT==== 26-Apr-2012::12:15:59 ===
Adding mirror of queue 'test_35' in vhost '/' on node 'rabbit at rabbitmq-dev-2':
<3066.4782.0>

=INFO REPORT==== 26-Apr-2012::12:15:59 ===
Adding mirror of queue 'test_36' in vhost '/' on node 'rabbit at rabbitmq-dev-2':
<3066.4786.0>

=INFO REPORT==== 26-Apr-2012::12:15:59 ===
Adding mirror of queue 'test_37' in vhost '/' on node 'rabbit at rabbitmq-dev-2':
<3066.4790.0>

=INFO REPORT==== 26-Apr-2012::12:15:59 ===
Adding mirror of queue 'test_38' in vhost '/' on node 'rabbit at rabbitmq-dev-2':
<3066.4795.0>

=INFO REPORT==== 26-Apr-2012::12:15:59 ===
Adding mirror of queue 'test_39' in vhost '/' on node 'rabbit at rabbitmq-dev-2':
<3066.4799.0>

=ERROR REPORT==== 26-Apr-2012::12:15:59 ===
Discarding message
{'$gen_call',{<0.1955.0>,#Ref<0.0.0.5513>},{add_on_right,{9,<0.1955.0>}}}
from <0.1955.0> to <0.26823.834>
 in an old incarnation (2) of this node (3)


=INFO REPORT==== 26-Apr-2012::12:15:59 ===
Adding mirror of queue 'test_40' in vhost '/' on node 'rabbit at rabbitmq-dev-2':
<3066.4803.0>

=INFO REPORT==== 26-Apr-2012::12:15:59 ===
Adding mirror of queue 'test_41' in vhost '/' on node 'rabbit at rabbitmq-dev-2':
<3066.4807.0>

=ERROR REPORT==== 26-Apr-2012::12:15:59 ===
Discarding message
{'$gen_call',{<0.1983.0>,#Ref<0.0.0.5648>},{add_on_right,{9,<0.1983.0>}}}
from <0.1983.0> to <0.26850.834>
 in an old incarnation (2) of this node (3)


=INFO REPORT==== 26-Apr-2012::12:15:59 ===
Adding mirror of queue 'test_42' in vhost '/' on node 'rabbit at rabbitmq-dev-2':
<3066.4811.0>

=INFO REPORT==== 26-Apr-2012::12:15:59 ===
Adding mirror of queue 'test_43' in vhost '/' on node 'rabbit at rabbitmq-dev-2':
<3066.4816.0>

=INFO REPORT==== 26-Apr-2012::12:16:39 ===
rabbit on node 'rabbit at rabbitmq-dev-3' up

=WARNING REPORT==== 26-Apr-2012::12:18:15 ===
closing AMQP connection <0.1433.0> (172.16.51.100:16360 -> 10.7.1.21:5672):
connection_closed_abruptly

=INFO REPORT==== 26-Apr-2012::12:23:57 ===
    application: rabbitmq_management
    exited: shutdown
    type: permanent

=INFO REPORT==== 26-Apr-2012::12:23:57 ===
stopped TCP Listener on 0.0.0.0:5672

=ERROR REPORT==== 26-Apr-2012::12:23:57 ===
** Generic server <0.1800.0> terminating
** Last message in was {'$gen_cast',{gm_deaths,[<0.4684.0>]}}
** When Server state == {state,
                            {amqqueue,
                                {resource,<<"/">>,queue,<<"test_29">>},
                                true,false,<0.1433.0>,
                                [{<<"x-ha-policy">>,longstr,<<"all">>},
                                 {<<"x-message-ttl">>,signedint,600000}],
                                <0.1799.0>,[],all},
                            <0.1801.0>,
                            {dict,0,16,16,8,80,48,

{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                 []},
                                {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                  [],[]}}},
                            #Fun<rabbit_mirror_queue_master.1.2951048>,
                            #Fun<rabbit_mirror_queue_master.2.72654940>}
** Reason for termination ==
** {{case_clause,{ok,<3066.9234.0>,[<0.4683.0>]}},
    [{rabbit_mirror_queue_coordinator,handle_cast,2},
     {gen_server2,handle_msg,2},
     {proc_lib,wake_up,3}]}

=ERROR REPORT==== 26-Apr-2012::12:23:57 ===
** Generic server <0.1714.0> terminating
** Last message in was {'$gen_cast',{gm_deaths,[<0.4653.0>]}}
** When Server state == {state,
                            {amqqueue,
                                {resource,<<"/">>,queue,<<"test_23">>},
                                true,false,<0.1433.0>,
                                [{<<"x-ha-policy">>,longstr,<<"all">>},
                                 {<<"x-message-ttl">>,signedint,600000}],
                                <0.1712.0>,[],all},
                            <0.1715.0>,
                            {dict,0,16,16,8,80,48,

{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                 []},
                                {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                  [],[]}}},
                            #Fun<rabbit_mirror_queue_master.1.2951048>,
                            #Fun<rabbit_mirror_queue_master.2.72654940>}
** Reason for termination ==
** {{case_clause,{ok,<3066.9155.0>,[<0.4652.0>]}},
    [{rabbit_mirror_queue_coordinator,handle_cast,2},
     {gen_server2,handle_msg,2},
     {proc_lib,wake_up,3}]}

=ERROR REPORT==== 26-Apr-2012::12:23:57 ===
** Generic server <0.1531.0> terminating
** Last message in was {'$gen_cast',{gm_deaths,[<0.4587.0>]}}
** When Server state == {state,
                            {amqqueue,
                                {resource,<<"/">>,queue,<<"test_10">>},
                                true,false,<0.1433.0>,
                                [{<<"x-ha-policy">>,longstr,<<"all">>},
                                 {<<"x-message-ttl">>,signedint,600000}],
                                <0.1530.0>,[],all},
                            <0.1532.0>,
                            {dict,0,16,16,8,80,48,

{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                 []},
                                {{[],[],[],[],[],[],[],[],[],[],[],[],[],[],
                                  [],[]}}},
                            #Fun<rabbit_mirror_queue_master.1.2951048>,
                            #Fun<rabbit_mirror_queue_master.2.72654940>}
** Reason for termination ==
** {noproc,{gen_server2,call,[worker_pool,next_free,infinity]}}
(END)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120426/e224031c/attachment.htm>