[rabbitmq-discuss] If 2 nodes out a 3 node cluster, the third one becomes unresponsive until one of the nodes is brought back.

Yamil Einar Asusta Santos yamil.asusta at upr.edu
Tue Aug 6 16:47:31 BST 2013


I have been testing my cluster and I have come across an unexpected 
behavior. As explained in the subject, the cluster runs well on 3 nodes. If 
I bring 1 node down, it still runs smoothly. But if I bring a second node 
down, the third one becomes unresponsive.
This is what the third node provides after running "rabbitmqctl report":

Reporting server status on {{2013,8,6},{15,19,8}}
 ...
Error: {aborted,{no_exists,rabbit_vhost}}


Then if I bring 1 or 2 of my nodes back up, the third one becomes 
responsive and everything is back to normal. 

Here is the report after bringing the nodes up again:

Reporting server status on {{2013,8,6},{15,26,31}}
 ...
Status of node rabbit at qcluster1 ...
[{pid,901},
 {running_applications,
     [{rabbitmq_management,"RabbitMQ Management Console","3.1.3"},
      {rabbitmq_management_agent,"RabbitMQ Management Agent","3.1.3"},
      {rabbit,"RabbitMQ","3.1.3"},
      {os_mon,"CPO  CXC 138 46","2.2.7"},
      {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.1.3"},
      {webmachine,"webmachine","1.9.1-rmq3.1.3-git52e62bc"},
      {mochiweb,"MochiMedia Web Server","2.3.1-rmq3.1.3-gitd541e9a"},
      {xmerl,"XML parser","1.2.10"},
      {inets,"INETS  CXC 138 49","5.7.1"},
      {mnesia,"MNESIA  CXC 138 12","4.5"},
      {amqp_client,"RabbitMQ AMQP Client","3.1.3"},
      {sasl,"SASL  CXC 138 11","2.1.10"},
      {stdlib,"ERTS  CXC 138 10","1.17.5"},
      {kernel,"ERTS  CXC 138 10","2.14.5"}]},
 {os,{unix,linux}},
 {erlang_version,
     "Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2] 
[async-threads:30] [kernel-poll:true]\n"},
 {memory,
     [{total,31689208},
      {connection_procs,5408},
      {queue_procs,5408},
      {plugins,340592},
      {other_proc,9166952},
      {mnesia,61984},
      {mgmt_db,10248},
      {msg_index,34160},
      {other_ets,1080168},
      {binary,6480},
      {code,17557932},
      {atom,1565809},
      {other_system,1854067}]},
 {vm_memory_high_watermark,0.4},
 {vm_memory_limit,153295257},
 {disk_free_limit,1000000000},
 {disk_free,77554466816},
 {file_descriptors,
    
 [{total_limit,924},{total_used,3},{sockets_limit,829},{sockets_used,1}]},
 {processes,[{limit,1048576},{used,187}]},
 {run_queue,0},
 {uptime,210}]
Cluster status of node rabbit at qcluster1 ...
[{nodes,[{disc,[rabbit at qcluster2,rabbit at qcluster0]},{ram,[rabbit at qcluster1]}]},
 {running_nodes,[rabbit at qcluster0,rabbit at qcluster2,rabbit at qcluster1]},
 {partitions,[]}]
Application environment of node rabbit at qcluster1 ...
[{auth_backends,[rabbit_auth_backend_internal]},
 {auth_mechanisms,['PLAIN','AMQPLAIN']},
 {backing_queue_module,rabbit_variable_queue},
 {cluster_nodes,{[rabbit at qcluster0,rabbit at qcluster1],ram}},
 {cluster_partition_handling,pause_minority},
 {collect_statistics,fine},
 {collect_statistics_interval,5000},
 {default_permissions,[<<".*">>,<<".*">>,<<".*">>]},
 {default_user,<<"guest">>},
 {default_user_tags,[administrator]},
 {default_vhost,<<"/">>},
 {delegate_count,16},
 {disk_free_limit,1000000000},
 {enabled_plugins_file,"/etc/rabbitmq/enabled_plugins"},
 {error_logger,{file,"/var/log/rabbitmq/rabbit at qcluster1.log"}},
 {frame_max,131072},
 {heartbeat,600},
 {hipe_compile,false},
 {included_applications,[]},
 {log_levels,[{connection,info}]},
 {msg_store_file_size_limit,16777216},
 {msg_store_index_module,rabbit_msg_store_ets_index},
 {plugins_dir,"/usr/lib/rabbitmq/lib/rabbitmq_server-3.1.3/sbin/../plugins"},
 {plugins_expand_dir,"/var/lib/rabbitmq/mnesia/rabbit at qcluster1-plugins-expand"},
 {queue_index_max_journal_entries,65536},
 {reverse_dns_lookups,false},
 {sasl_error_logger,{file,"/var/log/rabbitmq/rabbit at qcluster1-sasl.log"}},
 {server_properties,[]},
 {ssl_cert_login_from,distinguished_name},
 {ssl_listeners,[]},
 {ssl_options,[]},
 {tcp_listen_options,[binary,
                      {packet,raw},
                      {reuseaddr,true},
                      {backlog,128},
                      {nodelay,true},
                      {linger,{true,0}},
                      {exit_on_close,false}]},
 {tcp_listeners,[{"auto",5672}]},
 {trace_vhosts,[]},
 {vm_memory_high_watermark,0.4}]
Status of node rabbit at qcluster2 ...
[{pid,1940},
 {running_applications,
     [{rabbitmq_management,"RabbitMQ Management Console","3.1.3"},
      {rabbitmq_management_agent,"RabbitMQ Management Agent","3.1.3"},
      {rabbit,"RabbitMQ","3.1.3"},
      {os_mon,"CPO  CXC 138 46","2.2.7"},
      {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.1.3"},
      {webmachine,"webmachine","1.9.1-rmq3.1.3-git52e62bc"},
      {mochiweb,"MochiMedia Web Server","2.3.1-rmq3.1.3-gitd541e9a"},
      {mnesia,"MNESIA  CXC 138 12","4.5"},
      {amqp_client,"RabbitMQ AMQP Client","3.1.3"},
      {xmerl,"XML parser","1.2.10"},
      {inets,"INETS  CXC 138 49","5.7.1"},
      {sasl,"SASL  CXC 138 11","2.1.10"},
      {stdlib,"ERTS  CXC 138 10","1.17.5"},
      {kernel,"ERTS  CXC 138 10","2.14.5"}]},
 {os,{unix,linux}},
 {erlang_version,
     "Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2] 
[async-threads:30] [kernel-poll:true]\n"},
 {memory,
     [{total,32032800},
      {connection_procs,5408},
      {queue_procs,5408},
      {plugins,315040},
      {other_proc,9172832},
      {mnesia,61952},
      {mgmt_db,89704},
      {msg_index,34160},
      {other_ets,1111736},
      {binary,33384},
      {code,17743902},
      {atom,1603433},
      {other_system,1855841}]},
 {vm_memory_high_watermark,0.4},
 {vm_memory_limit,153295257},
 {disk_free_limit,1000000000},
 {disk_free,77553033216},
 {file_descriptors,
    
 [{total_limit,924},{total_used,3},{sockets_limit,829},{sockets_used,1}]},
 {processes,[{limit,1048576},{used,191}]},
 {run_queue,0},
 {uptime,617}]
Cluster status of node rabbit at qcluster2 ...
[{nodes,[{disc,[rabbit at qcluster0,rabbit at qcluster2]},{ram,[rabbit at qcluster1]}]},
 {running_nodes,[rabbit at qcluster1,rabbit at qcluster0,rabbit at qcluster2]},
 {partitions,[]}]
Application environment of node rabbit at qcluster2 ...
[{auth_backends,[rabbit_auth_backend_internal]},
 {auth_mechanisms,['PLAIN','AMQPLAIN']},
 {backing_queue_module,rabbit_variable_queue},
 {cluster_nodes,{[rabbit at qcluster0,rabbit at qcluster1,rabbit at qcluster2],ram}},
 {cluster_partition_handling,pause_minority},
 {collect_statistics,fine},
 {collect_statistics_interval,5000},
 {default_permissions,[<<".*">>,<<".*">>,<<".*">>]},
 {default_user,<<"guest">>},
 {default_user_tags,[administrator]},
 {default_vhost,<<"/">>},
 {delegate_count,16},
 {disk_free_limit,1000000000},
 {enabled_plugins_file,"/etc/rabbitmq/enabled_plugins"},
 {error_logger,{file,"/var/log/rabbitmq/rabbit at qcluster2.log"}},
 {frame_max,131072},
 {heartbeat,600},
 {hipe_compile,false},
 {included_applications,[]},
 {log_levels,[{connection,info}]},
 {msg_store_file_size_limit,16777216},
 {msg_store_index_module,rabbit_msg_store_ets_index},
 {plugins_dir,"/usr/lib/rabbitmq/lib/rabbitmq_server-3.1.3/sbin/../plugins"},
 {plugins_expand_dir,"/var/lib/rabbitmq/mnesia/rabbit at qcluster2-plugins-expand"},
 {queue_index_max_journal_entries,65536},
 {reverse_dns_lookups,false},
 {sasl_error_logger,{file,"/var/log/rabbitmq/rabbit at qcluster2-sasl.log"}},
 {server_properties,[]},
 {ssl_cert_login_from,distinguished_name},
 {ssl_listeners,[]},
 {ssl_options,[]},
 {tcp_listen_options,[binary,
                      {packet,raw},
                      {reuseaddr,true},
                      {backlog,128},
                      {nodelay,true},
                      {linger,{true,0}},
                      {exit_on_close,false}]},
 {tcp_listeners,[{"auto",5672}]},
 {trace_vhosts,[]},
 {vm_memory_high_watermark,0.4}]
Status of node rabbit at qcluster0 ...
[{pid,873},
 {running_applications,
     [{rabbitmq_management,"RabbitMQ Management Console","3.1.3"},
      {rabbitmq_management_agent,"RabbitMQ Management Agent","3.1.3"},
      {rabbit,"RabbitMQ","3.1.3"},
      {os_mon,"CPO  CXC 138 46","2.2.7"},
      {rabbitmq_web_dispatch,"RabbitMQ Web Dispatcher","3.1.3"},
      {webmachine,"webmachine","1.9.1-rmq3.1.3-git52e62bc"},
      {mochiweb,"MochiMedia Web Server","2.3.1-rmq3.1.3-gitd541e9a"},
      {xmerl,"XML parser","1.2.10"},
      {inets,"INETS  CXC 138 49","5.7.1"},
      {mnesia,"MNESIA  CXC 138 12","4.5"},
      {amqp_client,"RabbitMQ AMQP Client","3.1.3"},
      {sasl,"SASL  CXC 138 11","2.1.10"},
      {stdlib,"ERTS  CXC 138 10","1.17.5"},
      {kernel,"ERTS  CXC 138 10","2.14.5"}]},
 {os,{unix,linux}},
 {erlang_version,
     "Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2] 
[async-threads:30] [kernel-poll:true]\n"},
 {memory,
     [{total,33671216},
      {connection_procs,5408},
      {queue_procs,5408},
      {plugins,452592},
      {other_proc,10742928},
      {mnesia,61984},
      {mgmt_db,10248},
      {msg_index,34160},
      {other_ets,1117456},
      {binary,35464},
      {code,17740702},
      {atom,1602625},
      {other_system,1862241}]},
 {vm_memory_high_watermark,0.4},
 {vm_memory_limit,153295257},
 {disk_free_limit,1000000000},
 {disk_free,77554319360},
 {file_descriptors,
    
 [{total_limit,924},{total_used,3},{sockets_limit,829},{sockets_used,1}]},
 {processes,[{limit,1048576},{used,196}]},
 {run_queue,0},
 {uptime,335}]
Cluster status of node rabbit at qcluster0 ...
[{nodes,[{disc,[rabbit at qcluster0,rabbit at qcluster2]},{ram,[rabbit at qcluster1]}]},
 {running_nodes,[rabbit at qcluster1,rabbit at qcluster2,rabbit at qcluster0]},
 {partitions,[]}]
Application environment of node rabbit at qcluster0 ...
[{auth_backends,[rabbit_auth_backend_internal]},
 {auth_mechanisms,['PLAIN','AMQPLAIN']},
 {backing_queue_module,rabbit_variable_queue},
 {cluster_nodes,{[rabbit at qcluster0,rabbit at qcluster1],disc}},
 {cluster_partition_handling,pause_minority},
 {collect_statistics,fine},
 {collect_statistics_interval,5000},
 {default_permissions,[<<".*">>,<<".*">>,<<".*">>]},
 {default_user,<<"guest">>},
 {default_user_tags,[administrator]},
 {default_vhost,<<"/">>},
 {delegate_count,16},
 {disk_free_limit,1000000000},
 {enabled_plugins_file,"/etc/rabbitmq/enabled_plugins"},
 {error_logger,{file,"/var/log/rabbitmq/rabbit at qcluster0.log"}},
 {frame_max,131072},
 {heartbeat,600},
 {hipe_compile,false},
 {included_applications,[]},
 {log_levels,[{connection,info}]},
 {msg_store_file_size_limit,16777216},
 {msg_store_index_module,rabbit_msg_store_ets_index},
 {plugins_dir,"/usr/lib/rabbitmq/lib/rabbitmq_server-3.1.3/sbin/../plugins"},
 {plugins_expand_dir,"/var/lib/rabbitmq/mnesia/rabbit at qcluster0-plugins-expand"},
 {queue_index_max_journal_entries,65536},
 {reverse_dns_lookups,false},
 {sasl_error_logger,{file,"/var/log/rabbitmq/rabbit at qcluster0-sasl.log"}},
 {server_properties,[]},
 {ssl_cert_login_from,distinguished_name},
 {ssl_listeners,[]},
 {ssl_options,[]},
 {tcp_listen_options,[binary,
                      {packet,raw},
                      {reuseaddr,true},
                      {backlog,128},
                      {nodelay,true},
                      {linger,{true,0}},
                      {exit_on_close,false}]},
 {tcp_listeners,[{"auto",5672}]},
 {trace_vhosts,[]},
 {vm_memory_high_watermark,0.4}]
Connections:
Channels:
Queues on /:
Exchanges on /:
name type durable auto_delete internal arguments policy
direct true false false []
amq.direct direct true false false []
amq.fanout fanout true false false []
amq.headers headers true false false []
amq.match headers true false false []
amq.rabbitmq.log topic true false false []
amq.rabbitmq.trace topic true false false []
amq.topic topic true false false []
Bindings on /:
Consumers on /:
Permissions on /:
user configure write read
guest .* .* .*
Policies on /:
Parameters on /:
...done. 

Any help would be appreciated. 
Thanks 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130806/4961e5c9/attachment.htm>


More information about the rabbitmq-discuss mailing list