[rabbitmq-discuss] Cluster size limit
Matthew Sackman
matthew at rabbitmq.com
Mon Mar 28 18:19:36 BST 2011
Hi Advait,
On Sun, Mar 27, 2011 at 11:01:53AM +0530, Advait Alai wrote:
> I'm trying to add 150 nodes to a RabbitMQ cluster. After around 50 nodes,
> the stop-reset-cluster-start iteration starts giving the error:
>
> Stopping node rabbit at node130 ...
> ...done.
> Resetting node rabbit at node130 ...
> ...done.
> Clustering node rabbit at node130 with [rabbit at node117] ...
> ...done.
> Starting node rabbit at node130 ...
> Error: {cannot_start_application,rabbit,
> {bad_return,
> {{rabbit,start,[normal,[]]},
> {'EXIT',{rabbit,failure_during_boot}}}}}
Hmm, curious. I don't know why that happens. I've just successfully
built a cluster of 100 nodes. I'm building from source, so using the
Makefile. I did:
1. Edit the Makefile. Around line 153, remove the
RABBITMQ_ALLOW_INPUT=true \
line completely.
2. rabbitmq-server$ for t in $(seq 1 100); do make RABBITMQ_NODENAME=rabbit_$t RABBITMQ_NODE_PORT=$(echo "5672+$t"|bc) TMPDIR=/home/matthew/ssd/ cleandb run & done
3. rabbitmq-server/scripts$ for t in $(seq 2 100); do ./rabbitmqctl -n rabbit_$t at hazel stop_app ; ./rabbitmqctl -n rabbit_$t at hazel reset && ./rabbitmqctl -n rabbit_$t at hazel cluster rabbit_1 at hazel rabbit_$t at hazel && ./rabbitmqctl -n rabbit_$t at hazel start_app; done
4. Go make a cuppa tea.
It takes a while, and slows down on a quadratic (which isn't surprising
because we're forming an all-to-all mesh), and it eats up a lot of RAM.
But it did get there for me:
rabbitmq-server/scripts$ ./rabbitmqctl -n 'rabbit_1 at hazel' status
Status of node rabbit_1 at hazel ...
[{pid,7933},
{running_applications,[{rabbit,"RabbitMQ","%%VSN%%"},
{mnesia,"MNESIA CXC 138 12","4.4.17"},
{os_mon,"CPO CXC 138 46","2.2.5"},
{sasl,"SASL CXC 138 11","2.1.9.3"},
{stdlib,"ERTS CXC 138 10","1.17.3"},
{kernel,"ERTS CXC 138 10","2.14.3"}]},
{nodes,[{disc,[rabbit_100 at hazel,rabbit_99 at hazel,rabbit_98 at hazel,
rabbit_97 at hazel,rabbit_96 at hazel,rabbit_95 at hazel,
rabbit_94 at hazel,rabbit_93 at hazel,rabbit_92 at hazel,
rabbit_91 at hazel,rabbit_90 at hazel,rabbit_89 at hazel,
rabbit_88 at hazel,rabbit_87 at hazel,rabbit_86 at hazel,
rabbit_85 at hazel,rabbit_84 at hazel,rabbit_83 at hazel,
rabbit_82 at hazel,rabbit_81 at hazel,rabbit_80 at hazel,
rabbit_79 at hazel,rabbit_78 at hazel,rabbit_77 at hazel,
rabbit_76 at hazel,rabbit_75 at hazel,rabbit_74 at hazel,
rabbit_73 at hazel,rabbit_72 at hazel,rabbit_71 at hazel,
rabbit_70 at hazel,rabbit_69 at hazel,rabbit_68 at hazel,
rabbit_67 at hazel,rabbit_66 at hazel,rabbit_65 at hazel,
rabbit_64 at hazel,rabbit_63 at hazel,rabbit_62 at hazel,
rabbit_61 at hazel,rabbit_60 at hazel,rabbit_59 at hazel,
rabbit_58 at hazel,rabbit_57 at hazel,rabbit_56 at hazel,
rabbit_55 at hazel,rabbit_54 at hazel,rabbit_53 at hazel,
rabbit_52 at hazel,rabbit_51 at hazel,rabbit_50 at hazel,
rabbit_49 at hazel,rabbit_48 at hazel,rabbit_47 at hazel,
rabbit_46 at hazel,rabbit_45 at hazel,rabbit_44 at hazel,
rabbit_43 at hazel,rabbit_42 at hazel,rabbit_41 at hazel,
rabbit_40 at hazel,rabbit_39 at hazel,rabbit_38 at hazel,
rabbit_37 at hazel,rabbit_36 at hazel,rabbit_35 at hazel,
rabbit_34 at hazel,rabbit_33 at hazel,rabbit_32 at hazel,
rabbit_31 at hazel,rabbit_30 at hazel,rabbit_29 at hazel,
rabbit_28 at hazel,rabbit_27 at hazel,rabbit_26 at hazel,
rabbit_25 at hazel,rabbit_24 at hazel,rabbit_23 at hazel,
rabbit_22 at hazel,rabbit_21 at hazel,rabbit_20 at hazel,
rabbit_19 at hazel,rabbit_18 at hazel,rabbit_17 at hazel,
rabbit_16 at hazel,rabbit_15 at hazel,rabbit_14 at hazel,
rabbit_13 at hazel,rabbit_12 at hazel,rabbit_11 at hazel,
rabbit_10 at hazel,rabbit_9 at hazel,rabbit_8 at hazel,rabbit_7 at hazel,
rabbit_6 at hazel,rabbit_5 at hazel,rabbit_4 at hazel,rabbit_3 at hazel,
rabbit_2 at hazel,rabbit_1 at hazel]}]},
{running_nodes,[rabbit_100 at hazel,rabbit_99 at hazel,rabbit_98 at hazel,
rabbit_97 at hazel,rabbit_96 at hazel,rabbit_95 at hazel,
rabbit_94 at hazel,rabbit_93 at hazel,rabbit_92 at hazel,
rabbit_91 at hazel,rabbit_90 at hazel,rabbit_89 at hazel,
rabbit_88 at hazel,rabbit_87 at hazel,rabbit_86 at hazel,
rabbit_85 at hazel,rabbit_84 at hazel,rabbit_83 at hazel,
rabbit_82 at hazel,rabbit_81 at hazel,rabbit_80 at hazel,
rabbit_79 at hazel,rabbit_78 at hazel,rabbit_77 at hazel,
rabbit_76 at hazel,rabbit_75 at hazel,rabbit_74 at hazel,
rabbit_73 at hazel,rabbit_72 at hazel,rabbit_71 at hazel,
rabbit_70 at hazel,rabbit_69 at hazel,rabbit_68 at hazel,
rabbit_67 at hazel,rabbit_66 at hazel,rabbit_65 at hazel,
rabbit_64 at hazel,rabbit_63 at hazel,rabbit_62 at hazel,
rabbit_61 at hazel,rabbit_60 at hazel,rabbit_59 at hazel,
rabbit_58 at hazel,rabbit_57 at hazel,rabbit_56 at hazel,
rabbit_55 at hazel,rabbit_54 at hazel,rabbit_53 at hazel,
rabbit_52 at hazel,rabbit_51 at hazel,rabbit_50 at hazel,
rabbit_49 at hazel,rabbit_48 at hazel,rabbit_47 at hazel,
rabbit_46 at hazel,rabbit_45 at hazel,rabbit_44 at hazel,
rabbit_43 at hazel,rabbit_42 at hazel,rabbit_41 at hazel,
rabbit_40 at hazel,rabbit_39 at hazel,rabbit_38 at hazel,
rabbit_37 at hazel,rabbit_36 at hazel,rabbit_35 at hazel,
rabbit_34 at hazel,rabbit_33 at hazel,rabbit_32 at hazel,
rabbit_31 at hazel,rabbit_30 at hazel,rabbit_29 at hazel,
rabbit_28 at hazel,rabbit_27 at hazel,rabbit_26 at hazel,
rabbit_25 at hazel,rabbit_24 at hazel,rabbit_23 at hazel,
rabbit_22 at hazel,rabbit_21 at hazel,rabbit_20 at hazel,
rabbit_19 at hazel,rabbit_18 at hazel,rabbit_17 at hazel,
rabbit_16 at hazel,rabbit_15 at hazel,rabbit_14 at hazel,
rabbit_13 at hazel,rabbit_12 at hazel,rabbit_11 at hazel,
rabbit_10 at hazel,rabbit_9 at hazel,rabbit_8 at hazel,rabbit_7 at hazel,
rabbit_6 at hazel,rabbit_5 at hazel,rabbit_4 at hazel,rabbit_3 at hazel,
rabbit_2 at hazel,rabbit_1 at hazel]}]
...done.
Matthew
More information about the rabbitmq-discuss
mailing list