[rabbitmq-discuss] Cluster size limit

[Advait Alai]

Thanks for the reply --

On Sun, Mar 27, 2011 at 9:38 PM, Jerry Kuch <jerryk at vmware.com> wrote:

> 150 is a pretty big sounding cluster...  Out of curiosity, what's
> motivating you to go so big (if you don't mind saying)?
>

I am doing a QoS analysis of publish-subscribe overlays using RabbitMQ in
wide area networks (These are country wide networks, so even a 1000 nodes
might be insufficient :-)) So stuff like packet delay, loss, out-of-order
delivery etc.


> On that note, because RabbitMQ clustering is based on Erlang distribution,
> the current practical limit you'll probably run up against is somewhat lower
> than the 150 you have in mind.  Something more like 32 to 64.
>

Is there a configuration that would let me scale to >64 nodes, even if it
would not be practical? And in case Erlang does not scale well, I'll
probably have to resort to entirely another middleware -- any suggestions
that would work on a larger number of nodes?


> If you can say more about your goals it's likely that someone on the Rabbit
> team can suggest something helpful.
>

The analysis I'm carrying out on >100 nodes is actually on a single system.
These 'nodes' are actually many light weight linux containers (more or less
virtual machines) connected by a simulated NS3 network topology. But I doubt
this would be the cause of the clustering problem, as <50 nodes were
clustering without any difficulty.



>
> On Mar 26, 2011, at 10:32 PM, "Advait Alai" <advaitalai at gmail.com> wrote:
>
> > Hi,
> >
> > I'm trying to add 150 nodes to a RabbitMQ cluster. After around 50 nodes,
> the stop-reset-cluster-start iteration starts giving the error:
> >
> > Stopping node rabbit at node130 ...
> > ...done.
> > Resetting node rabbit at node130 ...
> > ...done.
> > Clustering node rabbit at node130 with [rabbit at node117] ...
> > ...done.
> > Starting node rabbit at node130 ...
> > Error: {cannot_start_application,rabbit,
> >            {bad_return,
> >                {{rabbit,start,[normal,[]]},
> >                 {'EXIT',{rabbit,failure_during_boot}}}}}
> >
> > Note that I am sequentially adding nodes to build a cluster (as an
> initialization step) before creating any queues/exchanges or running any
> amqp script.
> >
> > How do I solve this problem? Is it because RabbitMQ imposes a hard
> cluster size limit?
> >
> > Also, does RabbitMQ scale well to around 1000 nodes?
> >
> > Thanks
> > _______________________________________________
> > rabbitmq-discuss mailing list
> > rabbitmq-discuss at lists.rabbitmq.com
> > https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110327/05b20fdc/attachment.htm>