[rabbitmq-discuss] Cluster size limit

Sun Mar 27 20:27:12 BST 2011

Advait

Interesting project!

However - RabbitMQ clustering is not designed as a way to create
pubsub overlays for the wide area.  Its primary goal is scalability of
one broker by adding nodes on the same LAN.  This is for, eg, cases
where the number of subscriptions (or "bindings") on the broker grows
beyond what one machine can physically cope with.

Now, this does not stop you wiring up 1,000s of RabbitMQ brokers.  But
using RabbitMQ clustering is the wrong way to do that.  Can you tell
us a bit more about the pubsub topology you want to study, please?
That may help us suggest the optimal way to solve your problem..

Also, can you explain why QoS studies require the study of 1,000s of
nodes?  Perhaps you mean 1,000s of clients?

alexis

On Sun, Mar 27, 2011 at 7:11 PM, Advait Alai <advaitalai at gmail.com> wrote:
> Thanks for the reply --
> On Sun, Mar 27, 2011 at 9:38 PM, Jerry Kuch <jerryk at vmware.com> wrote:
>>
>> 150 is a pretty big sounding cluster...  Out of curiosity, what's
>> motivating you to go so big (if you don't mind saying)?
>
> I am doing a QoS analysis of publish-subscribe overlays using RabbitMQ in
> wide area networks (These are country wide networks, so even a 1000 nodes
> might be insufficient :-)) So stuff like packet delay, loss, out-of-order
> delivery etc.
>
>>
>> On that note, because RabbitMQ clustering is based on Erlang distribution,
>> the current practical limit you'll probably run up against is somewhat lower
>> than the 150 you have in mind.  Something more like 32 to 64.
>
> Is there a configuration that would let me scale to >64 nodes, even if it
> would not be practical? And in case Erlang does not scale well, I'll
> probably have to resort to entirely another middleware -- any suggestions
> that would work on a larger number of nodes?
>
>>
>> If you can say more about your goals it's likely that someone on the
>> Rabbit team can suggest something helpful.
>
> The analysis I'm carrying out on >100 nodes is actually on a single system.
> These 'nodes' are actually many light weight linux containers (more or less
> virtual machines) connected by a simulated NS3 network topology. But I doubt
> this would be the cause of the clustering problem, as <50 nodes were
> clustering without any difficulty.
>
>>
>> On Mar 26, 2011, at 10:32 PM, "Advait Alai" <advaitalai at gmail.com> wrote:
>>
>> > Hi,
>> >
>> > I'm trying to add 150 nodes to a RabbitMQ cluster. After around 50
>> > nodes, the stop-reset-cluster-start iteration starts giving the error:
>> >
>> > Stopping node rabbit at node130 ...
>> > ...done.
>> > Resetting node rabbit at node130 ...
>> > ...done.
>> > Clustering node rabbit at node130 with [rabbit at node117] ...
>> > ...done.
>> > Starting node rabbit at node130 ...
>> > Error: {cannot_start_application,rabbit,
>> >            {bad_return,
>> >                {{rabbit,start,[normal,[]]},
>> >                 {'EXIT',{rabbit,failure_during_boot}}}}}
>> >
>> > Note that I am sequentially adding nodes to build a cluster (as an
>> > initialization step) before creating any queues/exchanges or running any
>> > amqp script.
>> >
>> > How do I solve this problem? Is it because RabbitMQ imposes a hard
>> > cluster size limit?
>> >
>> > Also, does RabbitMQ scale well to around 1000 nodes?
>> >
>> > Thanks
>> > _______________________________________________
>> > rabbitmq-discuss mailing list
>> > rabbitmq-discuss at lists.rabbitmq.com
>> > https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>