<div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">However - RabbitMQ clustering is not designed as a way to create<br>
pubsub overlays for the wide area. Its primary goal is scalability of<br>
one broker by adding nodes on the same LAN. This is for, eg, cases<br>
where the number of subscriptions (or "bindings") on the broker grows<br>
beyond what one machine can physically cope with.<br>
<br>
Now, this does not stop you wiring up 1,000s of RabbitMQ brokers. But<br>
using RabbitMQ clustering is the wrong way to do that. Can you tell<br>
us a bit more about the pubsub topology you want to study, please?<br>
That may help us suggest the optimal way to solve your problem..<br></blockquote><div><br></div><div>The topology looks like this: <a href="http://imgur.com/8iep0" target="_blank">http://imgur.com/8iep0</a> (taken from <a href="http://ctieware.eng.monash.edu.au/twiki/bin/view/Simulation/LargePacket-switchingNetworkTopologies" target="_blank">http://ctieware.eng.monash.edu.au/twiki/bin/view/Simulation/LargePacket-switchingNetworkTopologies</a>)</div>
<div><br></div><div>The terminal nodes shown in the topology are either publishers or subscribers, and I want RabbitMQ to run as a middleware over the entire network. The network (routers and links) are simulated by NS3, So it is essentially a set of point to point links, where each link is a separate network. The terminal nodes are VMs (that would run the amqp/rabbitmq scripts) which I am attempting to cluster so they can use a common set of queues and exchanges. So all terminal nodes are in different LANs.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Also, can you explain why QoS studies require the study of 1,000s of<br>
nodes? Perhaps you mean 1,000s of clients?<br></blockquote><div><br></div><div>I'm not sure I understood -- I'm trying to provide high fidelity results, so I thought the testbed should be atleast of the same order of size as the network on which it needs to be deployed (>50000 nodes). There are a few strict IEEE standards regarding this network that specify the minimum QoS guarantees required.</div>
<div><br></div><div>Is there an easier way to do this?</div><div><br></div><div>Thanks</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div>
On Sun, Mar 27, 2011 at 7:11 PM, Advait Alai <<a href="mailto:advaitalai@gmail.com" target="_blank">advaitalai@gmail.com</a>> wrote:<br>
> Thanks for the reply --<br>
> On Sun, Mar 27, 2011 at 9:38 PM, Jerry Kuch <<a href="mailto:jerryk@vmware.com" target="_blank">jerryk@vmware.com</a>> wrote:<br>
>><br>
>> 150 is a pretty big sounding cluster... Out of curiosity, what's<br>
>> motivating you to go so big (if you don't mind saying)?<br>
><br>
> I am doing a QoS analysis of publish-subscribe overlays using RabbitMQ in<br>
> wide area networks (These are country wide networks, so even a 1000 nodes<br>
> might be insufficient :-)) So stuff like packet delay, loss, out-of-order<br>
> delivery etc.<br>
><br>
>><br>
>> On that note, because RabbitMQ clustering is based on Erlang distribution,<br>
>> the current practical limit you'll probably run up against is somewhat lower<br>
>> than the 150 you have in mind. Something more like 32 to 64.<br>
><br>
> Is there a configuration that would let me scale to >64 nodes, even if it<br>
> would not be practical? And in case Erlang does not scale well, I'll<br>
> probably have to resort to entirely another middleware -- any suggestions<br>
> that would work on a larger number of nodes?<br>
><br>
>><br>
>> If you can say more about your goals it's likely that someone on the<br>
>> Rabbit team can suggest something helpful.<br>
><br>
> The analysis I'm carrying out on >100 nodes is actually on a single system.<br>
> These 'nodes' are actually many light weight linux containers (more or less<br>
> virtual machines) connected by a simulated NS3 network topology. But I doubt<br>
> this would be the cause of the clustering problem, as <50 nodes were<br>
> clustering without any difficulty.<br>
><br>
>><br>
>> On Mar 26, 2011, at 10:32 PM, "Advait Alai" <<a href="mailto:advaitalai@gmail.com" target="_blank">advaitalai@gmail.com</a>> wrote:<br>
>><br>
>> > Hi,<br>
>> ><br>
>> > I'm trying to add 150 nodes to a RabbitMQ cluster. After around 50<br>
>> > nodes, the stop-reset-cluster-start iteration starts giving the error:<br>
>> ><br>
>> > Stopping node rabbit@node130 ...<br>
>> > ...done.<br>
>> > Resetting node rabbit@node130 ...<br>
>> > ...done.<br>
>> > Clustering node rabbit@node130 with [rabbit@node117] ...<br>
>> > ...done.<br>
>> > Starting node rabbit@node130 ...<br>
>> > Error: {cannot_start_application,rabbit,<br>
>> > {bad_return,<br>
>> > {{rabbit,start,[normal,[]]},<br>
>> > {'EXIT',{rabbit,failure_during_boot}}}}}<br>
>> ><br>
>> > Note that I am sequentially adding nodes to build a cluster (as an<br>
>> > initialization step) before creating any queues/exchanges or running any<br>
>> > amqp script.<br>
>> ><br>
>> > How do I solve this problem? Is it because RabbitMQ imposes a hard<br>
>> > cluster size limit?<br>
>> ><br>
>> > Also, does RabbitMQ scale well to around 1000 nodes?<br>
>> ><br>
>> > Thanks<br>
>> > _______________________________________________<br>
>> > rabbitmq-discuss mailing list<br>
>> > <a href="mailto:rabbitmq-discuss@lists.rabbitmq.com" target="_blank">rabbitmq-discuss@lists.rabbitmq.com</a><br>
>> > <a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
><br>
><br>
> _______________________________________________<br>
> rabbitmq-discuss mailing list<br>
> <a href="mailto:rabbitmq-discuss@lists.rabbitmq.com" target="_blank">rabbitmq-discuss@lists.rabbitmq.com</a><br>
> <a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
><br>
><br>
</div></div></blockquote></div><br>