[rabbitmq-discuss] Cluster with Single Message queue

[Joseph Marlin]

See the attachment on this message for details on high availability
implementation coming soon that will change the ways queues on
clusters work signficantly:

http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2011-June/013304.html

On Jun 17, 1:05 pm, Mark Steele <mste... at beringmedia.com> wrote:
> Hi Jon,
>
> Rabbit will not scale to a large throughput of messages published to a
> single queue as they all get delivered to a single node.
>
> Here's one way to do it:
>
> http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2011-March/01200...
>
> The gist is:
>
>    - Rabbit doesn't scale on per queue basis. The answer: create many
>    queues, bound to the same exchange
>    - Use a direct exchange, which will rely on routing keys to decide which
>    queue to deliver a message to
>    - Pre-declare the queues on the nodes inside your cluster, bind them
>    using a known set of routing keys (eg: queueA with routing keys 1-100 on
>    server1, queueB with routing keys 101-200 on server2, etc...). Assuming
>    durable queues here...
>    - Have your publisher randomly pick a routing key and node to connect to
>    (or use a load balancer)
>    - Use the mandatory flag to ensure your message reaches a queue, if it
>    fails re-publish using a different randomly picked routing key
>    - Have your consumers connect to all queues
>
> Using the technique described above, you can load balance incoming TCP
> connections to a cluster of rabbit nodes. Load will get distributed based on
> the randomly picked routing keys used by the publishers.
>
> The problem with this is that HA is not addressed.
>
> Mark Steele
> Bering Media Inc.+1 (416) 888-1009
>
> On Fri, Jun 17, 2011 at 11:51 AM, Jon Charette <jon.chare... at gmail.com>wrote:
>
>
>
>
>
>
>
> > Greetings,
>
> > We are investigating using RabbitMQ in a clustered environment to provide
> > load balancing and some fault tolerance for an application currently under
> > development.
>
> > I'm not a developer, I'm the lead of our Operations team.  Clustering seems
> > the best approach for my team as it simplifies our application configuration
> > significantly.
>
> > My question is this.  Our Dev team has told me that since clustering only
> > provides node fault tolerance, it doesn't add any additional gain to our
> > design.  Specifically, they mentioned that since a message queue only
> > resides on one node,
>
> > "All data/state required for the operation of a RabbitMQ broker is
> > replicated across all nodes, for reliability and scaling, with full ACID
> > properties. An exception to this are message queues, which currently only
> > reside on the node that created them, though they are visible and reachable
> > from all nodes. Future releases of RabbitMQ will introduce migration and
> > replication of message queues."
>
> > clustering doesn't allow us to scale based on the numbers of messages being
> > sent to one queue.  Is this true, or does the cluster distribute messages
> > sent to the same message queue throughout off of its member nodes?  My
> > assumption based on the documentation is that this is the case, but if a
> > node is lost, messages that were currently residing on that node are lost as
> > well due to no fault tolerance.
>
> > Thanks much.
>
> > _______________________________________________
> > rabbitmq-discuss mailing list
> > rabbitmq-disc... at lists.rabbitmq.com
> >https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss