[rabbitmq-discuss] RabbitMQ clustering
stephanion2002 at gmail.com
Mon Nov 29 10:30:47 GMT 2010
For POC it worked, but we know that since ours is a "cloud" based
application, it will scale. So we have to chose a broker carefully. Do we
have a workaround for this ?
HP R&D Division
On Mon, Nov 29, 2010 at 3:49 PM, Marek Majkowski <majek04 at gmail.com> wrote:
> You have a valid point, just a quick question...
> On Mon, Nov 29, 2010 at 07:18, Kshitiz Garg <stephanion2002 at gmail.com>
> > Hi Gustavo,
> > Sorry for replying late,
> > We don't have any figure yet, but on our pub-sub system, millions of of
> > publishers will be publishing messages.
> > Our system is as follows:
> > 1. A publisher publishes message on our rest APIs.
> > 2. From there, we first store those messages in DB.
> > 3. After storing, we publish the message on RabbitMQ setup using a load
> > balancer.
> > 4. Behind the load balancer, we have a cluster of 4 rabbitMQ nodes.
> > 5. At the consuming end, we have perl and java consumers fetching
> > 7. These internal consumers send messages to the subscribers as per the
> > subscription protocol.
> > Now, if we add 4 RabbitMQ nodes in a cluster, all the credentials,
> > and bindings get copied to each mnesia db. However, since the queues are
> > specific to nodes, the message directed to a particular queue will be
> > (if the node on which that queue was created is down). This seems to be a
> > weak link for a scalable and High availability solution we need. Will the
> > message be persisted somewhere and delivered to that queue after that
> > particular node comes up?
> > If the message is gonna lost, can we have a workaround that we create
> > of the queues explicitly on each node. Now since they are copies, the
> > exchange will send the message to all the nodes and even if a node is
> > message will be delivered to all the copies of that queue.
> > We know that Clustering!=HA, but we do need Clustering with HA. Here
> > even after clustering of 4 rabbitMQ nodes, the active-passive setup for
> > of these 4 nodes becomes the bottleneck because of the complexity of the
> > non-replication of "queues". Is there any workaround or any other setup u
> > can suggest ?
> ... have you actually tried a setup with only a single RabbitMQ node, and
> it really became a bottleneck?
> > Regards,
> > Kshitiz Garg
> > On Sun, Nov 28, 2010 at 10:01 PM, Gustavo Aquino <
> aquino.gustavo at gmail.com>
> > wrote:
> >> Hi Garg,
> >> Could you give to us more details about what you need? eg. it's
> >> throughput is per second? and etc.
> >> Regards.
> >> On Wed, Nov 24, 2010 at 10:35 AM, Kshitiz Garg <
> stephanion2002 at gmail.com>
> >> wrote:
> >>> Hi,
> >>> Can someone present a pictorial representation of 4-5 Rabbit MQ nodes
> >>> with Clustering and HA (it can also go on the RabbitMQ site). I wanted
> >>> know about a RabbitMQ setup which is able to handle millions of
> >>> I hope there would be many more users with such a requirement.
> >>> Thanks
> >>> Kshitiz Garg
> >>> _______________________________________________
> >>> rabbitmq-discuss mailing list
> >>> rabbitmq-discuss at lists.rabbitmq.com
> >>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> > _______________________________________________
> > rabbitmq-discuss mailing list
> > rabbitmq-discuss at lists.rabbitmq.com
> > https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rabbitmq-discuss