[rabbitmq-discuss] Why Clustering ? Load Balancing

[tluck]

Hi Simon,

Thanks for your reply. 

>I assume that each consumer is churning through messages quite fast? It 
would need to be in order for the short pause mentioned about to be 
enough to have only 5-8 messages unacknowledged in this scenario. 

Consumers are taking long time to consume each message (30+ seconds/
message). While consuming a message each consumer has to do some tasks (DB
update, Data Scrapping, sending email etc...) and all this takes certainly a
good amount of time. I also tried to slow down message consuming process by
adding sleep function in consumer code, but still I see only  5-8 messages
being consumed at a time by all the consumers. This means, load is not being
properly balanced among the consumers.


> I'm not sure what this means. From your description it sounds like you 
*do* have a cluster. You say you have multiple RabbitMQ servers, and you 
are observing the message stats for all of them in one management 
interface? That sounds like a cluster to me. 

Sorry for not being clear. I've one producer server which is responsible for
creating queues and adding messages into them. RabbitMQ management plugin is
also installed in the producer server. On the other hand, I have 3 consumers
(on different servers) and every consumer sever is running 5 parallel
processes (because I've 5 queues) . Please also see the attached file to
understand producer and consumer association.  ProducerConsumerDiagram.pdf
<http://rabbitmq.1065348.n5.nabble.com/file/n26292/ProducerConsumerDiagram.pdf> 
. RabbitMQ has tutorial/guidelines (at
http://www.rabbitmq.com/clustering.html) for implementing clustering on
rabbitmq servers. This tutorial asks developers to copy erlang.cookie from
one server to another and there is step wise process described to apply
clustering on the rabbitmq severs. When started to setup my severs, I never
followed such tutorial (I mean, i didn't copy cookie or ran join_cluster or
any other clustering related commands), But all my consumer servers are
still being able to communicate with producer server through private IP. I'm
trying to say that I didn't follow clustering guidelines provided by
Rabbitmq, is this can be the possible cause for my problem  ? (load should
be balanced properly among the consumers).

Thanks
T.luck










--
View this message in context: http://rabbitmq.1065348.n5.nabble.com/Why-Clustering-Load-Balancing-tp26284p26292.html
Sent from the RabbitMQ mailing list archive at Nabble.com.