[rabbitmq-discuss] How to structure Master with many slaves? RPC??

[theBuckWheat]

I have a list of discrete tasks sitting on the master server and a (dynamic)
list of remote slaves, and I want to pick a slave from the list and send it
one of these tasks and then do the same for the remaining tasks until all
available slaves are busy.  

After some (variable) time, a slave will be finished and can send the result
file back to the master, after which it will be available to accept the next
task.  
 
So, the architecture issues with respect to using RabbitMQ are:

1) How best to structure pushing work to slaves;  At present, on the master
I keep a table of all slaves and a status flag.  If I set the status to
available, the master (based on a priority selection scheme) can pick one
off the list and push work to it.  The status is then set to 'busy'.  I can
set the status to unavailable at any time and that only removes the slave
from the list of slaves the master can push work to.  

2) How to structure fetching the finished work file?  Push or pull?  At
present, when a slave is finished, it calls the master and only sets a flag. 
Based on a priority scheme, the master will eventually pull the finished
work file from that slave, after which it marks that slave as being
available for work.

The load-balance model seems to apply, but only if I can control which slave
gets the message.  I don't want to broadcast to all slaves due to bandwidth
constraints.  As I read the various queue configurations, I understand how
each can be used, but I don't see a published one that applies to this
particular master-slave configuration because of its dynamic nature.   This
leads me to ponder the RPC model.  Am I missing something?

--tBW








--
View this message in context: http://rabbitmq.1065348.n5.nabble.com/How-to-structure-Master-with-many-slaves-RPC-tp22782p22804.html
Sent from the RabbitMQ mailing list archive at Nabble.com.