[rabbitmq-discuss] RabbitMQ for Master worker with monitoring?

Thu Dec 23 13:25:42 GMT 2010

On Thu, Dec 23, 2010 at 12:26 PM, Emile Joubert <emile at rabbitmq.com> wrote:

>
> Hi Yoav
>
>
> On 22/12/10 20:24, yoav glazner wrote:
>
>>
>> On Wed, Dec 22, 2010 at 2:10 PM, Emile Joubert <emile at rabbitmq.com
>> <mailto:emile at rabbitmq.com>> wrote:
>>
>>    Hi Yoav,
>>
>>
>>    On 21/12/10 21:35, yoav glazner wrote:
>>
>>        This is my first post to this list, so Hi!
>>        I'm trying to design a Master/Worker implement-ion with RabbitMQ.
>>        I have a few requierments due to lagacy issues:
>>        * works get very long tasks so they should be able to get ABORT
>>        messages
>>        while working,
>>
>>
>>    You could send updated status information about work items to a
>>    dedicated status exchange that workers subscribe to in order to
>>    obtain abort messages or changes in priority. This assumes that
>>    tasks are interruptible and that workers have multiple threads of
>>    control.
>>
>> that's good
>>
>
> It sounds like you want to be able to communicate two different things to
> workers:
>
> 1. tasks
> 2. task status updates
>
> The updates (abort, reprioritise, pause) to tasks are the difficult part
> and the reason why frameworks like Celery won't work. I suggest you
> communicate task status updates to workers via a different route from the
> way you submit tasks. Translating that to AMQP means using using a separate
> exchange and queues, so workers accept tasks via a relatively slow-moving
> queue and obtain updates about task status via a different queue. This
> assumes that workers have more than one thread of control for listening on
> multiple queues and know how to interrupt running tasks. There may be lots
> of other ways of accomplishing your goal.

But then I'll lose messages when a worker die, no?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20101223/e36507ce/attachment.htm>