[rabbitmq-discuss] RabbitMQ for Master worker with monitoring?

[yoav glazner]

On Wed, Dec 22, 2010 at 2:10 PM, Emile Joubert <emile at rabbitmq.com> wrote:

> Hi Yoav,
>
>
> On 21/12/10 21:35, yoav glazner wrote:
>
>> This is my first post to this list, so Hi!
>> I'm trying to design a Master/Worker implement-ion with RabbitMQ.
>> I have a few requierments due to lagacy issues:
>> * works get very long tasks so they should be able to get ABORT messages
>> while working,
>>
>
> You could send updated status information about work items to a dedicated
> status exchange that workers subscribe to in order to obtain abort messages
> or changes in priority. This assumes that tasks are interruptible and that
> workers have multiple threads of control.

that's good

>
>
>  * no task can be lost
>>
>
> The use persistence and transactions is indicated.\

yes, I saw that.

>
>

>  * each task has priority and that priority may change when the task is
>> already in a queue.
>>
>
> If the task is stuck in a long queue you might be able to send a cancel
> message via the status exchange and requeue the task at the correct
> priority.


I have no idea what this means :(

>
>
>  * tasks can be removed from the work queue (user wants to pause/cancel
>> an operation)
>>
>
> Depending on exactly what pausing means, this could be accomplished to
> sending an appropriate message to the status exchange.

again, no clue on this, what is the "status exchange"?

>
>
>  * An client of rabbitMQ can fetch the Pending tasks to know what is the
>> status of the big operation.
>>
>
> If you explain what a bit more about your problem I might be able to make a
> suggestion here - what is a big operation?
>
> This big operation is a crypto-calculation on many number, the operation is
devided to many tasks (each can take a few hours), so I want to be able to
know what is the operation status. (I can have a few big operations in the
same time)

>
>  After googling I found out that for priorities I'll need a few work
>> queues, but will I be able to switch a priority of a task in one
>> transaction?
>>
>
> No.
>
ok, but You said:
"If the task is stuck in a long queue you might be able to send a cancel
message via the status exchange and requeue the task at the correct
priority."
so I guess this will be transaction-less?

>
>
>
> Regards
>
> Emile
>

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20101222/e91cf392/attachment-0001.htm>