<div dir="ltr"><br><br><div class="gmail_quote">On Thu, Dec 23, 2010 at 12:26 PM, Emile Joubert <span dir="ltr"><<a href="mailto:emile@rabbitmq.com">emile@rabbitmq.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
Hi Yoav<div class="im"><br>
<br>
On 22/12/10 20:24, yoav glazner wrote:<br>
</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">
<br>
On Wed, Dec 22, 2010 at 2:10 PM, Emile Joubert <<a href="mailto:emile@rabbitmq.com" target="_blank">emile@rabbitmq.com</a><br></div><div class="im">
<mailto:<a href="mailto:emile@rabbitmq.com" target="_blank">emile@rabbitmq.com</a>>> wrote:<br>
<br>
Hi Yoav,<br>
<br>
<br>
On 21/12/10 21:35, yoav glazner wrote:<br>
<br>
This is my first post to this list, so Hi!<br>
I'm trying to design a Master/Worker implement-ion with RabbitMQ.<br>
I have a few requierments due to lagacy issues:<br>
* works get very long tasks so they should be able to get ABORT<br>
messages<br>
while working,<br>
<br>
<br>
You could send updated status information about work items to a<br>
dedicated status exchange that workers subscribe to in order to<br>
obtain abort messages or changes in priority. This assumes that<br>
tasks are interruptible and that workers have multiple threads of<br>
control.<br>
<br>
that's good<br>
</div></blockquote>
<br>
It sounds like you want to be able to communicate two different things to workers:<br>
<br>
1. tasks<br>
2. task status updates<br>
<br>
The updates (abort, reprioritise, pause) to tasks are the difficult part and the reason why frameworks like Celery won't work. I suggest you communicate task status updates to workers via a different route from the way you submit tasks. Translating that to AMQP means using using a separate exchange and queues, so workers accept tasks via a relatively slow-moving queue and obtain updates about task status via a different queue. This assumes that workers have more than one thread of control for listening on multiple queues and know how to interrupt running tasks. There may be lots of other ways of accomplishing your goal. </blockquote>
<div><br></div><div>But then I'll lose messages when a worker die, no? </div></div></div>