<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Have you looked into celery? <a href="http://celeryproject.org/">http://celeryproject.org/</a>. You might be able to bolt the additional features you need on top or use it as-is with creative segmenting / driver scripts.<div><br></div><div><br><div><div><div>On Dec 22, 2010, at 12:24 PM, yoav glazner wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div dir="ltr"><div class="gmail_quote"><br><div dir="ltr"><div class="gmail_quote"><div class="im">On Wed, Dec 22, 2010 at 2:10 PM, Emile Joubert <span dir="ltr"><<a href="mailto:emile@rabbitmq.com" target="_blank">emile@rabbitmq.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Yoav,<div><br>
<br>
On 21/12/10 21:35, yoav glazner wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
This is my first post to this list, so Hi!<br>
I'm trying to design a Master/Worker implement-ion with RabbitMQ.<br>
I have a few requierments due to lagacy issues:<br>
* works get very long tasks so they should be able to get ABORT messages<br>
while working,<br>
</blockquote>
<br></div>
You could send updated status information about work items to a dedicated status exchange that workers subscribe to in order to obtain abort messages or changes in priority. This assumes that tasks are interruptible and that workers have multiple threads of control.</blockquote>
</div><div>that's good </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im"><div><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
* no task can be lost<br>
</blockquote>
<br></div></div>
The use persistence and transactions is indicated.\</blockquote><div>yes, I saw that. </div><div class="im"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div> </div>
</blockquote>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
* each task has priority and that priority may change when the task is<br>
already in a queue.<br>
</blockquote>
<br></div>
If the task is stuck in a long queue you might be able to send a cancel message via the status exchange and requeue the task at the correct priority.</blockquote><div> </div></div><div>I have no idea what this means :( </div>
<div class="im"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
* tasks can be removed from the work queue (user wants to pause/cancel<br>
an operation)<br>
</blockquote>
<br></div>
Depending on exactly what pausing means, this could be accomplished to sending an appropriate message to the status exchange.</blockquote></div><div>again, no clue on this, what is the "status exchange"? </div>
<div class="im"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
* An client of rabbitMQ can fetch the Pending tasks to know what is the<br>
status of the big operation.<br>
</blockquote>
<br></div>
If you explain what a bit more about your problem I might be able to make a suggestion here - what is a big operation?<div><br></div></blockquote></div><div>This big operation is a crypto-calculation on many number, the operation is devided to many tasks (each can take a few hours), so I want to be able to know what is the operation status. (I can have a few big operations in the same time)</div>
<div class="im">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
After googling I found out that for priorities I'll need a few work<br>
queues, but will I be able to switch a priority of a task in one<br>
transaction?<br>
</blockquote>
<br></div>
No.<br></blockquote></div><div class="im"><div>ok, but You said:</div><div>"If the task is stuck in a long queue you might be able to send a cancel message via the status exchange and requeue the task at the correct priority." </div>
</div><div>so I guess this will be transaction-less?</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
<br>
Regards<br><font color="#888888">
<br>
Emile<br>
</font></blockquote></div><br></div><div dir="ltr">Thanks!</div>
</div><br></div>
_______________________________________________<br>rabbitmq-discuss mailing list<br><a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss<br></blockquote></div><br></div></div></body></html>