<br><div class="gmail_quote">On Tue, Mar 23, 2010 at 1:31 PM, Jonathan Stott <span dir="ltr"><<a href="mailto:jonathan.stott@gmail.com">jonathan.stott@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
Hi all<br>
<br>
I'm setting up a simple webservice. Since it involves quite a few file operations and some potentially long running tasks, I want those tasks handled by a separate process (or potentially, processes). This also lets me separate concerns. So I want to use rabbitmq to communicate between the two (and handle the mechanics of queueing jobs).<br>
<br>
The job submission side is easy. I route messages to a queue the worker(s) is subscribed to. It pulls the jobs off the queue, does the work. But what if I want to send back a confirmation? The obvious way is to set up a queue to route back messages to the front end process. I can even send a 'return to' 'address' along with the job.<br>
<br>
My question is: How many queues do I need to send back to?<br>
<br>
If I have one queue for the return messages, then how do I split out different job outputs from the stream? so that I send correct status reports to each client.<br>
<br>
But if each job has the webservice setup a queue just to listen for the reply, I'll be setting up and tearing down queues all the time.<br>
<br>
So which way would people recommend or am I thinking about this the wrong way? Is there an option I've missed? Is there something which would fit better than rabbit?<br>
<br>
Regards<br>
Jon<br>
<br></blockquote><div><br>I use something similar for "load balancing" some scripting requests in my system. Essentially two queues and one manager that exposes the API and communicates with the workers over the two queues, one work request queue and one for replies. The requests get a correlation id and the manager uses that to find the appropriate client request to return the result to.<br>
<br>Now, this uses Erlang, where the gen_server behaviour copes rather elegantly with long running server requests using noreply return values from the initial call and later sending the reply using the gen_server:reply/2 function.<br>
<br>In essence, the initial work call creates a correlation id and stores that together with the callers process id. When a worker sends a reply, it quotes the request's correlation id. The manager then uses that to find the pid that needs to receive the result.<br>
<br>Sorry for the slight vagueness of this, I'm on holiday and it would take me a little too long to swap the knowledge back into my brain at the moment.<br><br>Robby<br><br></div></div>