<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div>On 19 Jun 2013, at 10:41, Flavio Pompermaier wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">Hi to everybody,<div>I'm new to RabbitMQ but from what I see it could be the solution for a buffer implementation in one of my project.</div><div>In my scenario there will be a big queue of tasks (i.e. document indexing) that should be served as fast as possible,</div><div><br></div><div>Looking at the tutorial I think that what I need is somehow similar to the Work queues (<a href="http://www.rabbitmq.com/tutorials/tutorial-two-python.html">http://www.rabbitmq.com/tutorials/tutorial-two-python.html</a>) plus fair dispatching.</div><div>However, the aforementioned constraint "served as fast as possible" is not clear how to design a correct size of workers, in a scalable way..</div><div>In every example, a worker is a standalone java process, whereas I would like to manage workers more like a pool and take the first available one..</div><div><br></div></blockquote><div><br></div><div>You can open multiple channels on a single connection (i.e., an application running in the JVM can open many connections and many channels per connection - each channel usually being assigned to a thread). As for "take the first available one", that's exactly what the "fair dispatch" section in the tutorial describes.</div><br><blockquote type="cite"><div>2) If my poolable solution is correct, how can I set up a pool? How do I decide the best size of the pool?</div></blockquote><div><br></div><div>That's totally application specific, depending on how many messages you're expecting to process and so on. You can `re-size' the pool any time you like by adding more consumers - the prefetch-count will ensure that rabbit doesn't send any one consumer more than the specified number of un-acked messages. </div><br><blockquote type="cite"><div>3) What if I discover that my server is overloaded and I want to distribute the load? </div></blockquote><div><br></div><div>Add more consumers, as I said above. </div><div><br></div><div><blockquote type="cite"><div>How can I avoid to store all the queues message in just one server (if I want persistable queues)?</div></blockquote><div><div><br></div></div><div>You can set up a cluster and use HA to mirror (i.e., replicate) the queue contents across multiple nodes. See <a href="http://www.rabbitmq.com/ha.html">www.rabbitmq.com/ha.html</a> for details. Of course, with HA you'll store all the queue's messages on all of the (mirrored) nodes. If you were thinking of 'sharding' the data across multiple nodes, then we don't offer a solution for that out of the box, though you could probably cook one up yourself using federation (see <a href="http://www.rabbitmq.com/federation.html">www.rabbitmq.com/federation.html</a>).</div><div><br></div><div>Cheers,</div><div>Tim</div></div></div></body></html>