[rabbitmq-discuss] Work Queues design
zachary.w.austin at gmail.com
Wed Jul 10 01:10:19 BST 2013
If you haven't already, you may want to investigate setting up competing
On Wednesday, June 19, 2013 4:41:14 AM UTC-5, Flavio Pompermaier wrote:
> Hi to everybody,
> I'm new to RabbitMQ but from what I see it could be the solution for a
> buffer implementation in one of my project.
> In my scenario there will be a big queue of tasks (i.e. document indexing)
> that should be served as fast as possible,
> Looking at the tutorial I think that what I need is somehow similar to the
> Work queues (http://www.rabbitmq.com/tutorials/tutorial-two-python.html)
> plus fair dispatching.
> However, the aforementioned constraint "served as fast as possible" is not
> clear how to design a correct size of workers, in a scalable way..
> In every example, a worker is a standalone java process, whereas I would
> like to manage workers more like a pool and take the first available one..
> So, I think that I can summarize my questions as:
> 1) What is the best RabbitMQ architecture to solve my problems?
> 2) If my poolable solution is correct, how can I set up a pool? How do I
> decide the best size of the pool?
> 3) What if I discover that my server is overloaded and I want to
> distribute the load? How can I avoid to store all the queues message in
> just one server (if I want persistable queues)?
> Best regards,
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rabbitmq-discuss