[rabbitmq-discuss] Work Queues design
pompermaier at okkam.it
Wed Jun 19 10:41:14 BST 2013
Hi to everybody,
I'm new to RabbitMQ but from what I see it could be the solution for a
buffer implementation in one of my project.
In my scenario there will be a big queue of tasks (i.e. document indexing)
that should be served as fast as possible,
Looking at the tutorial I think that what I need is somehow similar to the
Work queues (http://www.rabbitmq.com/tutorials/tutorial-two-python.html)
plus fair dispatching.
However, the aforementioned constraint "served as fast as possible" is not
clear how to design a correct size of workers, in a scalable way..
In every example, a worker is a standalone java process, whereas I would
like to manage workers more like a pool and take the first available one..
So, I think that I can summarize my questions as:
1) What is the best RabbitMQ architecture to solve my problems?
2) If my poolable solution is correct, how can I set up a pool? How do I
decide the best size of the pool?
3) What if I discover that my server is overloaded and I want to distribute
the load? How can I avoid to store all the queues message in just one
server (if I want persistable queues)?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rabbitmq-discuss