[rabbitmq-discuss] Help with my priority queue design

[Emile Joubert]

Hi yoav,

On 22/02/11 20:54, yoav glazner wrote:
> Hi,
>
> I wanted to show my design for a priority based task queue and get
> some insight from experts.
>
> The client create a queue named: "some_tasks_to_work_on2" and put his
>  tasks there, then it puts the following message in a queue named
> "batch_states": [{batch="some_tasks_to_work_on2", priority=3}] now
> each worker can take the status message and get tasks from the
> batch. if a few batches are in the status message then the worker
> check the queue by the priority order.
>
> a worker should also return the status message after reading it so
> other workers can also read it. now this is needed because i want
> workers to be able to wake up any time and get the latest status. In
> this approch I can add new batches on the fly and also change
> priorities.
>
> Do you think there is a better way to do it?

The usual way of implementing priorities is to use separate queues for 
each priority. So if you need two priority levels then declare queues Q1 
and Q2. Route messages to those queues based on priority and have 
clients consume from queues in that order. So Q2 messages won't be 
consumed unless Q1 is empty.

You might find this simpler than using indirection, which is what you 
appear to be doing.



Emile