[rabbitmq-discuss] Prioritizing Between Queues

Mon Apr 22 12:14:00 BST 2013

On 21/04/13 12:39, Tomer Paz wrote:
> Hi
> Two issues, starting with the simple one:
>
> 1. If a consumer (sub) declared a queue with topic binding key that has
> wild pattern, like "type_a.*"
>      Then a publisher declared a queue that meets this binding key as
> well via routing key "type_a.something".

Bear in mind this is not really declaring a queue. Publishers stick 
routing keys on messages. At the moment the message is received by the 
broker, the routing key is compared to the bindings that exist, and the 
message is added to all matching queues.

>      Will the broker dynamically 'subscribe' the consumer above to the
> new queue created by the publisher? even though the consumer did not
> 'declare' such queue specifically, during its queue declaration? i.e.
> will broker push messages to that consumer from the new queue
> dynamically at run time?

As far as I understand what you mean, the answer is yes.

> 2.I have read everything about AMQP in-queue priority feature requirements.
>    I have not found if there is a way for RabbitMQ broker to do the
> following however:
>
>    let clients subscribe to several queues, But prioritize which queue
> they take messages from, in a fashion different than the simple round-robin.
>    In other words, we wish to have some algo that Broker could decide
> about *prioritizing queues* for consumers, rather than simple
> round-robin to them.
>    Example:
>     we have Q1 responsible for messages of task type A. Type A has
> priority 1
>     we have Q2 responsible for messages of task type B. Type B has
> priority 2
>     .... Q3... Type C.. priority 3.
>    Now consumer client(s) subscribe to consume messages from all Queues
> (1..3).
>    We wish Broker could give different priority for the three queues. It
> would be even more powerful if for example:
>    At some point, there are very little # of messages in Q1. However Q3
> is 'full', or to be more sophisticated, the Rate in which tasks in Q3
>    handled is too slow. We wish to Dynamically change priority for the
> three queues in such case (thus dynamic prioritization of queues)

Hmm. You can have your application consume from all three queues and 
control its own rate of consumption (possibly cancelling consumption 
from lower priority queues if it is too busy with higher priority ones). 
That does put more onus on the consumer though.

Cheers, Simon

-- 
Simon MacMullen
RabbitMQ, VMware