[rabbitmq-discuss] Scheduling algorithms

[Matthew Sackman]

Hi Dragan,

On Fri, Dec 25, 2009 at 11:41:58AM +0100, Dragan Zubac wrote:
> There are processes A,B and C that insert messages into RabbitMQ.
> All this messages should reach point D.

Right, so you have one exchange, and one queue, and you have D
consuming from that queue, and you have A, B and C publishing to the
exchange to which the queue is bound.

> If A,B and C insert various number of messages into RabbitMQ,what
> algorithms will RMQ use to deliver those messages to D ?

Delivery order is not defined except for messages from a single
publisher.

> Examples :
> 
> A inserts 100 messages
> B inserts 100 messages
> C inserts 100 messages

What, in parallel? Sequentially? Interleaved? In Rabbit, within a
one-node cluster, order is only determined once the message gets to the
queue. The exchange is a passive entity and can be used in parallel by
several channels.

> Will RMQ send all messages toward D in round-robin fashion or the
> delivery order will be undermined ? Or maybe in some other fashion ?

Once the messages are in the queue, they are delivered round-robin to
the consumers (with certain caveats). But in your case, you only have
one consumer so this doesn't affect you.

>From your terminology, I suspect you're thinking in a JMS-style way. You
may wish to read some of the introductions to AMQP.

> A inserts 100 messages
> B inserts 100 messages
> C inserts 1000 messages
> 
> What mechanisms one might have to deliver all this messages to D in
> desired fashion ? Somebody would like to deliver them in round-robin
> fashion (send 1 A's message,send 1 B's message,send 1 C's
> message),others might want to give higher priority to C's messages
> or so (example : send 10 A's messages,send 10 B's messages,send 100
> C's messages),or some other model of delivery that business model
> could desire ?

In AMQP, you never ever know to whom you're sending a message. You
publish a message to an exchange. After that, the publisher has (almost)
no idea what happens to the message. This is by design.

If, in the queue, you have A's 100 messages at the start, then B's 100
messages, then C's 1000 messages at the end, then that's the order
they'll come out in. You can't alter that. Also, Rabbit doesn't yet
implement message priorities.

What you could do instead, is to have A, B and C publish in such a way
that their messages get routed to three different queues. Then D can
consume from those queues as it wants - if it just wants all of A's
messages, it can just consume from that, and if it wants to force
round-robin, then by setting QoS prefetch count to 1, and careful use of
acking, it can consume from all three queues in a round robin manner.

Best wishes,

Matthew