[rabbitmq-discuss] Looking for some clarification on mirrored queue implementation

[Emile Joubert]

Hi Matt,

On 06/02/12 19:27, Matt Pietrek wrote:
> "The slaves apply the operations that occur to the master in exactly the
> same order as the master and thus maintain the same state. All actions
> other than publishes go only to the master, and the master then
> broadcasts the effect of the actions to the slaves. Thus clients
> consuming from a mirrored queue are in fact consuming from the master."
> 
> and
> 
> "messages published to a mirrored-queue are always published directly to
> the master and all slaves."
> --
> 
> One interpretation of the 2nd quote above ("published directly to...")
> is that clients are responsible for writing their messages to the master
> *as well as* to all slaves. Perhaps there's some mechanism where at
> connect time, the master sends a list of servers to the connecting
> client, and the client writes to all instances in the list.

No, that won't work because the number of slaves can change over the
lifetime of the queue. Publishing clients are not required to alter
their behaviour if their messages are routed to mirrored queues.

> Other people have a different interpretation of how this works, which is
> that publishes to *any* broker instances are forwarded by the receiving
> slave to the master instance, which in turn pushes the publish requests
> to all slaves. (Of course, if the master received the message initially,
> there's no need for forwarding.)
> 
> Simple question: Which of the above interpretations is correct?

Neither. RabbitMQ uses a separate Erlang process per channel and this
channel process is responsible for sending publish messages to each
slave as well as the master. The slaves and master make use of a
separate fault-tolerant framework (Guaranteed Multicast) to communicate.
The master uses GM to communicate all messages (including publish
messages) to slaves. Slaves therefore receive publish messages from the
channel as well as GM, and all other messages from GM only.

> Follow up question: What is the exact flow for a publish that goes to a
> slave? Does the slave do anything other than push the message to the
> master, which in turn handles the message as if it came to the master in
> the first place? Perhaps it writes the message to the local store first,
> then pushes it to the master? (Like I said, spirited debate.)

Slaves do not push messages to the master. Slaves simply mimic the
actions of the master in order to have an identical copy of the queue.
The messages from the master over GM impose a consistent ordering across
all the queues.


The sources contain some further detail which may be of interest:
http://hg.rabbitmq.com/rabbitmq-server/file/default/src/rabbit_mirror_queue_coordinator.erl
http://hg.rabbitmq.com/rabbitmq-server/file/default/src/gm.erl



-Emile