[rabbitmq-discuss] some questions about rabbitmq clustering, answered

Thu Nov 27 16:11:21 GMT 2008

Hi all,

We were approached by Gunnar Djurberg, a RabbitMQ user, off-list who
had some questions about clusters in RabbitMQ.  Thank-you Gunnar for
bringing these up and giving permission to share the conversation
here.

Here are the (edited) questions and answers:

> I would like to understand how clustering works.  I.e. what
> information/state is shared across nodes in the cluster,
> and what really constitutes a cluster-node.

The cluster is a set of nodes that shares a routing table.  The
concept of node is specific to RabbitMQ and not to AMQP, and is
derived from the underlying erlang/OTP, "node".   Each AMQP queue
exists on at most one node in the cluster, and if it is persistent then
it persists to a disk accessible from that node.  By contrast the
routing table is defined by a set of AMQP exchanges, and that data is
shared across all nodes in the cluster.

A message arriving at a given node will be routed to appropriate
queues via AMQP bindings and exchange routing rules.  For queues on
that node, this means 'zero hops', and for queues on other nodes, this
means 'one hop'.

> For example, I would expect exchanges, queues and bindings to be
> accesible to each other (ie to exist) across RabbitMQ nodes within the
> cluster.

Bindings are routing rules expressing a relationship between exchanges
and queues.  The cluster's routing table is the collection of all
these rules as expressed in this way using bindings and exchanges.
This routing table is shared across the cluster.  Queues are *visible*
and *accessible* throughout the cluster because messages can be routed
to them.  However each queue's data lives on one node (and possibly
also on disk).

> I would have thought that the cluster could be described as a
> virtual host that extends across several nodes/physical
> nodes - but is this correct?

Typically a set of physical nodes would be some machines on a LAN.  A
cluster would extend across this.  In erlang/OTP you can run more than
one node per physical machine, for example you might have one per
processing unit on an SMP.

> a queue is accessible (to consumers) across the cluster
> though it is located on a specific node?

Correct.

This is in the same sense that in a web application, you might access
a data store, located in one place, through a number of web sessions,
located in multiple places.

> Does this mean that the cluster handles the delivery of message from
> queue to consumer or does the consumer need to know the physical
> location of the queue and connect to that node?

A RabbitMQ cluster is a single logical AMQP server. Consumers and
producers are AMQP clients. It does not matter which node in the cluster
they connect to. End-to-end message routing involves at most two hops
inside the cluster: from the node to which the producer is connected to
the node on which the queue resides, and from there to the node to
which the consumer is connected.

So:

* RabbitMQ clusters are responsible for pushing data to consumers.
  So yes the cluster handles the delivery of the message.

* There is also a sense in which each queue is responsible for
   buffering messages on behalf of the consumer that has created
   that queue (or: consumers that share that queue).
   Therefore the consumer must have a physical connection with
   any nodes containing queues from which messages are being
   delivered to it.  But the cluster takes care of this for you.

Best wishes,

alexis