matthias at lshift.net
Sat Nov 3 10:33:13 GMT 2007
Ben Hood wrote:
>> I set up a cluster of 2 nodes - rabbit at node1 and rabbit at node2.
>> Several producers will connect to either node and send a series of
>> messages with topic hierarchy as routing key (for example,
>> "a.b.c.d.e.f"). Consumers will also connect to either node and will
>> register to receive some subset of what is being sent - in other
>> words, for example, some of them will request "a.#" while others
>> might want to get only "#.e.f". My goal is to make it happen if at
>> least one of the nodes in the cluster is up and running. Both
>> producers and consumers will know about existence of both cluster
>> nodes, but they need to be able to randomly connect to either.
> As far as I know, message queues are not yet fully clustered in
> rabbitmq, meaning that you cannot yet transparently failover or
> migrate a queue to a different node in the cluster. These queues are
> visible from any node in the cluster however.
Ben is right about the (lack of) queue clustering. Whether or not that
matters depends on what guarantees you expect on message delivery: Can
you afford to lose messages? Is it ok for a message to be delivered to a
consumer more than once?
You can guarantee exactly-once delivery if you use transactions, durable
queues and exchanges, and persistent messages, but only as long as any
failing node eventually recovers.
If you can afford to lose the odd message then the simplest setup is to
just have consumers detect when connections goes down, connect to the
other node and recreate their queues. Similarly, producers just
re-connect to the second node on failure of the first node, and start
sending messages to the same exchange as before.
If message loss is bad but duplication is ok then you can could set up
two parallel routing topologies with producers sending messages to both
nodes and each consumer consuming from two queues - one on each node.
There are various other options; which one is best really depends on the
specific requirements of your application.
All this is going to improve significantly in AMQP 0-10, where there is
a notion of a "session" whose lifecycle is independent from that of the
transport connection. Sessions can transparently reconnect to other
nodes and have built-in support for exactly-once delivery. We'd still
need to add queue clustering to RabbitMQ, but that's on our roadmap
More information about the rabbitmq-discuss