[rabbitmq-discuss] Persistent messages and durable queue questions

Fri Jul 8 18:23:18 BST 2011

On Thu, Jul 7, 2011 at 6:34 AM, Rob Harrop <rob at rabbitmq.com> wrote:

>
> Well, it's quite subtle. If the message was routed when the queue was up, a
> back is sent if the queue goes down before the message can be written to
> disk.
>
> If the binding has been removed due to queue down then no nack will be
> sent.
>
> Are you trying to ensure that a given message reaches a known set of
> queues?
>

Yes and no.  As I said, for any one message there may or may not be binding
matching the message to the durable queue.

The problem is as follows:  We have a system that generates events.  We have
a set of users that may be interested in some subset of those events.  What
events the users are interested in changes dynamically.  We have an agent
subscribed to a durable queue that will receive the events of interest and
process them on behalf of the users.  The bindings on the topic exchange
that match the events to the queue change dynamically with the user's
interests.  I need to know that if there is a binding for the event, that it
was successfully persisted to the queue.

We want to ensure that, if the queue is reachable and there is a binding
that matches the message to the queue, or the queue is reachable and there
is no binding that matches the message to the queue, (i.e. that the cluster
is working properly) that the producer will get an ack.  Publisher confirm
gives us this.

We want to ensure that, if the queue is not reachable and there is a binding
that matches the message to the queue, or the queue is not reachable and
there is no binding that matches the message to the queue, (i.e. that the
cluster has lost the node to which the queue is attached or the queue was
deleted), that the producer will get a nack.  Publisher confirm fails here.

Publisher confirm fails the second requirement because if the cluster is
partitioned, it will forget the bindings to the queue on the other node.  It
will response with an ack, rather than a nack.

What is needed is a way to define a binding as durable, so that it won't be
forgotten if the queue is references goes away because of a cluster
partition or the queue is deleted.  It gives the system a hint that the
queue in question should exist, but does not.  Alternatively, the system
should not forget a durable queue just because the node it was attached goes
away.  A durable queue should only be forgotten if it is actively deleted.

HA queue alleviate the problem, as the queue will exist in more nodes and
more nodes would have to disappear for it to be forgotten, but it does not
solve the problem, unless the queue is configured to exist on all nodes.

Savvy?

Elias Levy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20110708/3dce17a9/attachment.htm>