[rabbitmq-discuss] Persistent messages and durable queue questions

[Rob Harrop]

Elias Levy wrote:
> Good day,
>
> I was wondering whether someone my know the answer to the following
> questions.  I've read the FAQ and manuals, but they are not completely
> clear.
>
> The first question is does a persistent message will hit the disk even
> if there is a consumer on the queue the message is being delivered to,
> and do automatic acknowledgements (i.e. no-wait) make a difference?
>
> We are in a situation where writing to disk is expensive, and usually
> there will be a waiting and willing consumer attached to the queue, but
> we are not using automatic acks, rather we ack the message when the
> consumer has processed it, which may take a second or two.  So we'd like
> to know whether in the common case in this situation messages will be
> written to disk or not.
>
> The docs are unclear.  One could interpret them to mean that if there is
> a willing consumer the message will not be written to disk, but its
> unclear if this requires no-wait to be true.

When there is a consumer ready to receive the message, the message will 
only be written to disk if:

1. The queue is durable
2. The message is peristent
3. Automatic ack is disabled

>
>
> The second question is what happens to persistent messages coming in
> through a node that are to be routed to a durable queue attached to a
> different node?
>
> I believe this is answered by a question in the FAQ, "What happens to
> messages when a network outage removes the route between an exchange and
> a bound queue?", which states that the messages will be silently dropped
> unless I set the mandatory flag on basic.publish.
>
> Alas, that is not feasible in our use-case.  We are publishing to a
> topic exchange and creating bindings to to match a subset of the
> published messages to some durable queues.  Thus, we actually expect
> many messages to be routed to no queue.  Even if we set the mandatory
> flag, we'd not be able to tell whether the error received by the
> publisher was a result of no binding matching the message (a normal
> condition) or the loss a node, along with all the queues and bindings
> associated with it (an exceptional condition).
>

You can use publisher confirms to allow your application code to know 
when a message has successfully been routed or when it has been lost.

See: http://www.rabbitmq.com/extensions.html#confirms

> Any suggestions on how to handle this issue and distinguish between
> these two conditions?
>
> Thanks,
>
> Elias Levy
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss