[rabbitmq-discuss] Persistent messages on durable queue sometimes not delivered to consumers

[Emile Joubert]

Hi,

On 26/10/12 00:50, Tyler Power wrote:
> I am using RabbitMQ Server 2.4

That's quite old. You should consider upgrading.

> I am using the default exchange (""), which sounds adequate for our
> needs right now, but thought I'd point it out incase I'm not using it
> the way it was intended.

Using the default exchange is fine.

> In the server I can see the SubmitDeploymentToQueue() method always
> completes successfully, so it appears messages are /always/ being
> submitted to RabbitMQ, but in the Consume() method of the consumer,
> the consumer.Queue.Dequeue() method /sometimes/ does not return a
> message, so the next line (the console.writeline) is never called etc.
> and it will just sit there blocking.

This is normal behaviour if there is no message. If you think there is
an error then you should independently confirm using the management
plugin or using rabbitmqctl whether there are any messages.

> In this situation where messages appear to be lost, restarting the
> consumers doesn't fix it, the messages are simply never delivered and
> seem to be lost forever.

Messages should never inadvertently get lost.

You can use publisher confirms to make sure messages don't get lost
between the publisher and the broker:
http://www.rabbitmq.com/confirms.html

You can use the mandatory flag to make sure unroutable messages are
detected:
http://www.rabbitmq.com/amqp-0-9-1-reference.html#basic.publish.mandatory

You can declare queues durable and publish messages persistent to guard
against the broker crashing.

You can use dead-letter-exchanges to make sure messages don't get lost
from queues:
http://www.rabbitmq.com/dlx.html

You can use consumer acknowledgements to make sure messages don't get
lost when consumers crash while processing messages:
http://www.rabbitmq.com/amqp-0-9-1-quickref.html#basic.ack

If you have found a way for messages to get lost then please post a
minimal self-contained piece of code that demonstrates the error on the
latest released version.


-Emile