[rabbitmq-discuss] When are persistent messages flushed to disk?

Mon Sep 5 08:19:22 BST 2011

Hi Matthew,

Thanks for your detailed reply.  

On 02 September 2011 16:26, Matthew Sackman wrote:
> All of which is essentially why we introduced publisher confirms
(well,
> that, plus the fact that transactions are almost never what you
actually
> want - think of publisher confirms as being there to solve the fact
you
> can't pipeline transactions). Thus I'd recommend you read
> http://www.rabbitmq.com/extensions.html#confirms and use them, and
> architect your clients such that they understand that until they
receive
> confirmation from the broker that the broker has taken responsibility
> for the message (which in the case of a persistent msg sent to durable
> queues (normally) means the msg has been fsync'd to disk), the
> publishing client is still responsible for the message and should be
> prepared to republish the message should its connection to the broker
> fail before it receives the confirm.
I did look at publisher confirms but my initial use case requires at
most once delivery, where a cluster of ESBs are reading the queue
(making only-once from at-least-once hard).  I expect future use cases
require these features as we move more message types through RabbitMQ in
the future.

I experimented with publisher confirms and noticed that messages seemed
more likely to be persisted, but seemed to trickle into the server.
That is 5 minutes after my test finished messages were still arriving to
my queue.  I have since experimented with message size, my original
messages were very small and think that I need to modify this test to
understand these affects properly.

I hope to experiment more with this and ha once our first implementation
is released.

Thanks,
Iain.