[rabbitmq-discuss] persistence

Thu May 22 13:49:42 BST 2008

Lars,

Matthias Radestock wrote:
> Lars Bachmann wrote:
>> In our company we comparing different messaging solutions and rabbitmq 
>> seemed to be a good one, but the problems with the presistent messages 
>> are unfortunatelly a big minus.
> 
> The problem actually isn't with persistent messaging, but with sending 
> lots of messages at a rate greater than the broker can process, and 
> building up backlogs of 10th of thousands of messages in doing do. 
> Persistence increases the workload and thus lowers the processing rate, 
> which makes the problem more noticeable when persistence is enabled.
> 
> The bug is not causing any difficulties in practice unless *all* of the 
> following are true of your clients:
> - they publish messages to the broker at a rate that significantly 
> exceeds the brokers processing capacity
> - they sustain such a rate for a significant period of time (i.e. not 
> just a few second blip)
> - they disconnect shortly after sending all these messages

I forgot to mention that this is actually a grey area in the AMQP spec. 
What RabbitMQ is doing is perfectly compliant. We have initiated a 
discussion in the the AMQP working group to get the spec clarified. 
Independently of that, as I mentioned, we will fix the the problem in 
the next release.

I should also point out that AMQP's basic.publish is fundamentally 
asynchronous and thus limited in the guarantees it provides. You get 
stronger guarantees when wrapping your publish requests in transactions 
(see channel.tx{Select,Commit,Rollback}() in the Java API). The server 
guarantees that persistent messages have been written to disk by the 
time it returns tx.commit-ok, though obviously this carries a 
performance penalty.

Matthias.