[rabbitmq-discuss] RabbitMQ 1.7 and persistence limit?

Tue Oct 13 13:58:21 BST 2009

Hi there,

I am an SAP PI / XI consultant by trade so I guess my "message
acknowledgement" from a "broker" can also been seen as your SYNC whereby the
sending client gets an "ok" back from RabbitMQ.....I understand now
...thanks

And yes, you are 100% correct on the "buffers" along the way...

But most importantly is that you have cleared my thoughts that Mnesia was
used for persistence....great....and that it is actually the Erlang standard
DETS stuff that is the "problem"....I hope they make it 64 bit one day

You have helped a lot and got my brain around a few things ;-)

Chat later

Lynton

-----Original Message-----
From: rabbitmq-discuss-bounces at lists.rabbitmq.com
[mailto:rabbitmq-discuss-bounces at lists.rabbitmq.com] On Behalf Of Matthew
Sackman
Sent: 13 October 2009 02:34 PM
To: 'rabbitmq'
Subject: Re: [rabbitmq-discuss] RabbitMQ 1.7 and persistence limit?

Lynton,

On Tue, Oct 13, 2009 at 02:15:46PM +0200, Lynton Grice wrote:
> The SYNC transactions are not my concern, I do not really mind if those
> messages get lost, it is the ASYNC ones that I want 100% guarantee they
will
> never get lost.

I'm completely confused by what you mean by this. That is just not
possible. If you send an async message, eg basic.publish, there are
probably a good few dozen buffers in your local computer, the language
you're writing in, the OS network stack, the network card itself, and so
forth, all in which the message can be stored and lost in the event of
power failure. Not to mention buffers in network hubs/switches/routers,
and then in Rabbit itself, all of which can hold the message
temporarily, after it's been "sent" and before it's been written to
disk.

Unless you get some sort of reply from Rabbit, which says that "yes,
I've received the message and it's been written to disk", there is *no*
guarantee that any message you send has even been received by Rabbit,
let alone safely stored.

> You said " With the persister in 1.7, messages are kept in RAM but are
also
> written out to disk." - are the messages written to disk by default as
well?
> As in the choice between "ram_copies", "disk_copies" and
"disc_only_copies"?
> Or are the messages simple written anyway in the 1.7 persister?

By definition, persistent messages are written to disk. They also happen
to be stored in RAM in the persister in 1.7 (and before), for a variety
of reasons. This has *nothing* to do with the various modes of operation
of mnesia, as the persister does not use mnesia at all.

> Using transactions sounds like the answer for me for now, thanks.

Right, and thus this is effectively synchronous messaging as you get the
reply back from Rabbit - the commit-ok - when you commit on the channel.

> Last question: Do you ever forsee a day when there is no 2GB limit to
DETS?

It's possible that the Erlang/OTP folks will get round to rewriting it
to make it 64-bit safe, I don't know. Certainly it's not something we're
looking at doing.

Matthew

_______________________________________________
rabbitmq-discuss mailing list
rabbitmq-discuss at lists.rabbitmq.com
http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss