[rabbitmq-discuss] persistent messages stored where?

[Matthew Sackman]

On Wed, Oct 20, 2010 at 02:05:15PM -0700, Steve Haneman wrote:
> I'm looking to find out how much disk space 1,000,000 10k messages take up on disk.  I pushed the messages and then shut down the server.  If I do a "du -sh" in the /var/lib/rabbitmq/mnesia/ directory, is that it?

Yes. However, I can just do the sums for the best case. Worst case will
be double (due to dead space on disk that's pending GC etc).

Per message, we have about a 200 byte overhead for the erlang data
structure that holds the message. We then have a further 25 bytes
overhead on disk (the on disk format is self-describing).

But then we have further cost per queue (a message can go to several
queues). Per queue, assuming the queue journal is flushed, we'll have 18
bytes per message, assuming the message isn't delivered.

So, if you have 1e6 * 10kB msgs going to one queue, I reckon you'll be
looking at about 1e6 * (10240 + 250) bytes (assuming 1kB == 1024B),
which I make to be about 9.77GB.

But as I say, that's best case. It can be anywhere between that and
double that, depending on what's going on in the system.

Thus for 10kB msgs, we're looking at about a 2.5% inflation on disk.

Does that match what you measure? (You'll find the numbers are a bit
bigger because of other stuff you'll be measuring, including mnesia
databases etc. But the trend should be about the same)

Matthew