[rabbitmq-discuss] Question on RabbitMQ and file I/O characteristics

[Simon MacMullen]

That filename is typical of those of a queue index journal. That's the 
file that records the state of messages in the queue - when a message 
has been published, when it's been confirmed (transactional publish is 
internally implemented in terms of confirms), when it's been delivered, 
and when it's been acknowledged (I assume you are using acks).

So it is very possible to get four disk writes to the journal per 
message for publish / confirm / deliver / ack.

At 40 msg/s you are publishing slowly enough that RabbitMQ probably 
won't start to coalesce these events together. So as you start to 
publish faster you should start to see fewer writes per message - 
eventually you should see many messages per write.

Therefore I wouldn't try to scale the numbers you're seeing up to 
predict anything - you're likely to be much better off running some 
benchmark with MulticastMain or similar.

Cheers, Simon

On 03/08/2013 01:08, Matt Pietrek wrote:
> Just want to run this past the RabbitMQ devs and see if this jibes with
> their expectations.
>
> Our typical cluster runs RabbitMQ 3.02 on Ubuntu 10.04 on two nodes.
> All queues are mirrored. All our channels are opened in transactional
> mode. (We know, this causes things to go slower - That's OK with us, we
> really want to avoid message loss.)
>
> What I'm seeing is that for each message written to the broker, we see
> approximately four disk writes. That is, if the broker is doing 40
> messages/sec, we see ~160 disk writes. We're getting these number from
>
> Is this about what should be expected when running this way?
>
> If it helps, I dug in a bit more to see what the files were. The vast
> majority of the activity seems to be to files like this:
>
> msg_store_persistent/743.rdq
> queues/E10F54B1OGM7M4LTRX5Z3L4K0/journal.jif
>
> Any insight would be very valuable for our planning processes.
>
> Thanks,
>
> Matt
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>