<div dir="ltr"><div><div><div><div><div>Thanks for the background Simon.<br><br></div>One statement caught my attention though:<br><br>> At 40 msg/s you are publishing slowly enough that RabbitMQ probably won't start to coalesce these events together.<br>
<br></div>In my original message I didn't lay out all my results - I was trying to keep the question simple. That said, the 4 write/sec pattern held up at higher rates. For instance, on a machine with a faster disk, the same test could do 400 messages/sec, and per tools like sar, were seeing 1600 disk writes/sec.<br>
<br></div>As you can imagine, at those I/O rater we're rapidly coming close to the disk's ability to keep up, thus the question about the writes/message rates.<br><br></div>So with this additional context, any additional insight?<br>
<br></div>Thanks,<br><br>Matt<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Aug 5, 2013 at 2:59 AM, Simon MacMullen <span dir="ltr"><<a href="mailto:simon@rabbitmq.com" target="_blank">simon@rabbitmq.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">That filename is typical of those of a queue index journal. That's the file that records the state of messages in the queue - when a message has been published, when it's been confirmed (transactional publish is internally implemented in terms of confirms), when it's been delivered, and when it's been acknowledged (I assume you are using acks).<br>
<br>
So it is very possible to get four disk writes to the journal per message for publish / confirm / deliver / ack.<br>
<br>
At 40 msg/s you are publishing slowly enough that RabbitMQ probably won't start to coalesce these events together. So as you start to publish faster you should start to see fewer writes per message - eventually you should see many messages per write.<br>
<br>
Therefore I wouldn't try to scale the numbers you're seeing up to predict anything - you're likely to be much better off running some benchmark with MulticastMain or similar.<br>
<br>
Cheers, Simon<div><div class="h5"><br>
<br>
On 03/08/2013 01:08, Matt Pietrek wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">
Just want to run this past the RabbitMQ devs and see if this jibes with<br>
their expectations.<br>
<br>
Our typical cluster runs RabbitMQ 3.02 on Ubuntu 10.04 on two nodes.<br>
All queues are mirrored. All our channels are opened in transactional<br>
mode. (We know, this causes things to go slower - That's OK with us, we<br>
really want to avoid message loss.)<br>
<br>
What I'm seeing is that for each message written to the broker, we see<br>
approximately four disk writes. That is, if the broker is doing 40<br>
messages/sec, we see ~160 disk writes. We're getting these number from<br>
<br>
Is this about what should be expected when running this way?<br>
<br>
If it helps, I dug in a bit more to see what the files were. The vast<br>
majority of the activity seems to be to files like this:<br>
<br>
msg_store_persistent/743.rdq<br>
queues/<u></u>E10F54B1OGM7M4LTRX5Z3L4K0/<u></u>journal.jif<br>
<br>
Any insight would be very valuable for our planning processes.<br>
<br>
Thanks,<br>
<br>
Matt<br>
<br>
<br></div></div>
______________________________<u></u>_________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com" target="_blank">rabbitmq-discuss@lists.<u></u>rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/<u></u>cgi-bin/mailman/listinfo/<u></u>rabbitmq-discuss</a><br>
<br>
</blockquote>
<br>
</blockquote></div><br></div>