[rabbitmq-discuss] Persister log size calculation and rolling persister log frequency?

[John Apps]

What follows is probably answered somewhere, so forgive me for asking if it
is...

RabbitMQ 1.7.2
Windows 7 X64

Observing a test using the MulticastMain client, one producer and one
receiver, with the following parameters:

   - MulticastMain -y 0 -s 128 -e e -t fanout -r 1100 -f persistent
   - MulticastMain -x 0 -e e -t fanout -y 1 -n 500 -q 1000

I see that the size of the log varies tremendously, as does the frequency
with which it is 'rolled'.
The size is presumably determined by the number of messages pending, i.e.,
not yet acknowledged?

QueuesOverviewMessagesTransactionsParametersVhostNameConsumersMemoryCountMsg
readyMsg unackCountAcks uncommitedDurableAuto deleteArguments/
amq.gen-EIK8rQQfEX9IBiqLwF6h7Q==10MB22702271227falsetrue[]The log file at
the above moment in time is 128KB. (I paused the producer long enough for
the receiver to catch up.)

QueuesOverviewMessagesTransactionsParametersVhostNameConsumersMemoryCountMsg
readyMsg unackCountAcks uncommitedDurableAuto deleteArguments/
amq.gen-EIK8rQQfEX9IBiqLwF6h7Q==1105MB2493432483431000158falsetrue[]

The log file at the above moment is some 111,529KB, the rolled over log file
about the same size.

What I am wondering about is

   1. when does the roll-over take place?
   2. what role does the dump_log_write_threshold parameter play here? (It
   is set to 1,000 in these tests)
   3. why is the previous version deleted before the current one is rolled
   over? (GFS would appear to be the algorithm which should be used in order
   not to lose anything)
   4. the new version of the log file is created with a very small initial
   allocation? Is there a way to influence this size?  (yes, I know, modern
   file systems should be able to happily expand very fast, and I have asked
   this question before. However, the time taken to 'rename' the current to
   previous and create a new log causes quite significant waves on the client
   side. These are noticeable even to the human eye.)
   5. The severe delay is only experienced by the receiver. Here are a few
   samples:




   - recving rate: 507 msg/s
      - recving rate: 506 msg/s
      - recving rate: 475 msg/s
      - recving rate: 377 msg/s
      - *recving rate: 5 msg/s*
      - recving rate: 591 msg/s
      - recving rate: 490 msg/s













---
John Apps
(49) 171 869 1813
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20100401/46de1efc/attachment.htm