[rabbitmq-discuss] disk space not released after persistent messages are consumed

[Jianing Hu]

Thanks for the explanation. I ran a longer test and that is indeed the
behavior. The old file got deleted eventually.

Thanks,
- Jianing

On Thu, Feb 10, 2011 at 5:07 PM, Alexandru Scvorţov
<alexandru at rabbitmq.com> wrote:
> Hi,
>
>> After running this code I end up with a 3MB file in the
>> msg_store_persistent directory, even though rabbitmqctl list_queues
>> shows 0 messages in the queue. After restarting RabbitMQ server that
>> file grew by another 800KB. Any advice is greatly appreciated.
>
> That's more or less expected.
>
> The broker tries to write the messages to disk in files stored in that
> directory.  The persister always has at least one file open and it will
> append to it any new messages.  Since it only *appends* to the files,
> even if the messages get delivered to consumers, they'll still be on
> disk.  Normally, the persister will delete files that only contain stale
> messages, but it won't do this to the file it's currently writing to.
> When the current file exceeds msg_store_file_size_limit, it will start
> on a new file.  At that point, if the old file doesn't have any more
> messages, it will delete it.  The size limit is about 16 Mb by default,
> so it's unsurprising that you have a 3 Mb file lying around.
>
> You can set msg_store_file_size_limit in rabbitmq.config.
>
> Does this answer your question?
>
> Cheers,
> Alex
>
> On Mon, Feb 07, 2011 at 05:27:07PM -0800, Jianing Hu wrote:
>> Hi,
>>
>> RabbitMQ newbie here so apologies in advance if what I'm doing is
>> obviously wrong.
>>
>> I'm experimenting with RabbigMQ 2.1.1 and it seems my persistent
>> message storage file keeps growing despite of messages being consumed.
>> Here's the code I used for testing. I'm using perl with Net::RabbitMQ
>>
>>     use Net::RabbitMQ;
>>     my $mq = Net::RabbitMQ->new();
>>     $mq->connect("localhost", {
>>         user => 'guest',
>>         password => 'guest'
>>     });
>>     $mq->channel_open(1);
>>     $mq->queue_declare(1, 'test', {
>>         passive => 0,
>>         durable => 1,
>>         exclusive => 0,
>>         auto_delete => 0
>>     });
>>
>>     my $data = 'foobar' x 200;
>>     for (1..100000) {
>>         $mq->publish(1, 'test', $data, {}, { delivery_mode => 2});
>>     }
>>
>>     $mq->consume(1, 'test', {
>>         no_ack => 0
>>     });
>>     for (1..100000) {
>>         my $msg = $mq->recv();
>>         $mq->ack(1, $msg->{delivery_tag});
>>     }
>>
>> After running this code I end up with a 3MB file in the
>> msg_store_persistent directory, even though rabbitmqctl list_queues
>> shows 0 messages in the queue. After restarting RabbitMQ server that
>> file grew by another 800KB. Any advice is greatly appreciated.
>>
>> Thanks,
>> - Jianing
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>