[rabbitmq-discuss] Crash - Total persistent queue content loss - 2.5.1

[Emile Joubert]

Hi Joseph,

On 02/04/12 21:15, Joseph Marlin wrote:
> I know I'm working on a version that's pretty old by now. I looked at
> the log contents and was completely mystified. I'd like to understand
> what went wrong here.  
> 
> The Story: RabbitMQ crashes and when I bring it back up, all queue
> contents from all my persistent queues are no longer there.
> 
> -If it was my fault, what can I do to fix it?
> -If it is Rabbit's fault, has the issue been fixed?
> 
> Attached are the 'rabbitmqctl report' and the appropriate log from
> /var/log/rabbitmq/

The log contains {error,enospc}. This means the disk ran out of space.
The only way to prevent this condition is to monitor the disk free space
and stop publishers or provision more space before space runs out.

If you restart the broker while out of disk space then recovery of
messages will fail. The broker makes a backup of the data directory
first, but this would have failed.

The next version of RabbitMQ (after v2.8.1) will deal with this
condition more gracefully by throttling producers when disk space runs
low as well as when RAM runs low.


-Emile