[rabbitmq-discuss] Persistent queues

Fri Jun 7 15:12:33 BST 2013

Hi guys,

following a discussion on the kafka user mailing list in which Alexis 
Richardson replied, I realized I might have been wrong about what I assumed 
for the messages storage in RabbitMQ. It won't change the decision I made 
to use Kafka but I'd like to set my own mind right in my conception of how 
RabbitMQ stores messages. I want to use Rabbit in other projects as well so 
I want to have the right picture in my mind.

The setup I used was based on what is described in 
http://www.rabbitmq.com/tutorials/tutorial-four-python.html. 

RabbitMQ
- RabbitMQ 3.0.x (not exactly sure what x was)
- 5 brokers (behind HA proxy)
- Messages posted to multiple topics (called it A, B, C) where the topic is 
used as the routing key by the exchange
- Queues and messages are persistent (+ replicated)

Consumers
- Consumer 1 subscribes to Topic A (near real-time) - Queue 1
- Consumer 2 subscribes to Topic B (near real-time)  - Queue 2
- Consumer 3 subscribes to A, B and C (offline, BI) - Queue 3

Published 10M messages (size varying between 500b to 2k)