[rabbitmq-discuss] Persistent queues

Fri Jun 7 19:52:10 BST 2013

Marc,

just a few observations on your setup, to supplement the answers to your 
specific questions that Simon gave...

On 07/06/13 15:12, Marc Labbe wrote:
> RabbitMQ
> - RabbitMQ 3.0.x (not exactly sure what x was)
> - 5 brokers (behind HA proxy)
> - Messages posted to multiple topics (called it A, B, C) where the topic
> is used as the routing key by the exchange
> - Queues and messages are persistent (+ replicated)
>
> Consumers
> - Consumer 1 subscribes to Topic A (near real-time) - Queue 1
> - Consumer 2 subscribes to Topic B (near real-time)  - Queue 2
> - Consumer 3 subscribes to A, B and C (offline, BI) - Queue 3
>
> Published 10M messages (size varying between 500b to 2k)

So is that a cluster of five rabbit nodes, with ha-mode='all' set for 
all queues?

If so, note...

- RabbitMQ 3.1 has some significant bug fixes over 3.0, including to a 
memory leak in HA queues, and performance improvements (to HA queues and 
beyond); so you should really upgrade

- with five nodes, you probably ought to configure the ha-mode for the 
queues to be 'exactly' or 'nodes', to limit mirroring to a sub-set; 
unless you really want five replicas of your data.

- if there are really only three queues then you'll want to make sure 
their masters (and slaves) are not clumped together on just a few nodes. 
The 'nodes' policy is useful for that.

- with five nodes, you probably ought to configure all but two or three 
as ram nodes instead of disk nodes, to cut down on the number of 
synchronous disk operations performed for queue/exchange/binding 
creation/deletion. I guess this won't matter much in your case though, 
since it looks like the set of exchanges, queues and bindings is pretty 
static.

Regards,

Matthias.