[rabbitmq-discuss] Scaling with RabbitMQ

Tue Apr 22 18:35:51 BST 2014

Hello,

we are currently reviewing our current architecture, which is mostly based on RabbitMQ and Java.

To keep up with the expected growth of our whole system, I want to discuss if our current approach of using RabbitMQ is in a good shape.

We have several services, each running with at least two instances. Each service registers several queues (between 5 and 25). Each queue has two bindings. 
Every instance has one connection to rabbitmq, for every queue we are opening a new channel. One additional channel is used for publishing messages. 

The services communicate with each other through one topic exchange. We are not using any wild card bindings currently and do not plan to use many of them in the near future. 

We are not aiming for maximal throughput, our goal is maximum durability. Therefore we are using delivery_mode = 2, durable and ha queues. We are currently running a cluster which consists of two nodes (each has 4GB RAM and 4 CPUs, using EXT4).

These are the numbers of our production system:

34 Connections
431 Channels
227 Queues
393 Consumers

I tried to find these information on the rabbitmq website, but was not able to find them.

Here are the questions I currently don’t have an answer to:

1. Topic exchange is the slowest exchange type. Direct exchange would be faster. We would lose the ability to have wildcard bindings. Can someone put a number of how much would we gain when we switch to a direct exchange?

2. Publishing rate depends on how many queues have a binding for the published message. We use java class names as routing keys. (com.foo.bar.xxx.zzz). The messages start to differ from the xxx part. Does the structure of the routing key influences publishing speed? What would be the impact of drastically increasing the number of queues (and bindings)? I am thinking of having thousand queues.

3. Is it wise to run everything over the same exchange? As far as I read on the mailing list, different exchanges on the same cluster do not influence each other - as long as there are enough physical resources.

4. How do we know, that we hit some limit of rabbitmq by using it the wrong way? Is there any way to see and monitor internal rabbitmq metrics? What’s your favorite tool to monitor RabbitMQ?

5. What do you think about having a Channel per Queue? That decision made us able to set the prefetch size and thread count per queue on the client side.

Finally I want to thank you for reading my mail. If this is complete nonsense, please tell me. I am new to the land of rabbits (sorry for this bad joke).

And additionally I want to thank every developer who puts his time into rabbitmq. It is an awesome piece of software.

Thanks
Björn