[rabbitmq-discuss] Architecture Question
crashcourse at gmail.com
Tue May 28 05:49:16 BST 2013
I've been running some tests locally, trying to determine whether RabbitMQ
will meet our performance requirements.
I'm running the broker (3.1.1) on a Dual Xeon E5 (12 cores total) with 256
MB and Windows Server 2012, on Erlang R16B (64bit). One of the primary
workflows that we want to enable is using a cluster is to have several
workers behind the cluster taking messages from multiple clients.
With a single producer -> direct exchange -> single consumer on a single
broker, I can get about 38000 messages / second. (I'm not sure where the
performance limit is here, since I'm not queuing any messages, the consumer
is the equivalent of /dev/null, and the CPU on the broker doesn't go above
To test it, I ran two brokers on the same server, and tried setting up a
couple of different exchange/queue layouts:
- Producers publishing to a single consistent hash exchange, with each
consumer connecting to a private queue bound to the exchange
- Producers publishing to a single exchange / default exchange, with
each consumer connecting to a single queue off the exchange
In the first scenario above, if I use 24 consumers (and hence 24 queues)
distributed equally across both brokers, I can barely muster 52000 messages
/ second (and going up to 30% CPU on each Erlang broker)
In the second scenario, I stop at under 10000 messages / second.
This leads me to believe that the former is the better design, but what
kind of architecture would give a 4 fold or 10 fold increase in throughput?
Would we have to hash prior to RabbitMQ? An x-consistent-hash in front of
multiple x-consistent-hash exchanges?
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rabbitmq-discuss