[rabbitmq-discuss] Mirrored queue performance factors
simon at rabbitmq.com
Wed May 22 11:32:49 BST 2013
On 21/05/13 20:12, Paul Bowsher wrote:
> We operate two dual-node brokers, each broker having quite different
> queues and workloads. Each box has 24 cores (H/T) worth of Xeon E5645 @
> 2.4GHz with 48GB RAM, connected by Gigabit LAN with ~150μs latency,
> running RHEL 5.6, RabbitMQ 3.1, Erlang R16B with HiPE off. We've tried
> with HiPE on but it made no noticeable performance impact, and was very
> We appear to have hit a ceiling for our message rates of between 1,000/s
> and 1,400/s both in and out. This is broker-wide, not per-queue. Adding
> more consumers doesn't improve throughput overall, just gives that
> particular queue a bigger slice of this apparent "pool" of resource.
> Every queue is mirrored across the two nodes that make up the broker.
> Our publishers and consumers connect equally to both nodes in a
> persistant way. We notice an ADSL-like asymmetry in the rates too; if we
> manage to publish a high rate of messages the deliver rate drops to high
> double digits. Testing with an un-mirrored queue has much higher
> throughput, as expected. Queues and Exchanges are durable, messages are
> not persistent.
Just to check - how much faster is your unmirrored throughput?
But those performance figures sound wrong, I can go much faster than
that on a less powerful machine.
How large are messages? Are you using acks / confirms?
Our Java client ships with a performance measuring tool. It would be
good to test with that so that we are running the same tests. Can you
grab the Java client tarball, unpack it and run:
$ ./runjava.sh com.rabbitmq.examples.MulticastMain -a -h
and tell us what message rates you get? That command line should use
tiny unpersistent / unacked / unconfimed messages, so it should give us
a baseline to work with.
If not all queues are configured to be mirrored you might need to add
"-u <name of a queue to create that will be mirrored>".
> Is there any Erlang/OTP tuning we can do? delegate_count is the default
> 16, could someone explain what this does in a bit more detail please?
delegate_count is not itself used by mirroring; it (approximately)
provides an upper bound on the parallelism available for cross-cluster
channel -> queue message routing. It's pretty rare to want to change it.
More information about the rabbitmq-discuss