[rabbitmq-discuss] RMQ performance between high MHz/low core vs lower MHz/high core servers

[Matthias Radestock]

Chris,

On 20/12/12 17:27, Chris Schmidt wrote:
> RHEL 6.2 doesn't seem to have the latest version of Erlang available
> in the repositories. Does anyone have experience with manually
> building Erlang on that platform with success? We're on R14B04 right
> now. There are a huge number of erlang related rpms that get
> installed I'm not certain if a single Erlang version for CentOS
> would work properly or not.

You may want to give
https://my.vmware.com/web/vmware/details?downloadGroup=VFEL_15B02&productId=267
a try - it's R15B02 (so not quite the latest but close enough) with
"batteries included", i.e. no dependencies.

> I had +K true -smp enable +native +sbtps plus the other RMQ defaults

"+K true" is already part of the standard config. So is "-smp auto",
which enables smp when there is more than one core. "+native" is a
compile-time option and I would not recommend using it. As for +sbtps, I
suggest you leave that out and stick with the default.

As I said in my earlier email, playing with the +swt option may yield
some benefits.

> I also had 15 consumers running against the initial queue that was
> having problems.

Increasing the number of consumers only helps when the consumers, rather
than the queue, are the bottleneck.

> I also tried sending the data into 2 - 40 exchanges (and updated the
> queue to read from those exchanges) in order to see if I could spread
> the load that way.

Exchanges are not represented by processes, so you don't gain anything
by spreading the load over them. Increasing the number of producers can 
increase the message ingestion rate, but only if the messages get routed 
to different queues.

> Does a single process manage each queue?

yes.

> If so I can see that being a potential bottleneck due to the core's
> speed difference.

Exactly.

> Is there an erlang or rabbitmqctl command that I can run against the
> RMQ server to determine if the queues themselves are the bottleneck?

If the queues are growing then the bottleneck is on the outbound side 
(rabbit's AMQP encoding, network, consumers).

If the queues stay short/empty then the bottleneck is on the inbound 
side (producer, network, rabbit's AMQP decoding, routing, the queue). If 
connections get blocked due to flow control then the bottleneck is in 
the latter three. Narrowing it down any further is tricky.

Regards,

Matthias.