[rabbitmq-discuss] What is the currently fastest way to stream messages?

[Emile Joubert]

Hi Matt,

On 12/04/12 07:30, Matt wrote:
> message patterns that I use and where throughputs are not critical. But
> this byte array stream is throughput critical and I would appreciate any
> comments how to tune this. 

Batching the payloads into larger chunks will offer better throughput.
For payloads of only 16 bytes the rest of the message represents a
relatively large overhead.

You might even want to raise the maximum frame size if the chunks exceed
128kb. (See frame_max in http://www.rabbitmq.com/configure.html)
Raising the loopback MTU may also help throughput.

You should use auto-acks instead of relying on consumers to ack messages.
Don't use persistence, transactions or publisher confirms.
Don't use immediate of mandatory publish options.
Use a direct exchange.

If you have a very small number of queues then select a server with
fewer fast CPUs rather than many slow CPUs. Tuning the number of Erlang
schedulers (+S commandline option to erl.exe) may also help.
Make sure you have plenty of RAM to prevent paging messages to disk.

It is not possible to avoid serialisation, unless you make use of the
direct Erlang client. If you must use C# then this is not an option.

-Emile