[rabbitmq-discuss] Transfer rate improvement above 45mb/second?

[Simon MacMullen]

On 05/03/12 15:33, Matt wrote:
> Hello, I setup a rabbitmq server on my local machine under Windows 7,
> 64 bit, and set up a test instance in C#. I look for the fastest way
> to deliver messages in the same order than they were published onto
> the bus through a direct connection (no fan out needed). I do not need
> guaranteed delivery, all I am after is max speed to deliver the
> messages to one single consumer. Currently I seem to max out at about
> 40,000 messages per second which is impressive but does not solve my
> problem. I need to push around 115,000,000 messages and thus I started
> to chunk the messages into larger byte arrays. Still I dont seem to
> improve much as the max transfer rates maxes out at around 45mb/
> second. Is there any way to improve those numbers? Pushing the
> 115,000,000 messages to the consumer currently takes around 50 seconds
> no matter how I set the chunk size.

I suspect you're probably limited by the speed of the AMQP parser in 
this case. Unfortunately the only thing you can really do to make it run 
faster is turn on HiPE compilation or buy a a faster machine. And HiPE 
compilation only works on Unix I'm afraid.

If you can afford to lose ordering guarantees you could push the 
messages through more than one connection / queue, and thus take 
advantage of more cores.

Cheers, Simon

-- 
Simon MacMullen
RabbitMQ, VMware