[rabbitmq-discuss] Transfer rate improvement above 45mb/second?

[Matt]

Thats sounds very unsatisfactory simply because I am not sure it addresses 
the real problem and makes me wonder  how efficient/inefficient the AMQP 
parser really is. If I send large byte arrays what needs to be parsed? My 
understanding is that each message gets parsed to AMQP but not the content 
of the message body. So, I fail to understand how parsing makes a 
difference whether I send a byte array of size 10 bytes versus a message 
that contains a body that holds a 10 megabyte byte array. I do not see why 
the bus limits me to 45mb/second. My binary reader moves about 
200mb/second, reading from a physical storage into memory, including the 
parsing of the byte arrays to C# objects, thus the raw speed should be even 
a lot higher. There must be something else going on that limits RabbitMQ to 
go beyond the 45mb/second. The order of messages is important thus I cannot 
run more channels in parallel. Can anyone please chime in so I understand 
what I am overlooking here? Thanks a lot.


On Tuesday, March 6, 2012 1:36:53 AM UTC+9, Simon MacMullen wrote:
>
> I suspect you're probably limited by the speed of the AMQP parser in 
> this case. Unfortunately the only thing you can really do to make it run 
> faster is turn on HiPE compilation or buy a a faster machine. And HiPE 
> compilation only works on Unix I'm afraid.
>
> If you can afford to lose ordering guarantees you could push the 
> messages through more than one connection / queue, and thus take 
> advantage of more cores.
>
> Cheers, Simon
>
> -- 
> Simon MacMullen
> RabbitMQ, VMware
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120305/85d1909c/attachment.htm>