[rabbitmq-discuss] Throughput observation with RabbitMQ-3.1.3 and Erlang R16B01 - Single Node and Cluster Node

[Priyanki Vashi]

Hi ,

Simon - Thanks for comparitive values and some more insight into my
questions. The discussion is going in a direction to understand

1) how good or bad my pika clients efficiciency is and then how exactly my
system resources are used by both client and rabbit server.

So far I can only see that if I send 100 byes of payload, maximum
throughput I get is around 30,000 msg/sec and maximum bandwidth usage is
160-170 mbps. Trying to understand what is happening in system with RAM,
memory etc. etc....

As you mentioned I see comapritively better bandwidth usage with 500 bytes
and 1000 bytes and they are in range of 350 Mbps and 580 Mpbs. Which I
don't know is appropriate or not.

Do you know if one can saturate 8Gbps link with Rabbit ?
I mean does the other fastest client could really show bandwidth usage like
2, 3 or 5 Gbps, given there are 20 cores of CPU and 10GB of RAM exclusively
given to rabbit or even higher RAM.

My goal is to understand and hopefully to pin point what could be the
bottleneck in the environment I am currently running.



For the earlier question of Micheal on number of cores, here is some more
info.

I have 10 physical cores and they are hyperthreaded so that's how I have 20
cores. I have such two machines. So then,

Machine-1 with 20 cores running exclusively my Producer & Receiver based on
pika
Machine-2, again with 20 cores running exclusively RABBITMQ server

I us KVM hypervisor and create virtual machine and through this control
both cores, RAM and DISK resources for both client and rabbit server.

I had noticed one more thing, may be worth mentioning. When I run

dstat --top-latency ( which I understand will show me a process with
highest latency) it most of the time display 'rcu_sched' process. After
some googling, I learnt that it's Read,Copy update scheduling available
with linux kernel > 2.6.x (I guess). This is a kind of improvement, which
allow to read memory while it's being updated. I don't know much of it.

Thank you again !

Best Regards,
Priyanki.



On Fri, Aug 16, 2013 at 6:10 PM, Simon MacMullen <simon at rabbitmq.com> wrote:

> I'm not completely sure where this discussion is going :-) but I'll pitch
> in anyway...
>
>
> On 16/08/13 10:33, Priyanki Vashi wrote:
>
>> In my wireshark traces, I see following frames for every message I send.
>> I have publisher confirm and consumer ack enabled with prefetch_count
>> not set.
>>
>
> Of course publisher confirm is optional. Also if the broker is busy enough
> it can start to issue acks with multiple=true. So you don't have to pay the
> cost of an ack for every message published.
>
>
>  content header = 88 bytes ( does not show any direct info so not sure)
>> basic.publish frame = 85 bytes ( shows details of queue, exchange,
>> routing key etc)
>> content body = 137 bytes ( in this my payload is 100 bytes of string +
>> 37 bytes might be checksum etc.)
>>
>
> I don't see this much. Publishing with:
>
> $ runjava.sh com.rabbitmq.examples.**MulticastMain -C 1 -y0 -s 100
>
> (i.e. publish 1 message of 100 bytes, don't consume anything, then stop)
>
> I get a 255 byte (Ethernet) frame sent for the publish:
>
> 14 bytes Ethernet II header
> 20 bytes IPv4 header
> 32 bytes TCP header
> 59 bytes AMQP basic.publish method (variable size, including exchange name
> and routing key)
> 22 bytes content header (i.e. minimal basic.properties)
> 108 bytes content (100 bytes payload)
>
> So I suspect there's something different with your client and / or test
> app. Note that things like whether you set anything in basic.properties can
> make quite a difference here, the AMQP encoding is designed to be shorter
> in the common case.
>
> For most people the framing costs per message are not excessive. I don't
> think there is any particular waste there. If the costs are excessive for
> you, then MQTT would offer lower (but still present) per-message framing
> costs.
>
> But you can't expect to send tiny messages and fill your network bandwidth
> anyway. Each message's framing has a fixed cost to decode, and then each
> message has a fixed cost for routing logic as well. (Even with fanout
> exchanges we still have to look up bindings after all.)
>
> tl;dr: you'll notice much higher bandwidth utilisation if you start
> sending longer messages.
>
> Cheers, Simon
>
> --
> Simon MacMullen
> RabbitMQ, Pivotal
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130816/a9dc2117/attachment.htm>