[rabbitmq-discuss] Will this work?

[Tony Garnock-Jones]

Hi Dinabandhu,

I think you're running into TCP_NODELAY issues. Try setting your
client's socket to nodelay mode, which disables Nagling:

  int one = 1;
  setsockopt(sockfd, IPPROTO_TCP, TCP_NODELAY, &one, sizeof(one));

This should fix the high observed latency in cases (a) and (b).

Regards,
  Tony



Dinabandhu Mitra wrote:
> Hi Alexis,
> 
> Any further advice on our earlier discussions?
> 
> 
> I am currently trying out using c API. I was trying to measure communication latency introduced by RabbitMQ. Following are some of the observations -
> 
> 1. When the test application starts, the latency for first few messages is very high. The number of messages with high latency depends on the throughput the application tries to generate. E.g -
> 	a) For 1/2 message per sec latency continues to be very high (over 200 millisecond) throughout.
> 	b) For 3/4 message per sec it seems to change between high (200) and low (0.200) on every alternate message.
> 	c) For 5 messages per sec it stabilizes to a low value (0.200 - 0.300) after first 5/6 messages.
> 	d) For 10 messages per sec it stabilizes to a low value (0.200 - 0.300) after first 5/6 messages.    
> 	e) For 50 messages per sec it stabilizes to a low value (0.200 - 0.300) after first 15/16 messages.
> 	f) For 100 messages per sec it stabilizes to a low value (0.200 - 0.300) after first 50/60 messages.
> 	g) For 500 messages per sec it stabilizes to a low value (0.300 - 0.500) after first 370/380 messages.
> 	h) For 1000 messages per sec it comes down to a low value (0.500 - 0.700) after first 500 messages. But I have seen intermittent bursts of a sequence of high latency (going over 20 millisecond) messages.
> 	i) Initial latencies increase with throughput. E.g at 1000 msg/sec it is as high as 380/390 millisecond for some messages.
> 
> 2. Both publisher and subscriber are in the same server as the broker. In fact the publisher and the subscriber are two threads in the same process.
> 
> 3. The server is a idle SUN Netra Dual CPU(Quad Core) Xeon class machine with 16G RAM running RHEL 4.7 64 bit.
> 
> 4. The platform is on the factory configuration; i.e. I have just installed the RPMs and started the RabbitMQ server without any other configuration.
> 
> 5. I am using approx 1KB sized messages for this.
> 
> Is there any configuration that I can do to have better (less variance) latency? Also, I am using 1 producer and 1 consumer (both trivial). Shall I get better results by using multiple producers/consumers?
> 
> Regards,
> Dinabandhu
> 
>> -----Original Message-----
>> From: Alexis Richardson [mailto:alexis.richardson at gmail.com]
>> Sent: Tuesday, January 05, 2010 10:33 PM
>> To: Dinabandhu Mitra
>> Cc: rabbitmq info
>> Subject: Re: [rabbitmq-discuss] Will this work?
>>
>> Dinabandhu
>>
>> Answers / comments below also.  Let me know if you want to cc this to
>> rabbitmq-discuss.  FYI - info at rabbitmq.com is our private (internal
>> team) list.
>>
>> alexis
>>
>>
>>
>> On Tue, Jan 5, 2010 at 3:16 PM, Dinabandhu Mitra
>> <Dinabandhu.Mitra at tecnotree.com> wrote:
>>> The scale of the application depends on the implementation. The
>> messaging load
>>> can vary from few hundred per second to over 30,000/40,000 per
>> second. Of course
>>> we will add additional equipment depending on the load but the
>> application should
>>> be able to use additional equipment effectively.
>> OK.  That should be fine.  What's the peak ingress per node?
>>
>> I.e. what is the max rate that a single publisher might produce in
>> messages per second?
>>
>> Also - are the messages 'small' (< 500 bytes) or 'large' (>64Kb)?  Is
>> this size consistent or volatile?
>>
>>
>>
>>
>>>> OK.  You will be able to do this with direct exchanges too.  They
>> are
>>>> faster (in lookup time) than our current topic implementation.  It
>>>> does not look like you need topic exchanges because you do not
>> appear
>>>> to be routing messages using wildcards, eg. you do not need bindings
>>>> of the form "PE.*".  Is that right?
>>> Yes. I was reading the AMQP specification concepts section few days
>> back
>>> and realized that my understanding of direct exchanges was wrong.
>> Direct
>>> exchange should be the right option for us.
>> Good.
>>
>>
>>
>>
>>>>>   a) When a application engine (LE/PE) boots and declares it's
>> queue
>>>> and
>>>>> binding using a connection to the local broker, the queue and
>>>> bindings
>>>>> becomes visible to all broker instances running in different
>> servers.
>>>> Is
>>>>> this correct?
>>>> Correct, provided that the other brokers are in the same cluster (as
>>>> in your diagram) and vhost (ie. namespace).
>>> It should be sufficient. As of now we are looking at a single cluster
>> and
>>> single vhost.
>> OK, cool.
>>
>>
>>
>>>>>   f) Anyone trying send a message that is immediately non-routable
>>>>> (possibly mandatory+immediate flag with auto-ack) gets a error and
>>>> broker
>>>>> drops the message. Is this correct?
>>>> I can't remember exactly how this works.  Basically the answer is
>> YES.
>>>>  I cannot recall under what conditions the broker drops the message
>>>> silently, and under what conditions it lets the publisher know that
>> an
>>>> error occurred.
>>> Well, this is not a very hard requirement. The LE instances are timer
>> controlled,
>>> in the sense that if a LE does not receive a response from any of the
>> PEs within
>>> certain time limit then the LE assumes that there is a problem is
>> handling the
>>> request and sends a failure response to the client that originated
>> the transaction.
>>> However, it is way better if the rejection can be done without
>> waiting for the
>>> timer to expire in case none of the relevant PE engines are live.
>> This is how the
>>> application behaves currently.
>> OK.  We can return to this point later.
>>
>> alexis
>>
>>
>>
>>>> alexis
>>>>
>>>>
>>>>
>>>>> Regards,
>>>>> Dinabandhu
>>>>> --
>>>>> View this message in context: http://old.nabble.com/Will-this-
>> work--
>>>> tp26875409p26875409.html
>>>>> Sent from the RabbitMQ mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> rabbitmq-discuss mailing list
>>>>> rabbitmq-discuss at lists.rabbitmq.com
>>>>> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-
>> discuss
>