[rabbitmq-discuss] RabbitMQ perfomance testing & troubles

Thu Apr 24 23:23:12 BST 2008

Ð’ Ð§Ñ‚, 24/04/2008 Ð² 21:16 +0100, Matthias Radestock Ð¿Ð¸ÑˆÐµÑ‚:

Matthias, thanks a lot for your answer !

> 
> > Well, I was configured the local cluster for 8 nodes:
> > 
> > # su rabbitmq -c "/usr/sbin/rabbitmq-multi start_all 8"
> > Starting all nodes...   
> > [...]
> > broker running
> > OK
> 
> Great. Glad it worked without problems.

And I'm still do think, that we are have a same stable Debian without
(white ? / black ?) magic  :-)

Mmm.. a little: usermod -s /bin/sh rabbitmq. 
Yes, a security hole, but I'm just testing.

> 
> > Is any way to improve perfomance and reduce latency in my N producers => M consumers scheme ?
> 
> Did you make the other changes I suggested previously, namely:
> 
> - change the exchange type from "topic" to "direct"

Well, just a second..

> 
> - make sure that all the producer & consumer connections are set up and 
> the queues have been created before starting to send any messages

Yes, all producers & consumers connections are set before start
send/recieve messages (class constructor) and thread sleep 10 sec after
queues creation and before starting to send any messages

I change the exchange type in producer & consumer from "topic" to
"direct" and start test:

$ java -jar RabbitmqFastMessagingProducerTest.jar 1000 > client.log

The server side of 8 local nodes with  10000 mps looks good:

top - 01:32:47 up 20 days,  8:01,  3 users,  load average: 0.84, 1.31,
0.71
Tasks: 152 total,   1 running, 151 sleeping,   0 stopped,   0 zombie
Cpu(s):  7.4%us,  1.3%sy,  0.0%ni, 89.6%id,  0.0%wa,  0.0%hi,  1.6%si,
0.0%st
Mem:   4139508k total,  1610992k used,  2528516k free,   334196k buffers
Swap:  4194296k total,        0k used,  4194296k free,   862924k cached

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
12613 rabbitmq  15   0 47604  22m 1596 S   12  0.6   0:37.40 beam
12931 rabbitmq  15   0 48552  23m 1596 S   11  0.6   0:38.55 beam
14203 rabbitmq  15   0 48552  23m 1596 S   11  0.6   0:38.15 beam
13567 rabbitmq  15   0 47612  23m 1596 S   10  0.6   0:37.39 beam
14521 rabbitmq  15   0 48556  23m 1596 S   10  0.6   0:35.25 beam
13885 rabbitmq  15   0 48560  23m 1596 S    9  0.6   0:37.93 beam
14839 rabbitmq  15   0 48556  22m 1596 S    9  0.6   0:35.80 beam
13249 rabbitmq  15   0 48556  23m 1596 S    8  0.6   0:34.43 beam

The client side is also looks good and very low delay (1-20 ms):

$ tail -f client.log | awk '/CLIENT_CONSUMER_THREAD_ID/{print "DELAY: "
$4-$8 " " $0}'

DELAY: 1 CLIENT_CONSUMER_THREAD_ID: 167 TIME: 1209073100621 MESSAGE:
4469 TIME: 1209073100620
DELAY: 2 CLIENT_CONSUMER_THREAD_ID: 988 TIME: 1209073100622 MESSAGE:
4469 TIME: 1209073100620
DELAY: 2 CLIENT_CONSUMER_THREAD_ID: 333 TIME: 1209073100622 MESSAGE:
4468 TIME: 1209073100620
DELAY: 3 CLIENT_CONSUMER_THREAD_ID: 56 TIME: 1209073100623 MESSAGE: 4468
TIME: 1209073100620
DELAY: 43 CLIENT_CONSUMER_THREAD_ID: 857 TIME: 1209073100623 MESSAGE:
4467 TIME: 1209073100580
DELAY: 3 CLIENT_CONSUMER_THREAD_ID: 892 TIME: 1209073100623 MESSAGE:
4462 TIME: 1209073100620
DELAY: 4 CLIENT_CONSUMER_THREAD_ID: 607 TIME: 1209073100624 MESSAGE:
4465 TIME: 1209073100620
DELAY: 4 CLIENT_CONSUMER_THREAD_ID: 553 TIME: 1209073100624 MESSAGE:
4464 TIME: 1209073100620
DELAY: 5 CLIENT_CONSUMER_THREAD_ID: 272 TIME: 1209073100625 MESSAGE:
4466 TIME: 1209073100620
DELAY: 6 CLIENT_CONSUMER_THREAD_ID: 654 TIME: 1209073100626 MESSAGE:
4463 TIME: 1209073100620

But, then I try to calculate message log I have a some confuse 

$ cat client.log | ggrep -v -E "(START|STOP)" | grep
CLIENT_PRODUCER_THREAD_ID | wc -l
 1000000

$ cat client.log | ggrep -v -E "(START|STOP)" | grep
CLIENT_CONSUMER_THREAD_ID | wc -l
  137983

Logs tells that I send 1000000 messages, but recieve only 137983 . 

Hmm... So, why ? 

I can't loss any messages in real play. The test code is attached.

P.S.
I'm very sorry, I'm realy need some sleep. 

WBR,
Alter

-------------- next part --------------
A non-text attachment was scrubbed...
Name: rabbitmqfastmessagingproducertest.tar.gz
Type: application/x-compressed-tar
Size: 2017 bytes
Desc: not available
Url : http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20080425/dcd6231e/attachment.bin 