<div dir="ltr">Hi there,<div><br></div><div style>I am doing a performance study of RabbitMQ-3.1.1 and this is my first time to do such a performance study with any messaging broker :))-</div><div style><br></div><div style>
1) I have thoroughly gone through rabbitMQ in action' and learnt important concepts. </div><div style><br></div><div style>2) Tried single node broker to get a feel of how it is working and then set up a four node cluster (with two disk and two RAM type of node). Also configured HAproxy TCP Load balancer so that I can just provide single port to connect to the Cluster.</div>
<div style><br></div><div style>3) I am simulating producer and consumers through Python scripts ( using Python-pika library methods to connect to server , publish subscribe etc.)</div><div style><br></div><div style>4) My scripts are working fine but where I am stuck is no matter what I do my throughout is always 300 msg/sec. </div>
<div style><br></div><div style><div>5) I have defines durable exchanges and queues</div><div><br></div><div style>My final requirement is to run atleast 10 to 15 producer and 60 to 70 consumer simultaneously and I want to start with linear increase in number of producer and consumer so that I can make conclusions about throughout, fault handling, processor utilization etc. etc but I am seriously stuck now after trying to start in initial steps only. This group's help would be really appreciated. </div>
<div><br></div></div><div style>I have started with following different scenarios but no matter what I do my throughput is more or less remaining same (300 msg/sec) except for Scenario-1</div><div style><br></div><div style>
Scenario-1</div><div style>-1 producer and No consumer and no queue binded to exchange</div><div style>-Producer is running in infinite loop and publishing to one fanout exchange</div><div style>- publisher/confirm disabled<br>
</div><div style>-Publisher rate - 6200 msg/sec ( checked through web management plugin)</div><div style><br></div><div style>Tried scenario-1 with also fanout type of exchange and it's the same publish rate</div><div style>
I know that Scenario-1 is not really useful, since there are no queues and ultimately messages will be dropped but as a part of debugging process I tried this and I see above mentioned results.</div><div style><br></div><div style>
<div>Scenario-2</div><div>-1 producer and 1 consumer </div><div>-Producer is running in infinite loop and publishing to one direct exchange</div><div>-A consumer has it's own dedicated queue and listening to above exchange </div>
<div>- publisher/confirm and consumer ack are disabled</div><div style>Throughput - 300 msg/sec ( which is basically publish rate = 300 msg/sec and deliver rate - 300 msg/sec)</div><div style><br></div><div style>Tried Scenario-2 also with fanout type of exchange and enabling publisher confirm and consumer ack</div>
<div style>Still the same throughput as 300 msg/sec</div></div><div style><br></div><div style><div>Scenario-3</div><div>-1 producer and 4 consumer </div><div>-Producer is running in infinite loop and publishing to four direct exchange</div>
<div>-A consumer has it's own dedicated queue and listening to respective exchange </div><div>- publisher/confirm and consumer ack are disabled</div><div>Throughput - 300 msg/sec ( which is basically publish rate = 300 msg/sec and deliver rate - 300 msg/sec)</div>
<div><br></div><div><div>Tried Scenario-3 also with fanout type of exchange and enabling publisher confirm and consumer ack</div><div>Still the same throughput as 300 msg/sec</div></div><div><br></div><div style>Tried configuring prefetch_count parameters also to 100 but it still gives me same throughput of 300 msg/sec.</div>
<div style>I am honestly going crazy with this.</div><div style><br></div></div><div style>After seeing this behavior, I am seriously suspecting that there is some serious limitation with my simulated producers and consumers.</div>
<div style>Has anyone else has tried Python-pika client and any idea on throughput with this version of rabbit ? </div><div style>Did anyone have rough idea about throughout with rabbitMQ-3.1.1 ?</div><div style><br></div>
<div style>I can also share my python scripts if required but I would really appreciate some light on this situation</div><div style>Also what points to take care, in order to improve throughput ?</div></div>