<div>After reading some posts on the list it seems like many problems with performance is nothing new. We are currently reorganizing a cluster consisting of a myriad of jetty instances to a rabbitmq and a message driven system. The current component we are moving is shortly a staging system for a relative slow consumer process.. </div>
<div><br></div><div> Our use-case:.</div><div><br></div><div>1. We have currently a small stream of relatively large messages, that varies in size. (1600 bytes -> 5000 bytes )</div><div>2. The incoming rate varies form 0 -> 58 per/s in our current test queues. </div>
<div>3. We have currently two brokers in a cluster </div><div>4. The queues are set up in mirrored (We can not lose any messages), durable and non-exclusive.</div><div>5. We use Publisher Confirms/Consumer Cancellation in the client implementations in Java</div>
<div>6. We use prefetch-count as a basic QOS parameter. We have tested with: 1, 10 , 15, 25</div><div>7. We ack per (deliveryTag % prefetch-count == 0) to minimize acking.</div><div>8. The client can nack a message but it is seldom happening (every 1 million message maybe)</div>
<div><br></div><div>Problem:</div><div>Over time we see a larger performance decrease in consuming the messages from the broker. The consumer is deadly fast in the start. 1-2 ms per message. Then the retrieval gradually decreases to 25-26 ms per message. The velocity of the speed decrease is varying by the prefetch size. When increasing the prefetch count to the time before it reaches 25ms is exponentially increasing. Up to 12 hours. If we re-initiate the connection it seem to recover and everything seem OK.</div>
<div><br></div><div>Versions: </div><div><br></div><div>Java client: 2.5.1</div><div>Broker: 2.5.1 (ubuntu build)</div><div><br></div><div>Regards</div><div><br></div><div><br></div>-- <br>Tarjei Romtveit<br>Data Management Director<br>
Integrasco A/S<br>Mobile: <a href="tel:%2B47%20990%2020%20074" value="+4799020074" target="_blank">+47 990 20 074</a><br>
Address: Hasseldalen 3, N-4878 Grimstad, Norway<br><a href="http://www.integrasco.com" target="_blank">www.integrasco.com</a> // <a href="http://twitter.com/integrasco" target="_blank">twitter.com/integrasco</a> // <a href="http://facebook.com/integrasco" target="_blank">facebook.com/integrasco</a><br>