[rabbitmq-discuss] Publisher throttled way too early

Fri Apr 13 21:23:00 BST 2012

Hi all!  

Apologies if this is duplicated, I can't see my message on Google Groups after 2 hours.  

I upgraded to 2.8.1 after my publishers were throttled way too late: I would always end up hitting the vm_memory_high_watermark before the throttling occurred. Now, the problem is reversed: I am throttled way too early. My publisher is barely publishing 200 messages per second, with another 600 coming from the rest of the system. In total, I haven't breached 900 messages/second.

After some research, I found a thread on the rabbitmq-discuss list, which pointed me to the flow control page: http://www.rabbitmq.com/memory.html

From the flow control page, I ran the following:

# while sleep 5; do rabbitmqctl list_connections pid name peer_address last_blocked_by last_blocked_age state | grep blocked ; done
<rabbit at amqp01.3.14347.20> 10.182.113.78:36734 -> 10.181.106.210:5672 10.182.113.78 flow 0.051966 blocked
<rabbit at amqp01.3.14347.20> 10.182.113.78:36734 -> 10.181.106.210:5672 10.182.113.78 flow 0.139588 blocked
<rabbit at amqp01.3.14347.20> 10.182.113.78:36734 -> 10.181.106.210:5672 10.182.113.78 flow 1.531009 blocked
<rabbit at amqp01.3.14347.20> 10.182.113.78:36734 -> 10.181.106.210:5672 10.182.113.78 flow 0.586091 blocked
<rabbit at amqp01.3.14347.20> 10.182.113.78:36734 -> 10.181.106.210:5672 10.182.113.78 flow 1.107713 blocked
<rabbit at amqp01.3.14347.20> 10.182.113.78:36734 -> 10.181.106.210:5672 10.182.113.78 flow 0.411649 blocked

As you can see, my publisher is essentially always blocked.

What I'm trying to achieve is replay history from a set of archive files. I run an event-sourced system, and I found the need to replay my history. A 3.5 GiB file is currently planned to finish in 6h, and I have more than 300 3.5 GiB files to replay.

I'm using Akka 1.1.3, RabbitMQ 2.8.1, Scala 2.9.0.1, OpenJDK-6, Ubuntu 11.10, all on Rackspace's virtual machines.

Did I forget something somewhere? 200 messages per second seems very paltry compared to the 2000 I had on 2.7.

Thanks!
--
François Beausoleil
http://blog.teksol.info/