[rabbitmq-discuss] Connection always blocked

Fri Apr 13 19:35:04 BST 2012

I'm replaying history in an event sourced system. I use Scala + Akka
to read serialized messages from disk, and publish to RabbitMQ, where
regular consumers will handle the messages and all that jazz. The
messages are published to a Topic exchange, with 12 queues bound to
the exchange, using left-anchored routing keys: "#", or "abc.#" and
similar. I'm using RabbitMQ 2.8.1, Akka 1.1.3, Scala 2.9.0.1, on
Rackspace's VMs, using Ubuntu 11.10.

I just researched on this mailing list and found a reference to the
flow control page: http://www.rabbitmq.com/memory.html. I went ahead
and read the document, and checked what my connection was doing:

# while sleep 5; do rabbitmqctl list_connections pid name peer_address
last_blocked_by last_blocked_age state | grep blocked ; done
<rabbit at amqp01.3.14347.20>      10.182.113.78:36734 ->
10.181.106.210:5672      10.182.113.78   flow    0.105658
blocked
<rabbit at amqp01.3.14347.20>      10.182.113.78:36734 ->
10.181.106.210:5672      10.182.113.78   flow    0.470772
blocked
<rabbit at amqp01.3.14347.20>      10.182.113.78:36734 ->
10.181.106.210:5672      10.182.113.78   flow    1.082906
blocked
<rabbit at amqp01.3.14347.20>      10.182.113.78:36734 ->
10.181.106.210:5672      10.182.113.78   flow    0.462744
blocked

As you can see, my connection is essentially always blocked.
Previously, under 2.7.1, my connection wouldn't block until I hit the
vm_memory_high_watermark. I upgraded to 2.8.1 specifically to prevent
such an issue from reoccuring, but I didn't anticipate the utter loss
of performance: under 2.7.1, I would publish 2000+ messages per
second, and I can barely achieve 900 now. I must have done something
wrong.

Can anyone help me achieve more performance?

Thanks a lot!
François Beausoleil