[rabbitmq-discuss] Problems with rabbit and hoping I can get some help

[Dathan Pattishall]

Let me first describe my setup.

root at webnode1]# rabbitmqadmin.py show overview
+--------------------+-----------------+--------------------+------------------+
| management_version |      node       | statistics_db_node |
statistics_level |
+--------------------+-----------------+--------------------+------------------+
| 2.7.1              | rabbit at webnode1 | rabbit at webnode1    | fine
    |
+--------------------+-----------------+--------------------+------------------+

Rabbit MQ's producers comes from PHP 5.3.8
http://www.php.net/manual/en/book.amqp.php. Each apache process could
produce a rabbit message, I am producing around 1000 messages a second on
 c1.xtralarge instance at ec2.

My erlang version is

/usr/local/bin/erl -v
Erlang R15B (erts-5.9) [source] [64-bit] [smp:8:8] [async-threads:0] [hipe]
[kernel-poll:false]


The PROBLEM:

After about 40 mins of rabbit accepting messages all connections block
causing a rather bad error on the front ends killing traffic. Turning
rabbit off and restarting the web servers forces a recovery.

Stats from Rabbit:

Roughly 5000 queues are made
Roughly 3600 exchanges are made
Each exchange can have at most 1200 queues bound to it.
Each Queue is setup for autodelete and so is the exchanges with delivery
type 1.
All data passed is JSON

The consumer is NODE and its keeping up with the consumption

RabbitMQ memlimit is around 5.3G
RabbitMQ mem used hits around 1.9G when it freezes produces
RabbitMQ proc used hits around 220K
RabbitMQ fd_total is 50K
RabbitMQ socks_total is around 45K and Socks used is 4K
mem_ets hists 100M // not sure what this is

Any idea what is going on? What limit am I hitting? Why does RabbitMQ
block? How can I detect that I am about to hit a block state? Any
suggestions or request of additional data would be great.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120131/1331150d/attachment.htm>