[rabbitmq-discuss] Stale connections when using rabbitmq
Venkat
s.ramanan.venkata at gmail.com
Wed May 9 07:35:25 BST 2012
Hi,
I have a web server which communicates with my backend using rabbitmq.
The web server receives req at the rate of 3000/sec. I had a rabbitmq
server on 8GB i7 machine. I saw rabbitmq refusing further connections
after a while. When I did rabbitmqctl list_connections, I saw lot of
connections in 'blocked'/'blocking' state. I recently moved from
memcache based queue for want of async nature. Please suggest me ways
to debug this situation.
[Note: I reduced the RAM to 2GB to reproduce it quicker]
Listing connections ...
guest blocked
guest blocked
guest 172.16.223.114 51914 blocked
guest 172.16.223.113 33600 blocked
guest 172.16.223.114 46209 blocked
guest 172.16.223.114 53108 blocked
guest 172.16.223.114 52984 blocked
guest 172.16.223.114 52310 blocked
guest 172.16.223.114 51776 blocked
guest blocked
guest 172.16.223.114 52390 blocked
guest blocked
guest 172.16.223.114 53374 blocked
guest 172.16.223.114 53252 blocked
guest blocked
guest 172.16.223.114 39293 blocking
guest 172.16.223.114 52772 blocked
guest 172.16.223.114 52000 blocked
guest blocked
guest 172.16.223.114 53072 blocked
guest 172.16.223.114 52852 blocked
guest 172.16.223.114 51862 blocked
guest 172.16.223.114 53601 blocked
guest blocked
guest 172.16.223.114 52258 blocked
guest blocked
guest blocked
guest 172.16.223.114 39286 blocked
guest 172.16.223.114 53200 blocked
guest blocked
guest 172.16.223.114 51746 blocked
guest 172.16.223.114 52940 blocked
guest 172.16.223.114 32803 blocked
guest 172.16.223.114 53020 blocked
guest 172.16.223.114 52346 blocked
root at testmachine3:~/$ sudo rabbitmqctl list_connections | wc -l
830
sudo rabbitmqctl status
Status of node rabbit at testmachine3 ...
[{pid,939},
{running_applications,[{rabbit,"RabbitMQ","2.7.1"},
{mnesia,"MNESIA CXC 138 12","4.4.17"},
{os_mon,"CPO CXC 138 46","2.2.5"},
{sasl,"SASL CXC 138 11","2.1.9.3"},
{stdlib,"ERTS CXC 138 10","1.17.3"},
{kernel,"ERTS CXC 138 10","2.14.3"}]},
{os,{unix,linux}},
{erlang_version,"Erlang R14B02 (erts-5.8.3) [source] [64-bit] [smp:
2:2] [rq:2] [async-threads:30] [kernel-poll:true]\n"},
{memory,[{total,1626268744},
{processes,1296143128},
{processes_used,1296128032},
{system,330125616},
{atom,3600969},
{atom_used,3573207},
{binary,6811184},
{code,11216694},
{ets,305365584}]},
{vm_memory_high_watermark,0.4},
{vm_memory_limit,1593409536}]
...done.
When my web server tries to connect to rabbitmq at this stage it give
me the following exception.
File "/usr/local/lib/python2.7/dist-packages/msgbroker.py", line
194, in __init__
self.connection =
pika.BlockingConnection(pika.ConnectionParameters(host=host_server))
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/
blocking_connection.py", line 32, in __init__
BaseConnection.__init__(self, parameters, None,
reconnection_strategy)
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/
base_connection.py", line 50, in __init__
reconnection_strategy)
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py",
line 170, in __init__
self._connect()
File "/usr/local/lib/python2.7/dist-packages/pika/connection.py",
line 228, in _connect
self.parameters.port or spec.PORT)
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/
blocking_connection.py", line 44, in _adapter_connect
self._handle_read()
File "/usr/local/lib/python2.7/dist-packages/pika/adapters/
base_connection.py", line 151, in _handle_read
data = self.socket.recv(self._suggested_buffer_size)
timeout: timed out
Thanks,
More information about the rabbitmq-discuss
mailing list