[rabbitmq-discuss] queue hands - timeout

[Thomas Stagl]

Hello,

we are running a queue in our test environment with different vhosts. When we do a long running perf test, we recognise that the responses from the queue are timing out and from that moment on, it is not possible to connect to the queue again. Neither with the java lib (which we use in our application) nor with celery (which I use for testing purposes.)

We have recognised that beam.smp is running on 30% CPU time from that moment on.

When we trace a connection, we see this with tcpdump:
12:23:52.700093 IP CLIENT.43248 > RABBIT.amqp: Flags [P.], seq 2421288989:2421289002, ack 2617563671, win 54, options [nop,nop,TS val 2047452 ecr 1377663677], length 13
12:23:52.700124 IP RABBIT.amqp > CLIENT.43248: Flags [.], ack 13, win 62, options [nop,nop,TS val 1377881158 ecr 2047452], length 0

From that moment on, the connection is hanging.

After a restart of the rabbitmq server, everything is running fine again, for a certain amount of time.

We also have a second test stack and we switched to the second rabbitmq test server, same behaviour here. It worked fine and after a couple of hours it stopped responding.

Any ideas? Any though about where we can start debugging would be very much appreciated.

Best regards, tom