[rabbitmq-discuss] tcp -> rabbitmq gives emfile + 541

Wed Jul 1 16:31:53 BST 2009

Dear Tony,

this is excellent news, I'll definitely give the new C client a try. This is
just a pilot project, we have got big messaging needs, and I'd like to
explore the possibility of Database --> Messaging workflows much further.

In regards to the TCP server, I don't claim to be an expert in such things
but I am under the impression all sort of different resources consume file
descriptors in POSIX systems, including sockets, and while a socket is in
TIME_WAIT state a fd is still being used. (see e.g.
http://judystak.tistory.com/136)

by raising the nofile ulimit for the right users, I think I have gone past
this problem and what happens now is that RabbitMQ crashes because all the
undelivered messages are written to disk (for persistence) but the whole set
stays in memory, which is eventually exhausted. I don't know if this is out
of date information, but see
http://www.nabble.com/RabbitMQ-memory-management-td19428592.html

Now, I distinctly remember Matthew speaking about a mechanism which monitors
free memory and offloads to disk to the point of RabbitMQ working as long as
there is still room for a single message in RAM, but I don't remember if
this is already available or when it's going to be.

If I am right, attaching a fast enough consumer to the queue while running
my tests will probably take care of the crashes, I have only experienced
crashes when the number of undelivered messages exceeded 200000 messages.

overall, I am very happy with rabbitmq, I'm only complaining (not really)
because you have set the bar high :)

regards, Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20090701/1d13a9e0/attachment.htm 