<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
I tried the high water mark setting and it now prevents too much memory
from being used (barely) but as my test continued it eventually had a
hard crash for another reason. I was hitting it with 2 producers
throttled to 1000 messages each to a separate queue and 5 consumers.
There was always 1000 messages each in the 2 queues and after 15
minutes one of the producers crashed with an error (from the logs)
{amqp_error,not_found,"no queue 'xxx' in vhost '/'", queue.declare'}. I
figured this was a timeout or something as the queue did exist and was
being emptied by 3 different consumers actively.<br>
<br>
5 seconds later the entire process died with an error (from the logs)
"exception on TCP connection ... connection_closed_abruptly" and then
"closing TCP connection".<br>
<br>
I am new to queuing but frankly I don't know what to do. My Experience
with this and other MQ products is less than compelling. Does this take
a ton of tweaking to get stable? How do people manage to push 500k
messages a day? <br>
<br>
<br>
Wayne<br>
<br>
<br>
Matthew Sackman wrote:
<blockquote cite="mid:20100507120026.GJ2739@mrnibble.lshift.net"
type="cite">
<pre wrap="">Hi Wayne,
On Fri, May 07, 2010 at 07:48:54AM -0400, Wayne Van Den Handel wrote:
</pre>
<blockquote type="cite">
<pre wrap="">I will test the basic consume, thanks for the tip. The machine I am running
is a single socket 4 core with 4 gigs ram but there are other things running
and taking up memory. Is there a gc interval that can be set to get memory
back sooner? As I add messages and remove them I can see the exact memory
taken with the admin tool and it keeps going up. It seems the rate at which
I add to the queue exceeds the ability the clear memory. Basically even
though there is little in the queue at any time the memory use keeps going
up.
</pre>
</blockquote>
<pre wrap=""><!---->
Rabbit assumes it gets the whole machine's memory to itself. If that's
not the case then you need to reduce the memory high watermark from 0.4
to some much lower figure - say, 0.1.
See <a class="moz-txt-link-freetext" href="http://www.rabbitmq.com/extensions.html#memsup">http://www.rabbitmq.com/extensions.html#memsup</a>
</pre>
<blockquote type="cite">
<pre wrap="">At this point I am ready to give up until they harden the persister to
go to disk as I assume the persister is the problem.
</pre>
</blockquote>
<pre wrap=""><!---->
Yup. The new persister currently requires you build from source, but it
is actively being QA'd and great progress has been made getting it into
default. We've gone from about 100 hunks difference a couple of weeks
ago to now about 13 - so things are very much moving in this direction.
In the mean time, many people are already using the new persister, which
is on branch "bug21673" and if you're comfortable compiling from source
and using some code which has not yet completed our QA process, then we
would encourage you to use it, with the understanding that there may
still be bugs in it. Further battle testing is always welcome and should
you come across any bugs, any bug reports you make are very gratefully
received.
Matthew
</pre>
</blockquote>
<br>
<br>
</body>
</html>