Hi,<div><br></div><div>I come back here about this assumption, it's a big concern today for us.</div><div><br></div><div>Let's explain the facts.</div><div><br></div><div>We have 1 exchange e 1 persistent queue binding for routing key = book.APPL</div>
<div><br></div><div>We have a publisher putting messages from exchange, each message have ~213 bytes and each routing key identify the goup of message like book.APPL, book.PETR4 and etc. We are publishing about ~3000 m/s</div>
<div><br></div><div>We have one consumer listening queue.</div><div><br></div><div>The problem.</div><div><br></div><div>We put test to run for 1 hour, and monitoring queue size with rabbitmqctl list_queues, all the time list have 0 size, so consumer are getting all on the fly, and now we stop publisher and here is the BIG PROBLEM, consumer continue to consume messages from queue... and it's remain for 20 minutes after stop publisher......</div>
<div><br></div><div>If we stop consumer we can see the queue growing for long time..., therefore looking for it's we can conclude that our consumer are 20 minutes late behind publisher and it's a f... problem last message from publisher will be received by consumer just 20 minutes after.</div>
<div><br></div><div>About environments.</div><div><br></div><div>We are using a P550 Machine with 4 PPC64 processors 5.4Ghz, and 100Gb of ram, during our tests it's have just 20% of processor busy, and 2Gb of memory allocated.</div>
<div><br></div><div>We are using Red Hat 5 for ppc in our tests.</div><div><br></div><div>We replicate this scenario using Intel x86_64 with Red Hat linux for 64bits, and Real Time Kernel, this machine have 8 cores and 32Gb of ram, and during the tests it consume just 25% of CPU and 2Gb of ram.</div>
<div><br></div><div>So can anyone help me ? Its's a common problem in Rabbit ? anybody here have this same problem ? We have talked about a possible exchange buffer cause and etc in this way, but to be sincerely in my thoughts ~3000 m/s is a bit easy to be processed for one message server.</div>
<div><br></div><div>Gustavo.</div><div><br></div><div><br></div><div> </div><div><br><br><div class="gmail_quote">On Fri, Mar 26, 2010 at 5:15 PM, Bryan Murphy <span dir="ltr"><<a href="mailto:bmurphy1976@gmail.com">bmurphy1976@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Make sense, although we don't care about the order so it doesn't affect us so much.<br><font color="#888888"><br>
Bryan</font><div><div></div><div class="h5"><br><br><div class="gmail_quote">On Fri, Mar 26, 2010 at 1:11 PM, Matthew Sackman <span dir="ltr"><<a href="mailto:matthew@lshift.net" target="_blank">matthew@lshift.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">Hi,<br>
<div><br>
On Fri, Mar 26, 2010 at 08:33:19AM -0500, Bryan Murphy wrote:<br>
> Another technique we use:<br>
><br>
> Start one consumer.<br>
><br>
> Start your other consumers.<br>
><br>
> Restart the first consumer.<br>
><br>
> This let's you keep the high prefetch settings while still getting the<br>
> messages to distribute more evenly.<br>
<br>
</div>I would not recommend that at all - you're likely to get messages in<br>
different orders with this scheme. QoS is much better idea, or, use<br>
channel.flow from the client (may only work in newer-than-1.7.2 - can't<br>
remember when it appeared) to prevent any messages being sent out<br>
*before* issuing the basic.consume.<br>
<br>
You could then have either a delay or some signal through some other<br>
exchange and queues (and channel) to get the clients to drop the<br>
channel.flow and start consuming.<br>
<font color="#888888"><br>
Matthew<br>
</font></blockquote></div><br>
</div></div><br>_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
<br></blockquote></div><br></div>