<span class="Apple-style-span" style="font-family: arial, sans-serif; font-size: 13.2px; border-collapse: collapse; ">Thanks for the pointer looking at the queues was very helpful figured out I was never cleaning out the server queues and I kept re-creating a new one each time it was restarted and they were all bound to the same topic. �So the sending of 100k messages from the client resulted in coping those 100k messages into tens of queues making millions of copies of messages.<div>
<br></div><div>I changed the server queue to be non-durable and it looked like I can do many passes with no crash now.</div><div><br></div><div>Thanks!</div><div><br></div><font color="#888888"><div>Jared</div></font></span><br>
<div class="gmail_quote">On Mon, Aug 2, 2010 at 11:36 AM, David Wragg <span dir="ltr"><<a href="mailto:david@rabbitmq.com">david@rabbitmq.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hi Jared,<br>
<div class="im"><br>
Jared Smith <<a href="mailto:jaredtsmith@gmail.com">jaredtsmith@gmail.com</a>> writes:<br>
> Everything works good for the first pass where I send 100k<br>
> request/response pairs and I'm getting on the order of 4-5k rpc calls<br>
> per second. �To make amqplib have blocking semantics I registered a<br>
> callback with basic_consume and have that deposit messages that are<br>
> read into a blocking queue which the client thread reads from this; it<br>
> seems to work ok as far as I can tell.<br>
><br>
> Problem I'm seeing is that on the second run of the program I get the<br>
> following errors:<br>
><br>
> ==============client error=================<br>
</div>[...]<br>
<div class="im">> u'PRECONDITION_FAILED - timeout waiting for channel.flow_ok{active=false}',<br>
> (0, 0), '')<br>
> =======================================<br>
><br>
> ============== matching server error ================<br>
</div>[...]<br>
<div class="im">> amqplib.client_0_8.exceptions.AMQPConnectionException: (503,<br>
> u'COMMAND_INVALID - basic.publish received after<br>
> channel.flow_ok{active=false}', (60, 40), 'Channel.basic_publish')<br>
> ==============================================<br>
<br>
</div>It's hard to be sure from the fragments of code you posted, but I wonder<br>
if the issue is that you aren't acknowledging the messages? �To do this,<br>
you either need to call basic_consume with no_ack=True (to tell rabbit<br>
not to expect acknowledgements), or call basic_ack in your consuming<br>
code to explicitly ack the messages.<br>
<br>
If you don't acknowledge the messages, then they will stay in the queue,<br>
and take up memory, eventually resulting in the channel.flow methods<br>
indicated above.<br>
<br>
You could confirm that this is the issue by doing a<br>
<br>
� �$ sudo rabbitmqctl list_queues<br>
<br>
This will list the queues, along with the count of messages in each<br>
queue. �If you do this after your first run, and see that a lot of<br>
messages are still present, then it is very likely to be the lack of<br>
acks causing the problem.<br>
<br>
David<br>
<font color="#888888"><br>
--<br>
David Wragg<br>
Staff Engineer, RabbitMQ<br>
SpringSource, a division of VMware<br>
</font></blockquote></div><br>