Brian,<div><br></div><div>It seems like you should be using Pika - which support channel.flow:</div><div><br></div><div><a href="http://github.com/tonyg/pika">http://github.com/tonyg/pika</a></div><div><br></div><div>As for the other issues, it seems like you'll certainly want to use high memory extra large instances (or larger) for the broker. You can also use beefier clients (fewer of them). I know these are just mitigating strategies but they seem to make sense for your situation.</div>
<div><br></div><div>-Adam</div><div><br><div class="gmail_quote">On Tue, Sep 28, 2010 at 11:42 AM, Brian_Fox <span dir="ltr"><<a href="mailto:genkuro@gmail.com">genkuro@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
<br>
<br>
Matthew Sackman wrote:<br>
><br>
> On Wed, May 19, 2010 at 11:06:23AM -0400, Tyler Williams wrote:<br>
>> =INFO REPORT==== 17-May-2010::14:24:30 ===<br>
>> application: rabbit<br>
>> exited: {{timeout_waiting_for_tables,[rabbit_disk_queue]},<br>
>> {rabbit,start,[normal,[]]}}<br>
>> type: temporary<br>
>><br>
>> =ERROR REPORT==== 17-May-2010::14:24:30 ===<br>
>> Error in process <0.100.0> on node 'rabbit@echonest03' with exit<br>
>> value:<br>
>> {badarg,[{ets,insert,[rabbit_disk_queue,[{dq_msg_loc,{{resource,<<1<br>
>> byte>>,queue,<<9 bytes>>},553567},false,true,<<16<br>
>> bytes>>},{dq_msg_loc,{{resource,<<1 byte>>,queue,<<18<br>
>> bytes>>},563611},false,true,<<16 bytes>>},{dq_msg_loc,{{resource,<<1<br>
>> byte>>,queue,<<18 bytes>>},80964},false,true,<<16<br>
>> bytes>>},{dq_msg_loc,{{resource,<<1 byte>>,queue...<br>
>><br>
>><br>
>> =INFO REPORT==== 17-May-2010::14:24:33 ===<br>
>> application: mnesia<br>
>> exited: stopped<br>
>> type: temporary<br>
>><br>
>> =INFO REPORT==== 17-May-2010::14:24:33 ===<br>
>> application: os_mon<br>
>> exited: stopped<br>
>> type: temporary<br>
>><br>
>> Can you shed any light on this error?<br>
><br>
> I might have been able to last October when I was writing that code. So<br>
> much has changed since then that I really can't remember too much about<br>
> what was going on back then.<br>
><br>
> Also, pyamqplib does not support channel.flow. Even with the new<br>
> persister, there are times where rabbit gets overwhelmed (disks just<br>
> aren't fast enough most of the time) and so needs to temporarily stop<br>
> publishers. It does this by using channel.flow. Given that pyamqplib<br>
> doesn't support channel.flow, you are very likely to run into problems.<br>
> There's a highly experimental patch by Tony (search the mailing list for<br>
> it) which adds support for channel.flow, but you might also try looking<br>
> at the other python clients such as txamqp and pika.<br>
><br>
> Matthew<br>
><br>
<br>
(Sorry to resurrect an older thread)<br>
<br>
Has this rabbitmq + pyamqplib problem ever been resolved?<br>
<br>
Our group is experimenting with bringing up a massive amount of AMQP clients<br>
using EC2. And we're running into the same error described above,<br>
presumably slamming memory on the broker. It seems like we have a few<br>
choices: bump up the memory (8GB currently), choose a different library with<br>
flow control, or choose a different strategy to rapidly boot several hundred<br>
clients. Of the three, the last would be the least attractive option.<br>
<br>
Any suggestions on the best alternative lib to avoid this error? I'm<br>
hearing mixed things about txAMQP's support of channel flow.<br>
<br>
Would clustering help the issue or make it worse? (we're currently a single<br>
broker on a development platform)<br>
<br>
Thanks much,<br>
Brian<br>
<font color="#888888">--<br>
View this message in context: <a href="http://old.nabble.com/Erlang-has-closed--tp28600093p29827331.html" target="_blank">http://old.nabble.com/Erlang-has-closed--tp28600093p29827331.html</a><br>
Sent from the RabbitMQ mailing list archive at Nabble.com.<br>
<br>
_______________________________________________<br>
rabbitmq-discuss mailing list<br>
<a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>
<a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
</font></blockquote></div><br><br clear="all"><br>-- <br>Adam<br>
</div>