[rabbitmq-discuss] chan.flow and vm_memory_high_watermark

Thu Jun 10 14:02:39 BST 2010

El 10 de junio de 2010 08:26, Simon MacMullen <simon at rabbitmq.com> escribió:

> On 09/06/10 22:27, Nicolás César wrote:
>
>> (rabbit at sendsorium)1> memory().
>> [{total,27420688},
>>
> <snip>
>
>  but in top I see beam.smp using:
>> VIRT=141m  RES=95m
>>
>
> So there is some disparity here but not as much as when you were seeing
> 400M used. I take it this was a different run?

Yess... the OOM assassin was with his gun killing my processes! Rabbit went
down fist! Thanks to you I have the memory(). trick for the next time.

>    * Are you stopping the queues from hibernating? If you've written
>>    something like a shell script to repeatedly invoke rabbitmqctl
>>    list_queues this will keep the queue processes from hibernating
>>    which in turn stops them from GCing on hibernate. If queues are busy
>>    this is not a problem as they also GC every (n) reductions, but if
>>    they're almost-but-not-quite idle it can be a problem. It doesn't
>>    have to be a shell script, anything that means the queues never get
>>    10 seconds or so idle.
>>
>>
>> This is really nice data!! Yes. I have scripts but they're cron'ed every
>> minute.... I'm changing that and testing again.
>>
>
> Every minute *should* be OK, it's more like every couple of seconds which
> can prevent the hibernation.

I've changed it to 10 minutes and there were no significant behavior
changes.

> One small detail. I'm using:
>
> -----------------------------------------------------
> while True:
>   msg = self.chan.basic_get(self.config['queue'])
>   if msg:
>      ....
>   time.sleep(2.0)
> ----------------------------------------------------
>
> because I need a fixed-rate consumer . ... could this be a similar case
> of the "almost-but-not-quite idle queue"?
>

Yes, as you'll still be waking the queue up every 2 seconds even if it's
> empty. I suspect this is your real problem.

Bingo! thanks Simon for your help in this issue!! now it's on the mail
archive, I hope this mail helps next googling rabbiter. :)

>
>  is there any way to tell the broker to consume at a fixed rate with
>> chan.basic_consume(...callback = f)?
>>
>
> Yes, basic.qos. This not-very-well-named method lets you specify how many
> unacked messages the server will send to you at any time. So set qos to a
> low value, then use basic_consume with a sleep before acking.
>

Nice to hear a yes here!! I'll be changing my code to get this working!
Thanks!

Nico Cesar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20100610/8178669e/attachment.htm>