[rabbitmq-discuss] chan.flow and vm_memory_high_watermark

Wed Jun 9 22:27:53 BST 2010

2010/6/9 Simon MacMullen <simon at rabbitmq.com>

> On 09/06/10 16:12, Nicolás César wrote:
>
>> Is there any other way to checkit?  Is there a way to get a memory dump
>> from the vm to see whats going on?
>>
>
> Well, there's erlang:memory/0.
>
> If you're using the Debian packages, type:
>
> $ sudo -u rabbitmq -H erl -remsh rabbit@`hostname` -sname foo
>
> to establish a connection to the RabbitMQ Erlang process then at the Erlang
> prompt:
>
> memory().
>
> to give some high level statistics about what's using memory.
>

Cool!!

(rabbit at sendsorium)1> memory().
[{total,27420688},
 {processes,8840004},
 {processes_used,8793876},
 {system,18580684},
 {atom,683329},
 {atom_used,650462},
 {binary,11081456},
 {code,5263067},
 {ets,452988}]

but in top I see beam.smp using:
VIRT=141m  RES=95m

none of those numbers get close....

How can I use this information to get better debugging?

> Other things to think about:
>
> * Are you using the most recent Erlang? (R13B04) GC has improved recently.
>

I've been using:
 Erlang R13B02 (erts-5.7.3) [source] [smp:2:2] [rq:2] [async-threads:0]
[kernel-poll:false]

Is that old? I'm trying other resources before changing erlang version (and
taking my Debian to 'testing' instead of 'stable') . Thanks for the advice.
Now I'm aware of that... if everything fails, I'll do it!

> * Are you stopping the queues from hibernating? If you've written something
> like a shell script to repeatedly invoke rabbitmqctl list_queues this will
> keep the queue processes from hibernating which in turn stops them from
> GCing on hibernate. If queues are busy this is not a problem as they also GC
> every (n) reductions, but if they're almost-but-not-quite idle it can be a
> problem. It doesn't have to be a shell script, anything that means the
> queues never get 10 seconds or so idle.

This is really nice data!! Yes. I have scripts but they're cron'ed every
minute.... I'm changing that and testing again.

One small detail. I'm using:

-----------------------------------------------------
while True:
  msg = self.chan.basic_get(self.config['queue'])
  if msg:
     ....
  time.sleep(2.0)
----------------------------------------------------

because I need a fixed-rate consumer . ... could this be a similar case of
the "almost-but-not-quite idle queue"?
is there any way to tell the broker to consume at a fixed rate with
chan.basic_consume(...callback = f)?

Greetings

Nico César
http://blog.nicocesar.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20100609/c250c728/attachment.htm>