[rabbitmq-discuss] Erlang crashes reports

Thu Sep 16 15:27:42 BST 2010

Hi Romary,

On 16/09/10 14:54, romary.kremer at gmail.com wrote:
> Well, I' ve done another run with the whole upgraded configuration
> - RabbitMQ version 2.1.0
> - Erlang R14B (version 5.8.1)
> - No SSL connections (but SSL listener still active on the broker, not
> used)
> 
> The memory_high_watermark is set to 0.8, 80% equivalent to 1609 MB !

That is too high. It is not advised to set the highwater mark above 50%.
Can you reproduce the problem while the vm_memory_high_watermark is set
to 0.4 ?

> I've monitored with rabbit_status, even someone had advised not to while
> benchmarking (why not ?)

rabbitmqctl or rabbit_status are invaluable for debugging the kind of
problem you are reporting. Continuous use incurs a small performance
penalty and can prevent quiescent resources from hibernating.

> The memory available raised the threshold before the end of connections
> of 10 000 peers.
> 
> Maybe the average memory occupied by a single, non SSL connection has
> somehow get bigger between
> release 1.8.x and 2.x.x ??

The data structures have changed from 1.8.1 to 2.0.0, but I don't think
that is the cause of the problem.

> Does anybody has experiment or knows the impact of the new release on
> the memory occupied by connections ?
> 
> I insist in the fact that, in our environment, we can toggle SSL
> authentication of the broker by peers, but we always
> keep the SSL listener running on the broker. The peer just "decide" to
> connect either on 5672 or 5671. In the later,
> SSL authentication will be enable, in the former, it won't !
> 
> Thanks for any other idea we can follow, because we are facing a bit of
> a dead end since we upgraded to 2.x.x !

I'm surprised that the logfile you posted previously stops after opening
ssl and tcp listeners. I would expect to see evidence of 4000+ clients
connecting.

Are you able to get output from "rabbitmqctl list_connections" before
the broker becomes unresponsive? What is the status of the beam or
beam.smp process at that point - is it still running? How much memory
and CPU is it consuming?

Regards

Emile