[rabbitmq-discuss] Erlang crashes reports

[Emile Joubert]

Hi Romary,

Thanks for this error report. Can you confirm whether the logfile
contained anything relevant before the broker became unavailable? Do you
override any default configuration variables in the broker config file?

If you find memory usage growing with SSL then try using Erlang R14B.
(Also see OTP-8810) What version of Erlang are you currently using?


Regards

Emile





On 16/09/10 10:32, Romary Kremer wrote:
> We are performing evaluation on rabbitmq message broker, and we
> currently encounter difficulties with release 2.0.0:
> 
> - Our application implies 10 000 peers producing messages periodically
> to a unique queue. This queue is listen asynchronously by another peer.
> - All peer are written in Java.
> - The production rate of a single peer is 4 messages / hour.
> - We can simulate a time-consuming task in the consumer callback,
> simulating more or less fast consumer.
> - we are using SSL certificate on the broker side to allow the peer to
> authenticate the broker.
> - we have noticed that the use of SSL as dramatic incidence on the
> memory occupied by Rabbitmq process
> 
> Since we upgraded to version *2.0.0*, we are no longer able to make a
> test scenario running. The symptoms are listed bellow :
> 
> on the broker console first, we get the message :
> 
> Erlang has closed
> Error: unable to connect to node rabbit at murphys: nodedown
> diagnostics:
> - nodes and their ports on murphys: [{rabbitmqctl22609,42767}]
> - current node: rabbitmqctl22609 at murphys
> - current node home dir: /var/lib/rabbitmq
> - current node cookie hash: qu0gh1hg7j7LKyzK0GLk+A==
> 
> we have kept the erl_crash.dump in case, but since i's about 200 MB  I
> cannot do nothing to send it to you.
> Maybe some one can give us some hints or some indicators to look out in
> the dump to help diagnostics, but we are not Erlang fluent !
> 
> What we know for sure is that the crash happens while the 10 000
> connections are established, at the beginning of the test. 
> We have monitored the  number of connections established and the crashes
> happens always around 4500 - 5000 connections, but never the same exact
> number. 
> We also tried with and without SSL but this does not help at all (same
> symptoms).
> 
> On the client side, our application registers a ShutdownListener to
> implement a connection retry logic upon shutdown. 
> The retries always failed with the error : connection refused.
> 
> here are some figures we gathered during the test start up about the
> maximum number of connection established before it crashes 
> 
> - with SSL : 5404, 4493, 4399
> 
> - without SSL : 4673
> we dont think that the problem is about file descriptors since we
> haven't changed anything in the configuration when we upgraded to 2.0.0.
> The same test used to run successfully on previous version of the broker
> (1.7.2, and 1.8.1).
> Moreover, the rabbit_status plugin tells us we have enough file
> descriptors as well as erlang processes
> - file descriptors (used / available)= 34 / 65535
> - elrang processes (used / available)= 160 / 1 000 000
> - memory (used / available)= 40 MB / 1609 MB
> 
> We haven't try the *2.1.0* yet because we would like to have your
> feedback about this issue before.
> 
> We would appreciate your feedbacks on that point before we migrate to
> release 2.1.0.
> 
> Best regards,
> 
> Romary.