[rabbitmq-discuss] Memory grows very fast
matthias at rabbitmq.com
Thu May 23 19:58:52 BST 2013
On 23/05/13 15:44, rabbit15 wrote:
> I see that on faster machine which has 32 CPU(Intel(R) Xeon(R) CPU E5-2670 0
> @ 2.60GHz) and 64GB RAM there can be store more messages about 10 milions.
> Previous machine was with 24 CPU (Intel(R) Xeon(R) CPU X5675 @
> 3.07GHz) and 92 GB RAM and there could 600k -2500k stored messages.
That's probably just due to the changes in timing.
You are clearly putting rabbit under extreme load. In the status report
you just posted, the system had been running for just ~12 minutes, yet
there were 800+ open connections, taking up 1.3GB of memory between
them. Now, connections only hold data very briefly, so this is a clear
indication that the rabbit is heavily loaded. Furthermore, in an earlier
report you posted, the run_queue sat at 148 - that's 148 Erlang
processes waiting for a CPU to become free to do some work.
With such a usage, memory consumption will always be high.
> Maybe there is some option for GC ?
There are plenty of Erlang gc options to tweak. Usually that makes
matters worse, but if you really want to go down that route then, for
starters, try setting the ERL_FULLSWEEP_AFTER environment variable to 0
or some other low value (the default is 65535) in
export ERL_FULLSWEEP_AFTER 0
and restart rabbit.
You can confirm the setting took effect by running
$ rabbitmqctl eval 'erlang:system_info(fullsweep_after).'
More information about the rabbitmq-discuss