[rabbitmq-discuss] Cluster Memory Usage

Wed Nov 16 20:06:57 GMT 2011

I have a cluster of 2 rabbitmq on 2 machines in the same datacenter. I
am sending messages to them at a rate of about 250 message a second
each message is about 2k in size on average. The software is able to
read them back out at about the same rate. Sometime slower usually
faster, which is why I am using a message queue. My question though is
about the memory usage. The machine has 24 gigs of ram but rabbitmq is
not the only thing running on it. I set the high water mark to .1 so
about 2.4 GB of RAM and have had it running for about a week now. The
cluster is using mirrored queue and confirmations on the writes as
well as batch acking on the consumes. The master rabbitmq is reporting
a memory usage by the erlang processer of about 200MB and the slave
rabbitmq is reporting around 800MB but sometimes jumps back and forth
between 400MB and 900MB rapidly. Is there any reason that the memory
usage is so different between the master and the slave, I have seen it
go into memory alert, which makes all the producers slow down, once
before and have been monitoring it every since. When it went into
memory alert I restarted both of the rabbitmqs and things have been
better since. There are about 10 publishers channels/connections, and
about 30 consumer channels/connections. The prefetch is set to 1000
for each channel of the consumers.

Thanks,
Cameron