[rabbitmq-discuss] rabbitmq memory grows with queue size

[Matthew Sackman]

For the list, I have replied to this off list. The reason being that the
new disk queue is not ready for a public release yet, we do expect there
still to be bugs in it, the code has not been QA'd yet, and about 25% of
the code base of Rabbit has changed. A substantial portion of the new
disk queue will change again over the coming weeks as it goes through
QA. Obviously, anyone wishing to try such experimental code would be
very well advised to treat it as such.

If you are desperate, despite such grave warnings, to try the new disk
queue then you may contact me directly off list. However, it will be
several weeks before this gets formally released, and I will not be
providing binary packages at this point whatsoever. Support is almost
certainly, at this very early point, to be "if you can't make it work,
then don't be surprised. Wait for the code to be released. (But, if it
does work for you, we'd be pretty interested in hearing how well it's
working for you!)"

In other words, if you're used to building Linux development kernels
(i.e. pre RC) then you may get some mileage out of this ;)

Matthew

On Mon, Jun 15, 2009 at 10:28:55AM -0400, Brian Whitman wrote:
> Also, which branch? I can't seem to find anything appropriate here:
> http://hg.rabbitmq.com/rabbitmq-server
> 
> 
> On Mon, Jun 15, 2009 at 10:15 AM, Brian Whitman <brian at echonest.com> wrote:
> 
> > Would love to try this out -- are there any configuration things I need to
> > know about, or will a default build from the branch "just work"?
> >
> >
> > On Mon, Jun 15, 2009 at 6:11 AM, Alexis Richardson <
> > alexis.richardson at gmail.com> wrote:
> >
> >> Chuck - that's correct :-)
> >>
> >> Jack, all,
> >>
> >> The "page to disk" feature is now getting into QA and should be
> >> visible on a branch on the repo.  If you would like to help us QA this
> >> please speak out.  Matthias, as per usual, is in charge of getting
> >> this feature into a releasable state.
> >>
> >> The new feature enables messages that are persisted to exist on disk
> >> and not in memory.  Previously persisted data would remain in memory
> >> as well as being written to disk.
> >>
> >> More generally, you can still drain queues into any store
> >> programmatically, in the meantime.
> >>
> >> alexis
> >>
> >>
> >>
> >> On Thu, Jun 11, 2009 at 11:46 PM, Chuck Remes<cremes.devlist at mac.com>
> >> wrote:
> >> >
> >> > On Jun 11, 2009, at 5:29 PM, Jack Moffitt wrote:
> >> >
> >> >> Perhaps I missed a config option, but I am running out of memory on
> >> >> the machines running rabbitmq-server.
> >> >>
> >> >> I have two queues and two fanout exchanges (each queue is bound to one
> >> >> of the exchanges).   As the queue size grows so does memory use, even
> >> >> though both queues are durable. and the messages persistent.  Does
> >> >> rabbitmq require everything in memory as well even for persistent
> >> >> queues?
> >> >
> >> > Regardless of queue type or settings, RabbitMQ stores all messages in
> >> > memory. There is work on a new "persister" ongoing in one of the
> >> > development branches to resolve this limitation. As far as I know
> >> > there hasn't been any date set for its release, but it should be
> >> > sometime this year.
> >> >
> >> > cr
> >> >
> >> >
> >> > _______________________________________________
> >> > rabbitmq-discuss mailing list
> >> > rabbitmq-discuss at lists.rabbitmq.com
> >> > http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> >> >
> >>
> >> _______________________________________________
> >> rabbitmq-discuss mailing list
> >> rabbitmq-discuss at lists.rabbitmq.com
> >> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> >>
> >
> >

> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss