[rabbitmq-discuss] Producers hanging when reaching high memory watermark on 1.8.1

Mon Aug 16 16:04:23 BST 2010

On Mon, Aug 16, 2010 at 10:44 AM, Matthias Radestock
<matthias at rabbitmq.com> wrote:
> If you can live with the massive performance degradation that swapping
> entails, and have plenty of swap configured, then disabling rabbit's memory
> monitoring is a viable alternative.

We can, and we'd take corrective action once we saw that condition arise.

> Though once the new persister is released, I don't see what you'd be gaining
> with that. The new persister provides a much more sophisticated and
> performant swapping mechanism than the kernel can provide.

Really looking forward to this.  We've held off on deploying 1.8.x
because of the flow control changes, but plan to roll out because of
the new persister.  Will the persister only write to disk based on
vm_memory_high_watermark, or is there another control?

> Once rabbit starts swapping, clients will experience pauses in publishing
> anyway, through TCP backpressure.
>
> By "shifting the burden of memory management" do you mean "having to deal
> with the channel.flow message"? The AMQP client library should take care of
> that.

When I modded py-amqplib to support libevent, I didn't implement flow
control.  I see its value, but for our usage, implementing it would
only shift the backlog that's consuming memory from the Rabbit hosts
to the client hosts.  I do plan to add it back when I refactor our
work for publication, though we're still not likely use it.

-Aaron

-- 
Aaron Westendorf
Senior Software Engineer
Agora Games
359 Broadway
Troy, NY 12180
Phone: 518.268.1000
aaron at agoragames.com
www.agoragames.com