[rabbitmq-discuss] Per-Connection Flow Control - RMQ 2.8.1

Mon Apr 16 11:55:51 BST 2012

I would have thought that in your scenario you would be more bound by 
disc performance than anything else - aren't you having messages paging 
off the disc and being expired?

Just to be clear, do you have a message rate for your use case that 
leads to *stable* memory use in 2.7.1 but which cannot be maintained by 
2.8.1? I was unable to find one with my attempts at replicating your 
scenario.

Cheers, Simon

On 12/04/12 15:52, DawgTool wrote:
> Hi Simon,
>
> Just a quick update. I have collected some stats, but mostly looking at
> the rabbitmqctl list_channels. Anyway, I will get those graphed at some
> point.
>
> I did however build the same setup (2.8.1/R15B) on a 16 core/96GB
> machine and I hit the same limits as the 4 core/16GB.
>
> So right now it doesn't appear to be a 'horse-power' issue, possible its
> a logic/functional issue. Let me know what you think.
>
> Thanks
>
>
> On 3/28/12 8:42 AM, Simon MacMullen wrote:
>> On 28/03/12 11:32, Simon MacMullen wrote:
>>> I will try to reproduce based on the above, but anything else you can
>>> tell me would be a great help.
>>
>> Hmm, I'm struggling with this.
>>
>> I can certainly get into a state with a combination of x-message-ttl,
>> lots of messages expiring, and enough memory pressure to push much of
>> the queue onto disc, such that the queue performance fluctuates quite
>> a bit (as it switches between accepting new messages / paging messages
>> out to disc / paging them back in / expiring them).
>>
>> But I can't get a message rate that works OK in this scenario with
>> flow control disabled but blocks for long periods with it enabled. If
>> the message rate is high enough for flow control to kick in, then it's
>> also high enough for memory use to go up as messages wait to enter the
>> queue if flow control is disabled. Eventually the memory alarm goes
>> off instead.
>>
>> So can you provide more information about your use pattern?
>>
>> Having said that, if you just want to disable flow control and are
>> prepared to build from source, you could apply the following patch:
>>
>> diff -r f0dafaca6cd1 src/rabbit_reader.erl
>> --- a/src/rabbit_reader.erl Mon Mar 26 17:12:41 2012 +0100
>> +++ b/src/rabbit_reader.erl Wed Mar 28 13:40:03 2012 +0100
>> @@ -360,7 +360,7 @@
>>
>> control_throttle(State = #v1{connection_state = CS,
>> conserve_memory = Mem}) ->
>> - case {CS, Mem orelse credit_flow:blocked()} of
>> + case {CS, Mem} of
>> {running, true} -> State#v1{connection_state = blocking};
>> {blocking, false} -> State#v1{connection_state = running};
>> {blocked, false} -> ok = rabbit_heartbeat:resume_monitor(
>>
>> But I would like to understand more about what you're seeing.
>>
>> Cheers, Simon
>>
>

-- 
Simon MacMullen
RabbitMQ, VMware