[rabbitmq-discuss] Publisher Confirms stop occurring when Consumer is present and queue is large

[Cameron Davison]

It has been a while since I wrote to the group. I was wondering if
this bug ever was fixed in a more recent release?

Thanks,
Cameron

On Fri, Nov 4, 2011 at 12:32 PM, Jerry Kuch <jerryk at vmware.com> wrote:
> Do watch for our upcoming "Cursing and Screaming as a Service" (CaSaaS)
> offering! :-)
>
> Jerry
>
> ----- Original Message -----
> From: "Matthew Sackman" <matthew at rabbitmq.com>
> To: rabbitmq-discuss at lists.rabbitmq.com
> Sent: Friday, November 4, 2011 8:38:22 AM
> Subject: Re: [rabbitmq-discuss] Publisher Confirms stop occurring when Consumer is present and queue is large
>
> Hi Cameron,
>
> On Wed, Nov 02, 2011 at 12:10:21AM -0500, Cameron Davison wrote:
>> Thank you for the reply. Do y'all have a bug tracker that I would be
>> able to watch so that I can know when y'all address this issue?
>
> We do have a bug tracker but alas it's not public because we all like to
> curse and scream at each other and generally feel that making it public
> would preclude us from doing that, which would be bad for morale.
>
>> Do you
>> know if this same thing would be even worse for a mirrored queue
>> rabbit mq cluster?
>
> I don't think this bug will affect mirrored queues in a worse way than
> non-mirrored queues...
>
>> I am seeing a lot of degradation in write
>> throughput while reading when doing mirrored queue cluster. All I
>> really want is high availability such that if one server crashed the
>> slave in the cluster will become the master and allow for continued
>> throughput. Is this the correct way to that?
>
> Yes - you are doing things the right way. Mirrored queues are much
> slower than non-mirrored queues due to the additional work that has to
> be done. You should probably expect to see about a ten-fold decrease in
> performance.
>
> http://old.nabble.com/Mirror-queues-and-poor-write-performance-td32727693.html
> reports that I can get a little over 2kHz on a single mirrored queue
> with one consumer keeping the queue empty.
>
> But the bug that you've identified will certainly affect mirrored queues
> as well as non-mirrored queues.
>
> One thing you could try is on the consumer:
> 1. Try a fairly healthy qos prefetch size N (eg N=100)
> 2. Only ack every N/2 (eg 50) msgs but turn on the "multiple" flag in
> the ack.
>
> The effect of these changes will be to reduce the consumer-related
> messages that the queue has to deal with, and so will hopefully allow
> the queue to process the publishes faster (and issue the confirms).
>
> If you're able to try these changes, I'd be interested in what
> improvements (if any) they achieve for you.
>
> Best wishes,
>
> Matthew
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss