[rabbitmq-discuss] Ensuring low latency for publishers

Fri Feb 24 18:07:38 GMT 2012

> Sure, I'm trying to gauge how widespread such a demand is. It looks like
> we could plausibly do something in this direction.

Seems like it's not very widespread.  That's fine - I'm probably
pushing towards a less common use case.  We can deal with somewhat
unreliable messaging but we'd really like a broker.  0MQ is great for
unreliable messaging but it's brokerless, and we'd rather not write
our own broker.  RabbitMQ has a great broker, it's just sort of...
narcissistic.  :)

> If it's any consolation the flow control stuff that's coming in 2.8.0
> will make it much harder to make the memory alarm go off, so blocking
> for seconds should be much less likely.

In reality, the throttling case is extremely unlikely anyway, right?
You basically have to create a case where publishers are outpacing
consumers and/or the speed of disk writes on the broker.  So you'd
have to publish something like 100mb/s with little or no consumption,
and our application is roughly 5mb/minute.  So we'd probably never
throttle.  It's just that having some kind of guarantee is nice from
an operational standpoint.

In conclusion, I finally did what I should have done in the first
place.  I mocked up the threading scenario we're likely to see in
production and benchmarked it against Java's LinkedBlockingQueue and
ArrayBlockingQueue.  Compared to each producer thread having its own
linked list (the completely uncontended scenario) the blocking queues
introduced virtually no additional waiting - a microsecond or two at
most.  That's the case even at rates 50x or more than we'd see in
production.  So blocking queue it is, and that's a comfortingly simple
solution.

Thanks for all the help and apologies for under-researching what I was
doing.

 - Eric

On Feb 24, 3:20 am, Simon MacMullen <si... at rabbitmq.com> wrote:
> On 23/02/12 20:27, Eric wrote:
>
> > Well... I would find it useful.  And the person in the thread I linked
> > to would have found it useful as well.
>
> Sure, I'm trying to gauge how widespread such a demand is. It looks like
> we could plausibly do something in this direction.
>
> > It's pretty tough to go into a production environment with the
> > statement "Publishing should be fast unless the broker is throttling.
> > If the broker is throttling we won't be able to tell, we don't have a
> > good way of knowing beforehand, and we don't know how long the threads
> > will be blocked".  It's useful in any case where an application would
> > rather not publish the message rather than waiting to publish it
> > indefinitely.
>
> Yeah, the problem is that we've historically optimised for Never
> Throwing Away Messages Ever Ever...
>
> > The net result is that a struggling broker will cause a chain
> > reaction; publishers will block indefinitely, which may bring the
> > publishing application to a halt.  In that case I'd like to abandon
> > the publishing process, but it's quite impossible at that point - it's
> > too late, the broker is already throttling, and there's no timeout.
> > The thread could block for seconds, or more.  I'm concerned about not
> > throwing the message away, but I'm more concerned about being able to
> > isolate services from RabbitMQ broker events.
>
> If it's any consolation the flow control stuff that's coming in 2.8.0
> will make it much harder to make the memory alarm go off, so blocking
> for seconds should be much less likely.
>
> > One more question along this line.  The flow control document says "By
> > default, when the RabbitMQ server uses above 40% of the installed RAM,
> > it raises a memory alarm... Connection heartbeat monitoring gets
> > disabled too."  If I set up a connection heartbeat on the shared
> > connection, will the heartbeat detect a broker alarm that exceeds the
> > interval and disconnect?  A disconnect would terminate the publisher
> > channels and free things up.
>
> No - the whole point of disabling heartbeats is to make sure connections
> stay up when this happens...
>
> Cheers, Simon
>
> --
> Simon MacMullen
> RabbitMQ, VMware
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss