[rabbitmq-discuss] Ensuring low latency for publishers

Mon Feb 27 10:42:00 GMT 2012

On 24/02/12 18:07, Eric wrote:
> In reality, the throttling case is extremely unlikely anyway, right?
> You basically have to create a case where publishers are outpacing
> consumers and/or the speed of disk writes on the broker.  So you'd
> have to publish something like 100mb/s with little or no consumption,
> and our application is roughly 5mb/minute.

With small messages in fact the first thing you run into is being 
CPU-bound making routing decisions. But the general principle holds, yes.

> In conclusion, I finally did what I should have done in the first
> place.  I mocked up the threading scenario we're likely to see in
> production and benchmarked it against Java's LinkedBlockingQueue and
> ArrayBlockingQueue.  Compared to each producer thread having its own
> linked list (the completely uncontended scenario) the blocking queues
> introduced virtually no additional waiting - a microsecond or two at
> most.  That's the case even at rates 50x or more than we'd see in
> production.  So blocking queue it is, and that's a comfortingly simple
> solution.
>
> Thanks for all the help and apologies for under-researching what I was
> doing.

Ah, good.

I had a quick look at implementing non-blocking publish on Java on 
Friday. Unfortunately it's not possible to do this without NIO on Java, 
so a prerequisite becomes "port the Java client to NIO", which... makes 
it less likely to happen I'm afraid.

Cheers, Simon

-- 
Simon MacMullen
RabbitMQ, VMware