[rabbitmq-discuss] RabbitMQ experience

[Martin Sustrik]

Tony Garnock-Jones wrote:
> Martin Sustrik wrote:
>> Hm, interesting. I've never thought of that kind of metric. Possibly 
>> something like "if there's a peak lasting N seconds containing M 
>> messages there's a 50% chance that the max latency will exceed X 
>> microseconds". The obvious problem is that there are to many free 
>> variables in the formula.
> 
> Yeah. Hmm. Well, LD50 is measured IIRC along the lines of "LD50 is this 
> many mg/kg in mice", so there's a dependency on species, amount of 
> bodyweight, and amount of drug. Still doesn't help with our problem, of 
> course :-)

What can be done for example is to simulate some particularly terrible 
real-world market data peak and show that the solution is capable of 
handling it with latencies below N in 99.99% of cases.

Still, while vendors love percentile statistics, customers want hard 
limits forgetting that they won't run the system on hard realtime OS 
anyway... something more "straightforward" would do a better marketing job.

> Those charts you linked earlier were interesting. The 1/lambda chart had 
> a sharp corner where the network took over from the CPU as the 
> bottleneck. Perhaps the location (and sharpness?) of the corner is a 
> relevant measure?

That metric may be pretty interesting to understand how the system 
behaves, but you can easily make it look better by slowing down 
processing of the message content (linearly bound part of the algorithm) 
instead of speeding up processing of the message as a unit (constantly 
bound part of the algorithm).

You can make it unambiguous by mentioning 2 values: "for message sizes 
below 32 bytes the system hits its processing limit at 250 nanoseconds 
per message i.e. 4 million messages a second."

Martin