[rabbitmq-discuss] [PATCH] Suggestions for improving the RabbitMQ Erlang Client

Wed Oct 9 14:26:37 BST 2013

Replying inline

On Tue, Oct 8, 2013 at 4:03 PM, Simon MacMullen <simon at rabbitmq.com> wrote:

>
> You are correct that the synchronous writing hurts performance - but it's
> not quite enough to just switch to async calls. The test suite starts
> failing, for a start :-)
>
> So there is a bit of work needed here to ensure that the messages to the
> writer get flushed before the socket gets closed. I've started work on
> this, and it should end up in 3.2.0.
>
>
Ah, I was afraid things might not be so obviously easy. I am glad you
sorted out the more specific parts :) Thanks for the fixes.

> <snip re channels manager>
>
> This is reasonable - but of course it's a more substantial change,
> unlikely to happen particularly soon.
>
>
If we are to do that, it is better to re-imagine the whole way the stack
works I think. There are other parts of the Erlang client that could do
with less misery. I fail to see the need for supervisor2 and gen_server2 in
the code at all for instance.

>
>  The writer process inside RabbitMQ is something I wanted to optimize as
>> well, so I tried changing two things in it. First, I avoided a
>> recomputation of `iolist_size/1` on every received message. And I just
>> dropped keeping a small 1414 byte packet size around. The rationale is
>> this: either we will be running at a high pace in the writer code and
>> then easily exceed the 1414 byte packet size. In this case, bumping it
>> to 64 kilobytes and making few calls to the underlying port seems to be
>> the right thing to do.
>>
>
> Hmm. I wasn't actually able to measure a discernable performance
> difference with these changes to the writer. Were you?
>
>
Keep it as is for now. Changing this messes with latency, potentially. My
worry is that you are pushing out small packets where a larger packet might
be equally easy to push. This then means more work on the line between the
client and the server. Keeping the outgoing queue full will prove to be a
problem though with the current setup. We can't guarantee that all
processes interested in pushing messages will be run before the process
doing the actual sending is.

And I did not manage to measure anything significantly better inside the
Erlang VM with this change. Chances are you may see less kernel-processing
though - but I did not measure that either. So in the interest of keeping
it stable, I propose not changing it.

-- 
J.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20131009/258c05e2/attachment.htm>