[rabbitmq-discuss] RabbitMQ periodic slowness

[Arun Rao]

I am using F5 as well but as 2-cluster with 2 nodes each.

>But in every test scenario under load we start to see the socket opened to
Rabbit on a write operation and no response for 20-30 seconds (with
>network tracing tools).

Is this no response on the client side? How does the network capture look
on the rabbitmq server side?

Thanks!


On Thu, Feb 6, 2014 at 7:34 PM, Ron Cordell <ron.cordell at gmail.com> wrote:

> Hello everyone!
>
> We are trying to understand RabbitMQ behavior that we see in a highly
> available cluster and I'm hoping someone here can shed some light on it.
>
> We have a 3-node cluster that is exposed to the application via an F5 load
> balancer by a virtual IP. The majority of the queues that are created are
> highly available queues in order to prevent message loss from happening in
> the event of a catastrophic failure of a node.
>
> The rabbit instances/nodes are 3.2.2 with Erlang 16B02.
>
> The application consists of a series of REST HTTP interfaces which place
> the incoming messages onto the rabbit queues.
>
> When we load test the application we are seeing a periodic delay in
> writing a message to Rabbit. The delay happens about once every few minutes
> and lasts up to 30 seconds.
>
> We have tested in several scenarios:
> 3 node load balanced cluster
> single node, non ha-policy applied machine
> on Windows Server 2008R2
> on Linux (Oracle Linux)
> on VMWare virtual machines on high speed SAN storage
> on physical machines with SSD
> direct connection with no LB to single node
>
> We haven't tested every combination in this list but we've tried to
> isolate I/O, operating system, machine characteristics, etc.
>
> But in every test scenario under load we start to see the socket opened to
> Rabbit on a write operation and no response for 20-30 seconds (with network
> tracing tools).
>
> Has anyone seen any behavior like this?
>
> Our concern is that under load some REST interfaces will show periodic
> slowness; we have SLAs of ~1 second on the interfaces.
>
> Thanks for any input!
>
> Cheers,
>
> Ron Cordell
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140206/c21834c4/attachment.html>