[rabbitmq-discuss] problem with ncreasing response times

Sun Sep 29 20:50:17 BST 2013

We use rabbitmq 3.1.5. This is installed on a red hat linux server (6.4),
the same server where our application is also running. The database
(postgresql) is on a separate server.

I ran a test where concurrent users post to a rest service (say R). This
service basically expects  two UUIDs (UUID_A and UUID_B), that identify two
database entities  A and B. Once the service is called with these two
values, it publishes to the default exchange. There are two queues bound to
this exchange. QueueA and QueueB. Two services,  Service A and Service B,
each listen on QueueA and QueueB respectively.  There are no other
consumers to these queues. Service A picks up the message and validates  UUID_A
exists in the database. Service B validates UUID_B exists. If they are both
valid, an entry is made to the db, mapping these two values.
I find as the test progresses, response  time of this overall post
increases.  A lot of the time is being spent in  Unsafe.park (boolean,long)
in sun.misc package.
Here is the call stack that shows classes where this being involved from.
It appears to be called from spring framework’s RabbitTemplate class.

A problem with increasing response times.

We use rabbitmq 3.1.5. This is installed on a red hat linux server (6.4),
the same server where our application is also running. The database
(postgresql) is on a separate server.

I ran a test where concurrent users post to a rest service (say R). This
service basically expects  two UUIDs (UUID_A and UUID_B), that identify two
database entities  A and B. Once the service is called with these two
values, it publishes to the default exchange. There are two queues bound to
this exchange. QueueA and QueueB. Two services,  Service A and Service B,
each listen on QueueA and QueueB respectively.  There are no other
consumers to these queues. Service A picks up the message and validates  UUID_A
exists in the database. Service B validates UUID_B exists. If they are both
valid, an entry is made to the db, mapping these two values.

I find as the test progresses, response  time of this overall post
increases.  A lot of the time is being spent in  Unsafe.park (boolean,long)
in sun.misc package. I am wondering why this is so…

Here is the call stack that shows classes where this being involved from.
It appears to be called from spring framework’s RabbitTemplate class.

Unsafe.park

-LockSupportparkNanos(Object,Long)

As the test progresses where the load increases from 50 users to 200 users
over a period of 20 mins, this response time increases from 50 ms to almost
2 seconds. 99% of the response time is spent in Unsafe.park.

Why is this so ? It looks like there is a wait for some resource lock. I am
not sure why this wait increases with time/ load.

We are using  rabbitMQ default configuration. Everyone is new to rabbitMQ,
so we haven’t changed any of the default parameters.

Also is there a plug-in to monitor cpu usage on the broker ?

Thanks

 As the test progresses where the load increases from 50 users to 200 users
over a period of 20 mins, this response time increases from 50 ms to almost
2 seconds. 99% of the response time is spent in Unsafe.park.

Why is this so ? It looks like there is a wait for some resource lock. I am
not sure why this wait increases with time/ load.

We are using  rabbitMQ default configuration. Everyone is new to rabbitMQ,
so we haven’t changed any of the default parameters.

Also is there a plug-in to monitor cpu usage on the broker ?

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130929/d82ea2b9/attachment.htm>