[rabbitmq-discuss] multiple message listeners on a single reply queue

[Gary Russell]

You cannot share reply queues across instances; unlike JMS, AMQP has no
notion of a message selector to choose messages from a queue, so each
instance needs its own reply queue.

When the reply goes to the wrong instance, the originating instance thread
will be suspended for (default) 5 seconds (modifiable by setting
replyTimeout).

You can increase throughput by increasing the concurrency on the listener
containers.

The upcoming 1.3 release of spring-AMQP allows the concurrency to vary on
demand; with 1.2.x you have to stop/start the container to change the
concurrency.


On Thu, Feb 6, 2014 at 1:07 PM, tgv amni <tgvt53 at gmail.com> wrote:

> Hi
>
> We are beginners in rabbitmq. We use rabbitmq 3.2.2. We use spring amqp.
>
>
> I recently observed an issue with multiple listeners on the same reply
> queue when performance testing
>
> our application.
>
>
>
> Here is the test scenario:
>
> 50 Concurrent users post a request to Service F to create an entity A. The
> message contains a user id and a timestamp. Each concurrent user posts
> roughly two messages per second.
>
>
>
> F publishes this message to a non-durable queue via a topic exchange
> (using convertSendAndReceive) and waits for the response on a fixed reply
> queue ReplyQ1.
>
>
>
> Service A picks up this message and creates this entity (basically an
> entry in a table) after
>
> validating that the user id wihin that message is valid.
>
>
>
> For the validation, Service A sends this message to another non-durable
> queue via a topic exchange (convertSendAndReceive) and waits for a response
> on
>
> another fixed reply queue, ReplyQ2.
>
>
>
> A validating service V picks up this user id, validates it (basically a
> select from a db table) and then sends the response back to ReplyQ2.
>
> Service A picks this up, based on the message, creates the entity and then
> sends response to ReplyQ1.
>
>
> All messages are non-persistent. No concurrency or prefetch settings
> configured on the listeners.
>
>
>
> *CreateEntity A Request -> Service F -> Q1 -> Service A -> Q2 - Service V*
>
> *Service V-> ReplyQ2 -> Service A (creates entity A if user valid) ->
> ReplyQ1 -> Service F*
>
>
>
> F, A and V all live within the same tomcat instance. When a second tomcat
> instance was started with service A and V, I started to notice warning
> messages in tomcat logs:
>
>
>
> *WARN : org.springframework.amqp.rabbit.core.RabbitTemplate - Reply
> received after timeout for b3fd4f57-0403-4704-8f16-8c3055ca0cd9*
>
>
>
> It looked like messages were not picked up by the correct listener from
> the validation response queues. Rabbitmq logs did not show any errors. From
> the management console i saw 40-60 messages always staying in ready state
>  in Q1, with a few in unacked state.
>
>
>
> Response times were very high for this create Entity A request (10
> seconds) and only about 5 transactions were processed per second. Tests
> were run in AWS on m1.large instances.
>
> High response time was not caused by network latency. Network latency was
> really small.
>
>
>
> On testing with separate reply queues for the user id validation response,
>
> (ReplyQ21 and ReplyQ22 instead of ReplyQ2), response times improved (1
> second) and more transactions (roughly 70) were processed per second.
>
>
>
> Is this a good design approach for this scenario? Or is there a better way
> to handle this ?
>
>
>
> Please let me know if you need any other information.
>
>
>
> Thanks.
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20140206/3f663fcc/attachment.html>