[rabbitmq-discuss] RabbitMQ server performance issue (erl.exe)

[Simon MacMullen]

I can't run code that is incomplete :-) but some things stand out:

You are declaring and binding queues on every delivery, and then 
deleting them shortly after as well. On a system where a lot's going on 
in parallel this might lead to lots of retried transactions, which could 
lead to high CPU use. It's not clear when call() gets called but if that 
is in a tight loop then you could be declaring and immediately deleting 
queues constantly.

You said in your original message that there would only ever be 100 
queues - if that is the case then can they just be created once? Or use 
  queue leases (http://www.rabbitmq.com/extensions.html#queue-leases) to 
make sure they get tidied up after some period of inactivity?

You might want to connect your application through the Tracer 
(http://www.rabbitmq.com/api-guide.html#tracer) to see what is happening 
at the protocol level.

Cheers, Simon

On 27/01/12 08:52, Yogesh Ketkar wrote:
> Hello Simon,
> This is how code looks like
> main(String[] argv) {
>      // Read messages from MainQueue
>      Channel ch = c.createChannel();
>      ch.basicConsume("MainQueue", false, "", new
> MainQueueConsumer(ch));
>
>      // Read messages from dynamically declared queues
>      ExecutorService es = Executors.newFixedThreadPool(tpSize);
>      Collection<QueueProcessor>  vAppProcessorCollection = new
> ArrayList<QueueProcessor>();
>      while(true) {
>          // Get the queues currently present using
>          // http://<rabbitmq-server>:55672/api/queues/%2f/
>          List<String>  qs = AMQPUtil.getvAppQueues();
>          vAppProcessorCollection.clear();
>          for(String q : qs) {
>              QueueProcessor p = new QueueProcessor(c, q);
>              vAppProcessorCollection.add(p);
>          }
>          es.invokeAll(vAppProcessorCollection);
>      }
> }
> MainQueueConsumer, in handleDelivery does something like
> public void handleDelivery() {
>      // Get Queue name from payload
>      c.queueDeclare(queueName, true, false, false,
> getQueueProperties());
>      c.queueBind(queueName, exchange, routingKey);
>      c.basicPublish(exchange, routingKey, props, body);
>      ch.basicAck(envelope.getDeliveryTag(), false);
> }
>
> And QueueProcessor does this in call() method
> public Boolean call() {
>      try {
>          Channel ch = c.createChannel();
>          while(true) {
>              GetResponse r = ch.basicGet(queue, false);
>              if(r != null) {
>                  // process message
>                  ch.basicAck(r.getEnvelope().getDeliveryTag(), false);
>              } else break;
>          }
>          Util.queueDelete(ch, queue, true, true);
>          if(ch.isOpen()) ch.close();
>      } catch(Throwable th) {
>          BaseUtil.logException(logger, th);
>      }
> }
>
> Just couple of things
> - If I don't delete the queue in call() method, things are much better
>    and I am even able to 50000 messages on MainQueue without much
> issues
> - Ideally I would like to get rid of while(true) { ... basicGet ... }.
>    Is there any pattern where I can do
>    basicConsume() on the dynamically generated queue and
>    remove the consumer and the queue when number of message on it are 0
>
> regards, Yogesh
>
> On Jan 25, 7:23 pm, Simon MacMullen<si... at rabbitmq.com>  wrote:
>> On 25/01/12 08:30, Yogesh Ketkar wrote:
>>
>>> Any clues how to debug this?
>>
>> That sounds odd. 10k messages should be nothing.
>>
>> Can you share your code somehow so I can try to replicate this?
>>
>> Cheers, Simon
>>
>> --
>> Simon MacMullen
>> RabbitMQ, VMware
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss


-- 
Simon MacMullen
RabbitMQ, VMware