<div dir="ltr">Steve,<div><br></div><div>Thanks very much for your response. I was able to re-implement the while loop without having Tomcat bug-out upon server stop. I achieved this by catching the the InterruptedException in a Thread:</div>
<div><br></div><div><div>while (!Thread.currentThread().isInterrupted()) {</div><div><br></div><div> ...</div><div><br></div><div><div> channel.basicQos(1);</div><div><br></div><div> QueueingConsumer consumer = new QueueingConsumer(channel);</div>
<div> channel.basicConsume(ReferenceData.CONSUME_QUEUE_NAME, false, consumer)</div></div><div><br></div><div> try {</div><div><br></div><div> QueueingConsumer.Delivery delivery = consumer.nextDelivery();</div>
<div> ...</div><div> channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);<br></div><div><br></div><div> } catch (InterruptedException ex) {</div><div> Thread.currentThread().interrupt();</div>
<div> break;</div><div> }</div></div><div><br></div><div>}</div><div><br></div><div>I find your comments about the polling interesting. For now, though, my requirements would seem to favor as little network latency as possible, so I'm going to go with the original push/round robin model.</div>
<div><br></div><div>I am more confident in deploying the consumers and producers as web applications, and your answer has helped to confirm that. I am finding benefit in manageability and portability and my ultimate goal, like you said, is a sophisticated web-based control mechanism. I want to have real-time control of my clients, and I want it to be user friendly. I've started looking into JMX and MBeans to achieve such a solution.</div>
<div><br></div><div><i><span style="font-family:arial,sans-serif;font-size:13px">If you impose a prefetch</span><br style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,sans-serif;font-size:13px">limit of 1 for each client you run, then the rabbit server will dispatch</span><br style="font-family:arial,sans-serif;font-size:13px">
<span style="font-family:arial,sans-serif;font-size:13px">messages to clients that have no unacknowledged messages (by the way,</span><br style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,sans-serif;font-size:13px">you should explicitly acknowledge messages when using non-zero</span><br style="font-family:arial,sans-serif;font-size:13px">
<span style="font-family:arial,sans-serif;font-size:13px">basicQos()).</span></i><br></div><div><i><span style="font-family:arial,sans-serif;font-size:13px"><br></span></i></div><div><font face="arial, sans-serif">Regarding your statement above, I had not taken that into consideration. I've coded my clients to only acknowledge upon successful processing of the delivered message. If one of our many web services or datastore connections goes down, then unacknowledged messages will start piling up quickly, and the messaging and queuing pipeline will essentially come to a halt. I would expect this to be the correct behavior, however, as I don't want to throw out messages until someone discovers that a service is down.</font></div>
<div><font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif">As I will be using QueuingConsumer, I'm now wondering <i>how </i>I should determine <i>what </i>to set my prefetch counts to. Do you have any further advice on this? I'm still a little confused there.</font></div>
<div><font face="arial, sans-serif"><br></font></div><div style><font face="arial, sans-serif">Thanks again,</font></div><div style><font face="arial, sans-serif"><br>Kevin Behr</font></div></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Mon, Feb 11, 2013 at 6:33 AM, Steve Powell <span dir="ltr"><<a href="mailto:steve@rabbitmq.com" target="_blank">steve@rabbitmq.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Kevin,<br>
<br>
Sorry for the delayed response.<br>
<br>
First, an observation about your original set-up:<br>
<div class="im"><br>
> while (true) {<br>
> QueueingConsumer.Delivery delivery = consumer.nextDelivery();<br>
> String message = new String(delivery.getBody());<br>
> System.out.println(" [x] Received '" + message + "'");<br>
> }<br>
<br>
<br>
</div>I assume that there are no Qos (prefetch) limits imposed on the channels<br>
these consumers use, and also I note that you are using the<br>
QueueingConsumer. I think this means that all of the consumers are able<br>
to receive the messages all the time, and so all messages will get<br>
distributed in round-robin fashion, until there are no messages left to<br>
deliver.<br>
<br>
In your 'workaround' solution, you are indeed pulling, not being pushed,<br>
and here all of the consumers will get a look-in: there is likely to be<br>
balancing, but not round-robin. This is based instead upon consumer<br>
request, so if one of them gets bogged down in work (initiated by the<br>
previous message, for example) then the other consumers can poll and<br>
take the next message, and, crucially, other messages won't be taken by<br>
this consumer while it is busy. Although this isn't 'round-robin'<br>
servicing, it is arguably better: the messages are not 'blindly' sent to<br>
consumers that are busy.<br>
<br>
The drawbacks are, as you note, an overhead of polling in the consumers,<br>
and a larger latency on the network -- each get involves a round-trip to<br>
the server, even when there is nothing to get.<br>
<br>
Now to your questions:<br>
<br>
> • Is it a bad idea to deploy consumers as Tomcat web applications? Are<br>
<div class="im">> there any advantages or disadvantages as compared to the alternative of<br>
> running system level Java clients?<br>
<br>
</div>I see no reason why, apart from the overhead implied by running a full<br>
web application, there should be any particular disadvantages of running<br>
a Tomcat web application that does messaging as its main work. In fact,<br>
I suspect it is easier to control (start/stop) than using a<br>
free-standing Java client: you can potentially build a sophisticated<br>
web-based control mechanism. However, I'm not best placed to answer<br>
this.<br>
<br>
> • In the case of work queues, am I better off retrieving messages that<br>
<div class="im">> are pushed down to my clients, and setting a prefetchCount<br>
> (channel.basicQos(1)) to ensure that I only receive one message at a<br>
> time?<br>
<br>
</div>This is the crux of the matter, as I see it. If you impose a prefetch<br>
limit of 1 for each client you run, then the rabbit server will dispatch<br>
messages to clients that have no unacknowledged messages (by the way,<br>
you should explicitly acknowledge messages when using non-zero<br>
basicQos()). Under these circumstances, you can achieve similar<br>
balancing as in the polling case, but without the polling. It will *not*<br>
be strictly round-robin after the first 'round' because the first<br>
consumer need not have acknowledged the previous message yet, and<br>
messages will be delivered to whatever consumers are ready for them --<br>
albeit in round-robin priority.<br>
<br>
The tutorial pages describe this pretty well (see [1]).<br>
<br>
There are two points to note:<br>
<br>
If you are using QueueingConsumer, then each message will be placed in a<br>
client Java queue, and nextDelivery() then unblocks with the message<br>
when it arrives. With explicit acknowledgements this means that one<br>
message at a time is transferred to the Java queue. You could increase<br>
the prefetchCount to 2, say, and then up to two messages would be placed<br>
in the queue but you would still process them serially through the<br>
nextDelivery() call. When one consumer gets two messages, then no other<br>
consumer will get these -- so if a message can take a long time before<br>
it is acknowledged, the message behind it can get delayed, even though<br>
there may be other consumers which are idle. Other messages can<br>
'overtake' and get processed before it.<br>
<br>
If you want to avoid the QueueingConsumer internal Java queue, you could<br>
write your own Consumer implementation (base it on DefaultConsumer) but<br>
you should be aware that the handleDelivery() methods are driven<br>
asynchronously, but serially. You shouldn't perform long pieces of work<br>
on these threads. It would be wise to pass the message to some<br>
serialised worker thread, especially if you have a prefetchCount larger<br>
than one. This will probably then require a queue of your own anyway, so<br>
the advantages to you are slim.<br>
<br>
I hope this helps. Please let us know how you get on.<br>
<br>
Steve Powell<br>
<br>
[1] <a href="http://www.rabbitmq.com/tutorials/tutorial-two-java.html" target="_blank">http://www.rabbitmq.com/tutorials/tutorial-two-java.html</a><br>
<br>
[M: <a href="tel:%2B44-7815-838-558" value="+447815838558">+44-7815-838-558</a>; H:<a href="tel:%2B44-1962-775-598" value="+441962775598">+44-1962-775-598</a>]<br>
SpringSource (a division of VMware), Virgo, RabbitMQ.<br>
-----------------------------------------------------------------------<br>
Good design:<br>
is innovative, useful, aesthetic;<br>
is understandable, unobtrusive, honest;<br>
is long-lasting, thorough, environmentally friendly;<br>
and is as little design as possible.<br>
Copyright Dieter Rams, amended March 2003; October 2009; and August 2012<br>
<div class="im"><br>
On 31 Jan 2013, at 01:33, Kevin Behr <<a href="mailto:behrk2@gmail.com">behrk2@gmail.com</a>> wrote:<br>
> Hello everyone,<br>
><br>
> I have five RabbitMQ consumer clients that are written and deployed as Tomcat web applications. My original intention was to consume messages in the typical fashion:<br>
</div>…(elided)<br>
<div class="im">> I would appreciate any advice or suggestions.<br>
><br>
> Thanks!<br>
</div>> _______________________________________________<br>
> rabbitmq-discuss mailing list<br>
> <a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>
> <a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
</blockquote></div><br></div>