<div dir="ltr">Steve,<div><br></div><div>Thanks very much for your response. �I was able to re-implement the while loop without having Tomcat bug-out upon server stop. �I achieved this by catching the the InterruptedException in a Thread:</div>
<div><br></div><div><div>while (!Thread.currentThread().isInterrupted()) {</div><div><br></div><div>� � � � � � ...</div><div><br></div><div><div>� � � � � � channel.basicQos(1);</div><div><br></div><div>� � � � � � QueueingConsumer consumer = new QueueingConsumer(channel);</div>
<div>� � � � � � channel.basicConsume(ReferenceData.CONSUME_QUEUE_NAME, false, consumer)</div></div><div><br></div><div>� � � � � � � � try {</div><div><br></div><div>� � � � � � � � � � QueueingConsumer.Delivery delivery = consumer.nextDelivery();</div>
<div>� � � � � � � � � � ...</div><div>� � � � � � � � � � channel.basicAck(delivery.getEnvelope().getDeliveryTag(), false);<br></div><div><br></div><div>� � � � � � � � } catch (InterruptedException ex) {</div><div>� � � � � � � � � � Thread.currentThread().interrupt();</div>
<div>� � � � � � � � � � break;</div><div>� � � � � � � � }</div></div><div><br></div><div>}</div><div><br></div><div>I find your comments about the polling interesting. �For now, though, my requirements would seem to favor as little network latency as possible, so I'm going to go with the original push/round robin model.</div>
<div><br></div><div>I am more confident in deploying the consumers and producers as web applications, and your answer has helped to confirm that. �I am finding benefit in�manageability�and portability and my ultimate goal, like you said, is a sophisticated web-based control mechanism. �I want to have real-time control of my clients, and I want it to be user friendly. �I've started looking into JMX and MBeans to achieve such a solution.</div>
<div><br></div><div><i><span style="font-family:arial,sans-serif;font-size:13px">If you impose a prefetch</span><br style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,sans-serif;font-size:13px">limit of 1 for each client you run, then the rabbit server will dispatch</span><br style="font-family:arial,sans-serif;font-size:13px">
<span style="font-family:arial,sans-serif;font-size:13px">messages to clients that have no unacknowledged messages (by the way,</span><br style="font-family:arial,sans-serif;font-size:13px"><span style="font-family:arial,sans-serif;font-size:13px">you should explicitly acknowledge messages when using non-zero</span><br style="font-family:arial,sans-serif;font-size:13px">
<span style="font-family:arial,sans-serif;font-size:13px">basicQos()).</span></i><br></div><div><i><span style="font-family:arial,sans-serif;font-size:13px"><br></span></i></div><div><font face="arial, sans-serif">Regarding your statement above, I had not taken that into consideration. �I've coded my clients to �only acknowledge upon successful processing of the delivered message. �If one of our many web services or datastore connections goes down, then unacknowledged messages will start piling up quickly, and the messaging and queuing pipeline will essentially come to a halt. I would expect this to be the correct behavior, however, as I don't want to throw out messages until someone discovers that a service is down.</font></div>
<div><font face="arial, sans-serif"><br></font></div><div><font face="arial, sans-serif">As I will be using QueuingConsumer, I'm now wondering�<i>how�</i>I should determine�<i>what�</i>to set my prefetch counts to. �Do you have any further advice on this? �I'm still a little confused there.</font></div>
<div><font face="arial, sans-serif"><br></font></div><div style><font face="arial, sans-serif">Thanks again,</font></div><div style><font face="arial, sans-serif"><br>Kevin Behr</font></div></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Mon, Feb 11, 2013 at 6:33 AM, Steve Powell <span dir="ltr"><<a href="mailto:steve@rabbitmq.com" target="_blank">steve@rabbitmq.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Kevin,<br>
<br>
Sorry for the delayed response.<br>
<br>
First, an observation about your original set-up:<br>
<div class="im"><br>
> while (true) {<br>
> � � � QueueingConsumer.Delivery delivery = consumer.nextDelivery();<br>
> � � � String message = new String(delivery.getBody());<br>
> � � � System.out.println(" [x] Received '" + message + "'");<br>
> }<br>
<br>
<br>
</div>I assume that there are no Qos (prefetch) limits imposed on the channels<br>
these consumers use, and also I note that you are using the<br>
QueueingConsumer. I think this means that all of the consumers are able<br>
to receive the messages all the time, and so all messages will get<br>
distributed in round-robin fashion, until there are no messages left to<br>
deliver.<br>
<br>
In your 'workaround' solution, you are indeed pulling, not being pushed,<br>
and here all of the consumers will get a look-in: there is likely to be<br>
balancing, but not round-robin. This is based instead upon consumer<br>
request, so if one of them gets bogged down in work (initiated by the<br>
previous message, for example) then the other consumers can poll and<br>
take the next message, and, crucially, other messages won't be taken by<br>
this consumer while it is busy. Although this isn't 'round-robin'<br>
servicing, it is arguably better: the messages are not 'blindly' sent to<br>
consumers that are busy.<br>
<br>
The drawbacks are, as you note, an overhead of polling in the consumers,<br>
and a larger latency on the network -- each get involves a round-trip to<br>
the server, even when there is nothing to get.<br>
<br>
Now to your questions:<br>
<br>
> �� Is it a bad idea to deploy consumers as Tomcat web applications? Are<br>
<div class="im">> there any advantages or disadvantages as compared to the alternative of<br>
> running system level Java clients?<br>
<br>
</div>I see no reason why, apart from the overhead implied by running a full<br>
web application, there should be any particular disadvantages of running<br>
a Tomcat web application that does messaging as its main work. In fact,<br>
I suspect it is easier to control (start/stop) than using a<br>
free-standing Java client: you can potentially build a sophisticated<br>
web-based control mechanism. However, I'm not best placed to answer<br>
this.<br>
<br>
> �� In the case of work queues, am I better off retrieving messages that<br>
<div class="im">> are pushed down to my clients, and setting a prefetchCount<br>
> (channel.basicQos(1)) to ensure that I only receive one message at a<br>
> time?<br>
<br>
</div>This is the crux of the matter, as I see it. If you impose a prefetch<br>
limit of 1 for each client you run, then the rabbit server will dispatch<br>
messages to clients that have no unacknowledged messages (by the way,<br>
you should explicitly acknowledge messages when using non-zero<br>
basicQos()). Under these circumstances, you can achieve similar<br>
balancing as in the polling case, but without the polling. It will *not*<br>
be strictly round-robin after the first 'round' because the first<br>
consumer need not have acknowledged the previous message yet, and<br>
messages will be delivered to whatever consumers are ready for them --<br>
albeit in round-robin priority.<br>
<br>
The tutorial pages describe this pretty well (see [1]).<br>
<br>
There are two points to note:<br>
<br>
If you are using QueueingConsumer, then each message will be placed in a<br>
client Java queue, and nextDelivery() then unblocks with the message<br>
when it arrives. With explicit acknowledgements this means that one<br>
message at a time is transferred to the Java queue. You could increase<br>
the prefetchCount to 2, say, and then up to two messages would be placed<br>
in the queue but you would still process them serially through the<br>
nextDelivery() call. When one consumer gets two messages, then no other<br>
consumer will get these -- so if a message can take a long time before<br>
it is acknowledged, the message behind it can get delayed, even though<br>
there may be other consumers which are idle. Other messages can<br>
'overtake' and get processed before it.<br>
<br>
If you want to avoid the QueueingConsumer internal Java queue, you could<br>
write your own Consumer implementation (base it on DefaultConsumer) but<br>
you should be aware that the handleDelivery() methods are driven<br>
asynchronously, but serially. You shouldn't perform long pieces of work<br>
on these threads. It would be wise to pass the message to some<br>
serialised worker thread, especially if you have a prefetchCount larger<br>
than one. This will probably then require a queue of your own anyway, so<br>
the advantages to you are slim.<br>
<br>
I hope this helps. Please let us know how you get on.<br>
<br>
Steve Powell<br>
<br>
[1] <a href="http://www.rabbitmq.com/tutorials/tutorial-two-java.html" target="_blank">http://www.rabbitmq.com/tutorials/tutorial-two-java.html</a><br>
<br>
[M: <a href="tel:%2B44-7815-838-558" value="+447815838558">+44-7815-838-558</a>; H:<a href="tel:%2B44-1962-775-598" value="+441962775598">+44-1962-775-598</a>]<br>
SpringSource (a division of VMware), Virgo, RabbitMQ.<br>
-----------------------------------------------------------------------<br>
Good design:<br>
� �is innovative, useful, aesthetic;<br>
� �is understandable, unobtrusive, honest;<br>
� �is long-lasting, thorough, environmentally friendly;<br>
� �and is as little design as possible.<br>
Copyright Dieter Rams, amended March 2003; October 2009; and August 2012<br>
<div class="im"><br>
On 31 Jan 2013, at 01:33, Kevin Behr <<a href="mailto:behrk2@gmail.com">behrk2@gmail.com</a>> wrote:<br>
> Hello everyone,<br>
><br>
> I have five RabbitMQ consumer clients that are written and deployed as Tomcat web applications. �My original intention was to consume messages in the typical fashion:<br>
</div>�(elided)<br>
<div class="im">> I would appreciate any advice or suggestions.<br>
><br>
> Thanks!<br>
</div>> _______________________________________________<br>
> rabbitmq-discuss mailing list<br>
> <a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>
> <a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss" target="_blank">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>
</blockquote></div><br></div>