<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 10pt;
font-family:Tahoma
}
--></style></head>
<body class='hmmessage'><div dir='ltr'>
Hello Steve,<br>thanks a lot for answering, I really hope we can solve the issue. First of all let's state that I am using RabbitMQ 2.8.1 (and the 2.8.1 java client)<br><br>Now, I tried the AtomicInteger thing because, drilling down into the cause of the exception, I've found this comment in your source code:<br>Class: com.rabbitmq.client.impl.ChannelManager<br>Method: releaseChannelNumber(ChannelN channel)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; // Warning, here be dragons. Not great big ones, but little baby ones<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; // which will nibble on your toes and occasionally trip you up when<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; // you least expect it. (Pixies? HP2)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; // Basically, there's a race that can end us up here. It almost never<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; // happens, but it's easier to repair it when it does than prevent it<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; // from happening in the first place.<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; // If we end up doing a Channel.close in one thread and a Channel.open<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; // with the same channel number in another, the two can overlap in such<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; // a way as to cause disconnectChannel on the old channel to try to<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; // remove the new one. Ideally we would fix this race at the source,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; // but it's much easier to just catch it here.<br><br>This is exactly what I believe is happening.<br><br>Anyway here are the additional information:<br>this is the code that uses the channel (called mainChannel)<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; mainChannel.addShutdownListener(this);<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; mainChannel.basicQos(1);<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; mainChannel.queueDeclare(endpoint.getQueueName(), endpoint.isDurable(),<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; !EXCLUSIVE, !AUTO_DELETE, args);<br><br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; // Ensure Exchange<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; mainChannel.exchangeDeclare(endpoint.getExchangeName(),<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; endpoint.getExchangeTypeName(), endpoint.isDurable(),<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; !EXCLUSIVE, !AUTO_DELETE, args);<br><br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; // Bind Queue-Exchange<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; mainChannel.queueBind(endpoint.getQueueName(),<br>&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; endpoint.getExchangeName(), endpoint.getRoutingKey());<br><br><br>This is the complete stacktrace:<br>2012-03-28 15:05:46,057 DEBUG [main] c.b.g.r.u.RabbitLinker [RabbitLinker.java:217] Error while building channel, retrying.<br>com.rabbitmq.client.ShutdownSignalException : connection error; reason: {#method&lt;connection.close&gt;(reply-code=503, reply-text=COMMAND_INVALID - second 'channel.open' seen, class-id=20, method-id=10), null, ""}<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at com.rabbitmq.client.impl.ChannelN.waitForConfirms( ChannelN.java:182)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at com.rabbitmq.client.impl.ChannelN.waitForConfirms( ChannelN.java:170)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at myapplication.rabbitmq.util.RabbitLinker.send(RabbitLinker.java:262)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at myapplication.rabbitmq.producer.AbstractRabbitProducer.send(AbstractRabbitProducer.java:69)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at myapplication.rabbitmq.producer.AbstractCamelProducer.process(AbstractCamelProducer.java:64)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:115)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:285)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:110)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:91)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:322)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:213)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcessor.java:45)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.processor.interceptor.DefaultChannel.process(DefaultChannel.java:303)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcessor.java:45)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.processor.UnitOfWorkProcessor.processAsync(UnitOfWorkProcessor.java:150)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:117)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:71)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:99)<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:86)2012-03-28 15:05:46,060 DEBUG [AMQP Connection 127.0.0.1:5672] c.b.g.r.u.RabbitFactory$ConnectionSaver [RabbitFactory.java:119] shutdownComplete:<br>com.rabbitmq.client.ShutdownSignalException : connection error; reason: {#method&lt;connection.close&gt;(reply-code=503, reply-text=COMMAND_INVALID - second 'channel.open' seen, class-id=20, method-id=10), null, ""}<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at com.rabbitmq.client.impl.AMQConnection.shutdown(AMQConnection.java:654) ~[amqp-client-2.8.1.jar:na]<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at com.rabbitmq.client.impl.AMQConnection.handleConnectionClose(AMQConnection.java:610) ~[amqp-client-2.8.1.jar:na]<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at com.rabbitmq.client.impl.AMQConnection.processControlCommand(AMQConnection.java:584) ~[amqp-client-2.8.1.jar:na]<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at com.rabbitmq.client.impl.AMQConnection$1.processAsync(AMQConnection.java:89) ~[amqp-client-2.8.1.jar:na]<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at com.rabbitmq.client.impl.AMQChannel.handleCompleteInboundCommand(AMQChannel.java:144) ~[amqp-client-2.8.1.jar:na]<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at com.rabbitmq.client.impl.AMQChannel.handleFrame( AMQChannel.java:91) ~[amqp-client-2.8.1.jar:na]<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:509) ~[amqp-client-2.8.1.jar:na]<br>2012-03-28 15:05:46,061 DEBUG [pool-18-thread-2] c.b.g.r.u.RabbitLinker [RabbitLinker.java:322] Internal RabbitMQ Exception while ACK-ing a message,<br><br><br>I hope you can help me figuring out what is happening.<br><br>Thanks<br><br><br><div><div id="SkyDrivePlaceholder"></div>&gt; Subject: Re: [rabbitmq-discuss] RMQ Java Client - ChannelN.waitForConfirm randomly throwing exceptions<br>&gt; From: steve@rabbitmq.com<br>&gt; Date: Wed, 28 Mar 2012 18:17:57 +0100<br>&gt; CC: rabbitmq-discuss@lists.rabbitmq.com<br>&gt; To: ghepardo_1982@hotmail.com<br>&gt; <br>&gt; Emanuele,<br>&gt; <br>&gt; There should be no need to use AtomicInteger to guarantee a new channel<br>&gt; number; createChannel should guarantee this for you.<br>&gt; <br>&gt; You have shown me the stack trace -- and I can see that your application<br>&gt; threads are issuing waitForConfirms().<br>&gt; <br>&gt; Can you tell me a little more about how your threads use the Channel?<br>&gt; I'd like to know the state of the channel the thread is using, at the<br>&gt; time of the call. What happened on that channel immediately before the<br>&gt; waitForConfirms() call?<br>&gt; <br>&gt; Perhaps some more snippets of the thread code would be useful, and a<br>&gt; little more of the stack trace would help.<br>&gt; <br>&gt; Finally, was there anything in the rabbit log around this time?<br>&gt; <br>&gt; Steve Powell  (a happy bunny)<br>&gt; ----------some more definitions from the SPD----------<br>&gt; chinchilla (n.) Cooling device for the lower jaw.<br>&gt; socialcast (n.) Someone to whom everyone is speaking but nobody likes.<br>&gt; literacy (n.) A textually transmitted disease usually contracted in childhood.<br>&gt; <br>&gt; On 28 Mar 2012, at 17:29, Emanuele Gheradini wrote:<br>&gt; <br>&gt; &gt; <br>&gt; &gt; I want also to add that what I understood is:<br>&gt; &gt; <br>&gt; &gt; for some reasons RabbitMQ java client gives the same channel (or channel<br>&gt; &gt; number) to different threads, then these threads will perform different<br>&gt; &gt; operations on that channel which will be closed by RabbitMQjavaclient, thus<br>&gt; &gt; resulting in the crash of the waitForConfirm method, at line 182 (here is<br>&gt; &gt; the snipped):<br>&gt; &gt; <br>&gt; &gt; 181: if (getCloseReason() != null) {<br>&gt; &gt; 182:                  throw Utility.fixStackTrace(getCloseReason());<br>&gt; &gt; 183:               }<br>&gt; &gt; <br>&gt; &gt; <br>&gt; &gt; I tried getting channels from rabbit in this way:<br>&gt; &gt; <br>&gt; &gt; AtomicInteger chanNumber = new AtomicInteger(0);<br>&gt; &gt; <br>&gt; &gt; connection.createChannel(chanNumber.getAndIncrement());<br>&gt; &gt; <br>&gt; &gt; to be sure that each thread gets a channel with a different channelNumber,<br>&gt; &gt; but this does not work either...<br>&gt; &gt; <br>&gt; &gt; <br>&gt; &gt; I hope someone can give me a suggestion.<br>&gt; &gt; <br>&gt; &gt; Thanks.<br>&gt; &gt; -- <br>&gt; &gt; View this message in context: http://old.nabble.com/RMQ-Java-Client---ChannelN.waitForConfirm-randomly-throwing-exceptions-tp33544834p33544845.html<br>&gt; &gt; Sent from the RabbitMQ mailing list archive at Nabble.com.<br>&gt; &gt; <br>&gt; &gt; _______________________________________________<br>&gt; &gt; rabbitmq-discuss mailing list<br>&gt; &gt; rabbitmq-discuss@lists.rabbitmq.com<br>&gt; &gt; https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss<br>&gt; <br></div>                                               </div></body>
</html>