<html><head><base href="x-msg://58/"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Emanuele,<div><br></div><div><div>Bravo for trying AtomicInteger; however the comment you found is an old</div><div>one, and probably (I say probably) does not apply anymore. It is</div><div>extremely unlikely that a recently used channel number is reallocated</div><div>(unless you explicitly request one) and even in that case I think we</div><div>have the locking correct now so that this race cannot happen. In any</div><div>event, using AtomicInteger will only make things worse.</div><div><br></div><div>Incidentally, the code following the comment should detect and fix the</div><div>clash, so you shouldn't see it anyhow. (Famous last words.) Of course,</div><div>like all naive concurrent code, it does rather assume there are only</div><div>ever two threads involved, so could have a very rare bug -- but I doubt</div><div>it.</div><div><br></div><div>I'm grateful for the extra information; however, it doesn't appear to be</div><div>enough to follow what's happening.</div><div><br></div><div>It looks as though your application</div><div>myapplication.rabbitmq.util.RabbitLinker.send(RabbitLinker.java:262) is</div><div>calling waitForConfirms() on the channel. I don't see this in the code</div><div>outline you sent. Can you validate that confirmSelect is called on the</div><div>channel; that the correct channel is being used top issue</div><div>waitForConfirms() and that you issue this call before closing the</div><div>channel.</div><div><br></div><div>What do you do in the shutdownListener you register, for example?</div><div><br></div><div>If it is possible, a look at your application code would be helpful.</div><div><div><br></div><div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-variant: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div style="font-style: normal; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div style="font-family: Georgia; font-size: medium; font-weight: normal; font-style: normal; "><div><font class="Apple-style-span" size="3"><span class="Apple-style-span" style="font-size: 12px; ">Steve Powell </span><span class="Apple-style-span" style="font-size: 11px; ">(<i>a happy bunny</i>)</span></font></div></div><div><font class="Apple-style-span"><div style="font-weight: normal; font-style: normal; font-family: Helvetica; font-size: 12px; "><i><font class="Apple-style-span" face="Georgia" size="2"><span class="Apple-style-span" style="font-size: 10px; ">----------some more definitions from the SPD----------</span></font></i></div></font></div></div></div><span class="Apple-style-span" style="font-style: normal; font-weight: normal; font-family: Helvetica; font-size: 12px; "><font class="Apple-style-span" face="Georgia" size="2" style="font-style: normal; "><span class="Apple-style-span" style="font-size: 10px; "><b>chinchilla</b> (</span></font><font class="Apple-style-span" face="Georgia" size="2"><span class="Apple-style-span" style="font-size: 10px; "><i>n.</i>) Cooling device for the lower jaw.</span></font></span><div style="font-style: normal; font-family: Georgia; font-size: medium; font-weight: normal; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Georgia; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "></span><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-variant: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><span class="Apple-style-span" style="font-family: Georgia; font-weight: normal; font-style: normal; font-size: 10px; "><b>socialcast</b></span><span class="Apple-style-span" style="font-family: Georgia; font-weight: normal; font-style: normal; font-size: 10px; "> (</span><span class="Apple-style-span" style="font-family: Georgia; font-weight: normal; font-size: 10px; "><i>n.</i>)</span><span class="Apple-style-span" style="font-family: Georgia; font-weight: normal; font-style: normal; font-size: 10px; "> Someone to whom everyone is speaking but nobody likes.</span></span></div><div style="font-family: Georgia; font-size: medium; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-variant: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><span class="Apple-style-span" style="font-family: Georgia; font-style: normal; font-size: 10px; "><b>literacy</b></span><span class="Apple-style-span" style="font-family: Georgia; font-weight: normal; font-style: normal; font-size: 10px; "><span class="Apple-converted-space"> </span>(</span><span class="Apple-style-span" style="font-family: Georgia; font-weight: normal; font-size: 10px; "><i>n.</i></span><span class="Apple-style-span" style="font-family: Georgia; font-weight: normal; font-style: normal; font-size: 10px; ">) A textually transmitted disease usually contracted in childhood.</span></span></div></div></span></div>
</div>
<br><div><div>On 29 Mar 2012, at 09:43, Emanuele G wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Georgia; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div class="hmmessage" style="font-size: 10pt; font-family: Tahoma; "><div dir="ltr">Hello Steve,<br>thanks a lot for answering, I really hope we can solve the issue. First of all let's state that I am using RabbitMQ 2.8.1 (and the 2.8.1 java client)<br><br>Now, I tried the AtomicInteger thing because, drilling down into the cause of the exception, I've found this comment in your source code:<br>Class: com.rabbitmq.client.impl.ChannelManager<br>Method: releaseChannelNumber(ChannelN channel)<br> // Warning, here be dragons. Not great big ones, but little baby ones<br> // which will nibble on your toes and occasionally trip you up when<br> // you least expect it. (Pixies? HP2)<br> // Basically, there's a race that can end us up here. It almost never<br> // happens, but it's easier to repair it when it does than prevent it<br> // from happening in the first place.<br> // If we end up doing a Channel.close in one thread and a Channel.open<br> // with the same channel number in another, the two can overlap in such<br> // a way as to cause disconnectChannel on the old channel to try to<br> // remove the new one. Ideally we would fix this race at the source,<br> // but it's much easier to just catch it here.<br><br>This is exactly what I believe is happening.<br><br>Anyway here are the additional information:<br>this is the code that uses the channel (called mainChannel)<br> mainChannel.addShutdownListener(this);<br> mainChannel.basicQos(1);<br> mainChannel.queueDeclare(endpoint.getQueueName(), endpoint.isDurable(),<br> !EXCLUSIVE, !AUTO_DELETE, args);<br><br> // Ensure Exchange<br> mainChannel.exchangeDeclare(endpoint.getExchangeName(),<br> endpoint.getExchangeTypeName(), endpoint.isDurable(),<br> !EXCLUSIVE, !AUTO_DELETE, args);<br><br> // Bind Queue-Exchange<br> mainChannel.queueBind(endpoint.getQueueName(),<br> endpoint.getExchangeName(), endpoint.getRoutingKey());<br><br><br>This is the complete stacktrace:<br>2012-03-28 15:05:46,057 DEBUG [main] c.b.g.r.u.RabbitLinker [RabbitLinker.java:217] Error while building channel, retrying.<br>com.rabbitmq.client.ShutdownSignalException : connection error; reason: {#method<connection.close>(reply-code=503, reply-text=COMMAND_INVALID - second 'channel.open' seen, class-id=20, method-id=10), null, ""}<br> at com.rabbitmq.client.impl.ChannelN.waitForConfirms( ChannelN.java:182)<br> at com.rabbitmq.client.impl.ChannelN.waitForConfirms( ChannelN.java:170)<br> at myapplication.rabbitmq.util.RabbitLinker.send(RabbitLinker.java:262)<br> at myapplication.rabbitmq.producer.AbstractRabbitProducer.send(AbstractRabbitProducer.java:69)<br> at myapplication.rabbitmq.producer.AbstractCamelProducer.process(AbstractCamelProducer.java:64)<br> at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)<br> at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)<br> at org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:115)<br> at org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:285)<br> at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:110)<br> at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)<br> at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)<br> at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)<br> at org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:91)<br> at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)<br> at org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:322)<br> at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:213)<br> at org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcessor.java:45)<br> at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)<br> at org.apache.camel.processor.interceptor.DefaultChannel.process(DefaultChannel.java:303)<br> at org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcessor.java:45)<br> at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)<br> at org.apache.camel.processor.UnitOfWorkProcessor.processAsync(UnitOfWorkProcessor.java:150)<br> at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:117)<br> at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)<br> at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)<br> at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)<br> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:71)<br> at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:99)<br> at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:86)2012-03-28 15:05:46,060 DEBUG [AMQP Connection 127.0.0.1:5672] c.b.g.r.u.RabbitFactory$ConnectionSaver [RabbitFactory.java:119] shutdownComplete:<br>com.rabbitmq.client.ShutdownSignalException : connection error; reason: {#method<connection.close>(reply-code=503, reply-text=COMMAND_INVALID - second 'channel.open' seen, class-id=20, method-id=10), null, ""}<br> at com.rabbitmq.client.impl.AMQConnection.shutdown(AMQConnection.java:654) ~[amqp-client-2.8.1.jar:na]<br> at com.rabbitmq.client.impl.AMQConnection.handleConnectionClose(AMQConnection.java:610) ~[amqp-client-2.8.1.jar:na]<br> at com.rabbitmq.client.impl.AMQConnection.processControlCommand(AMQConnection.java:584) ~[amqp-client-2.8.1.jar:na]<br> at com.rabbitmq.client.impl.AMQConnection$1.processAsync(AMQConnection.java:89) ~[amqp-client-2.8.1.jar:na]<br> at com.rabbitmq.client.impl.AMQChannel.handleCompleteInboundCommand(AMQChannel.java:144) ~[amqp-client-2.8.1.jar:na]<br> at com.rabbitmq.client.impl.AMQChannel.handleFrame( AMQChannel.java:91) ~[amqp-client-2.8.1.jar:na]<br> at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:509) ~[amqp-client-2.8.1.jar:na]<br>2012-03-28 15:05:46,061 DEBUG [pool-18-thread-2] c.b.g.r.u.RabbitLinker [RabbitLinker.java:322] Internal RabbitMQ Exception while ACK-ing a message,<br><br><br>I hope you can help me figuring out what is happening.<br><br>Thanks<br><br><br><div><div id="SkyDrivePlaceholder"></div>> Subject: Re: [rabbitmq-discuss] RMQ Java Client - ChannelN.waitForConfirm randomly throwing exceptions<br>> From:<span class="Apple-converted-space"> </span><a href="mailto:steve@rabbitmq.com">steve@rabbitmq.com</a><br>> Date: Wed, 28 Mar 2012 18:17:57 +0100<br>> CC:<span class="Apple-converted-space"> </span><a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>> To:<span class="Apple-converted-space"> </span><a href="mailto:ghepardo_1982@hotmail.com">ghepardo_1982@hotmail.com</a><br>><span class="Apple-converted-space"> </span><br>> Emanuele,<br>><span class="Apple-converted-space"> </span><br>> There should be no need to use AtomicInteger to guarantee a new channel<br>> number; createChannel should guarantee this for you.<br>><span class="Apple-converted-space"> </span><br>> You have shown me the stack trace -- and I can see that your application<br>> threads are issuing waitForConfirms().<br>><span class="Apple-converted-space"> </span><br>> Can you tell me a little more about how your threads use the Channel?<br>> I'd like to know the state of the channel the thread is using, at the<br>> time of the call. What happened on that channel immediately before the<br>> waitForConfirms() call?<br>><span class="Apple-converted-space"> </span><br>> Perhaps some more snippets of the thread code would be useful, and a<br>> little more of the stack trace would help.<br>><span class="Apple-converted-space"> </span><br>> Finally, was there anything in the rabbit log around this time?<br>><span class="Apple-converted-space"> </span><br>> Steve Powell (a happy bunny)<br>> ----------some more definitions from the SPD----------<br>> chinchilla (n.) Cooling device for the lower jaw.<br>> socialcast (n.) Someone to whom everyone is speaking but nobody likes.<br>> literacy (n.) A textually transmitted disease usually contracted in childhood.<br>><span class="Apple-converted-space"> </span><br>> On 28 Mar 2012, at 17:29, Emanuele Gheradini wrote:<br>><span class="Apple-converted-space"> </span><br>> ><span class="Apple-converted-space"> </span><br>> > I want also to add that what I understood is:<br>> ><span class="Apple-converted-space"> </span><br>> > for some reasons RabbitMQ java client gives the same channel (or channel<br>> > number) to different threads, then these threads will perform different<br>> > operations on that channel which will be closed by RabbitMQjavaclient, thus<br>> > resulting in the crash of the waitForConfirm method, at line 182 (here is<br>> > the snipped):<br>> ><span class="Apple-converted-space"> </span><br>> > 181: if (getCloseReason() != null) {<br>> > 182: throw Utility.fixStackTrace(getCloseReason());<br>> > 183: }<br>> ><span class="Apple-converted-space"> </span><br>> ><span class="Apple-converted-space"> </span><br>> > I tried getting channels from rabbit in this way:<br>> ><span class="Apple-converted-space"> </span><br>> > AtomicInteger chanNumber = new AtomicInteger(0);<br>> ><span class="Apple-converted-space"> </span><br>> > connection.createChannel(chanNumber.getAndIncrement());<br>> ><span class="Apple-converted-space"> </span><br>> > to be sure that each thread gets a channel with a different channelNumber,<br>> > but this does not work either...<br>> ><span class="Apple-converted-space"> </span><br>> ><span class="Apple-converted-space"> </span><br>> > I hope someone can give me a suggestion.<br>> ><span class="Apple-converted-space"> </span><br>> > Thanks.<br>> > --<span class="Apple-converted-space"> </span><br>> > View this message in context:<span class="Apple-converted-space"> </span><a href="http://old.nabble.com/RMQ-Java-Client---ChannelN.waitForConfirm-randomly-throwing-exceptions-tp33544834p33544845.html">http://old.nabble.com/RMQ-Java-Client---ChannelN.waitForConfirm-randomly-throwing-exceptions-tp33544834p33544845.html</a><br>> > Sent from the RabbitMQ mailing list archive at<span class="Apple-converted-space"> </span><a href="http://Nabble.com">Nabble.com</a>.<br>> ><span class="Apple-converted-space"> </span><br>> > _______________________________________________<br>> > rabbitmq-discuss mailing list<br>> ><span class="Apple-converted-space"> </span><a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br>> ><span class="Apple-converted-space"> </span><a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br>><span class="Apple-converted-space"> </span><br></div></div></div></span></blockquote></div><br></div></div></body></html>