<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div>Hi Guatam,</div><div><br></div><div>At first sight, this looks like it should work -- your (large) number of</div><div>tasks, will get gated through the 20 threads, which will use the 50</div><div>channels. It ought to be the case that there are always more than enough</div><div>channels to be 'in use' at any one time. The thread pool will dispatch</div><div>the tasks to threads as they become available....</div><div><br></div><div>I'll experiment to see if I can reproduce your experience, and get back</div><div>to you next week (after the bank holiday w/e here).</div><div><br></div><div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-variant: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div style="font-style: normal; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div style="font-family: Georgia; font-size: medium; font-weight: normal; font-style: normal; "><div><font class="Apple-style-span" size="3"><span class="Apple-style-span" style="font-size: 12px; ">Steve Powell </span><span class="Apple-style-span" style="font-size: 11px; ">(<i>a happy bunny</i>)</span></font></div></div><div><font class="Apple-style-span"><div style="font-weight: normal; font-style: normal; font-family: Helvetica; font-size: 12px; "><i><font class="Apple-style-span" face="Georgia" size="2"><span class="Apple-style-span" style="font-size: 10px; ">----------some more definitions from the SPD----------</span></font></i></div></font></div></div></div><span class="Apple-style-span" style="font-style: normal; font-weight: normal; font-family: Helvetica; font-size: 12px; "><font class="Apple-style-span" face="Georgia" size="2" style="font-style: normal; "><span class="Apple-style-span" style="font-size: 10px; "><b>chinchilla</b> (</span></font><font class="Apple-style-span" face="Georgia" size="2"><span class="Apple-style-span" style="font-size: 10px; "><i>n.</i>) Cooling device for the lower jaw.</span></font></span><div style="font-style: normal; font-family: Georgia; font-size: medium; font-weight: normal; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Georgia; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "></span><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-variant: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><span class="Apple-style-span" style="font-family: Georgia; font-weight: normal; font-style: normal; font-size: 10px; "><b>socialcast</b></span><span class="Apple-style-span" style="font-family: Georgia; font-weight: normal; font-style: normal; font-size: 10px; "> (</span><span class="Apple-style-span" style="font-family: Georgia; font-weight: normal; font-size: 10px; "><i>n.</i>)</span><span class="Apple-style-span" style="font-family: Georgia; font-weight: normal; font-style: normal; font-size: 10px; "> Someone to whom everyone is speaking but nobody likes.</span></span></div><div style="font-family: Georgia; font-size: medium; "><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-variant: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><span class="Apple-style-span" style="font-family: Georgia; font-style: normal; font-size: 10px; "><b>literacy</b></span><span class="Apple-style-span" style="font-family: Georgia; font-weight: normal; font-style: normal; font-size: 10px; "><span class="Apple-converted-space"> </span>(</span><span class="Apple-style-span" style="font-family: Georgia; font-weight: normal; font-size: 10px; "><i>n.</i></span><span class="Apple-style-span" style="font-family: Georgia; font-weight: normal; font-style: normal; font-size: 10px; ">) A textually transmitted disease usually contracted in childhood.</span></span></div></div></span></div>
</div>
<br><div><div>On 3 May 2012, at 04:34, Gautam Bakshi wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">Hi Everyone,<div><br></div><div>I'm playing around with Rabbitmq(sorry for flooding the list lately) and am having some weird problems in my tests. I have a multi-threaded application that I was considering using rabbitmq to manage the queue but I'm getting alot of blocking between my channels. Is there a preferred way to setup pools?
</div><div><br></div><div>I'm a bit new to Java so I used to the pools from apache commons but when I profile the channels are all blocking each other. To test if dedicated connections per thread worked better I made an example of it as well and it does not block when the same program has a single channel per connection. </div>
<div><br></div><div>So my question, do channels block each other or am I doing something wrong(using java api wrong or misunderstanding connection/channels)? Is there a more preferred way? Unrelated to the question, but I was also wondering is there any difference between channels and connections in terms of throughput(i.e. would there be any benefit of using dedicated connections ignoring the overhead in establishing/maintaining the connection)?</div>
<div><br></div><div>Here's the test code(it simply sends to a thread a bunch of data to write the queue). If it helps I'm using v2.7.1, installed from MacPorts):</div><div><br></div><div><br></div><div><div>import org.apache.commons.pool.BasePoolableObjectFactory;</div>
<div>import org.apache.commons.pool.ObjectPool;</div><div>import org.apache.commons.pool.PoolableObjectFactory;</div><div>import org.apache.commons.pool.impl.GenericObjectPool;</div><div><br></div><div>import java.io.IOException;</div>
<div>import java.util.NoSuchElementException;</div><div>import java.util.Set;</div><div>import java.util.concurrent.ExecutorService;</div><div>import java.util.concurrent.Executors;</div><div>import java.util.concurrent.LinkedBlockingDeque;</div>
<div>import java.util.concurrent.ThreadPoolExecutor;</div><div>import java.util.concurrent.TimeUnit;</div><div><br></div><div><br></div><div>import com.rabbitmq.client.ConnectionFactory;</div><div>import com.rabbitmq.client.Connection;</div>
<div>import com.rabbitmq.client.Channel;</div><div>import com.rabbitmq.client.MessageProperties;</div><div><br></div><div>public class PoolExample {</div><div> </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>private static ExecutorService executor_worker;</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>static {</div><div> final int numberOfThreads_ThreadPoolExecutor = 20;</div>
<div> executor_worker = Executors.newFixedThreadPool(numberOfThreads_ThreadPoolExecutor);</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div><span class="Apple-tab-span" style="white-space:pre"> </span></div>
<div> public static void main(String[] args) throws Exception {</div><div> <span class="Apple-tab-span" style="white-space:pre"> </span>System.out.println("starting.."); <span class="Apple-tab-span" style="white-space:pre"> </span> <span class="Apple-tab-span" style="white-space:pre"> </span></div>
<div> <span class="Apple-tab-span" style="white-space:pre"> </span>ObjectPool<Channel> pool =</div><div> new GenericObjectPool<Channel>(</div><div> new ConnectionPoolableObjectFactory(), 50);</div>
<div> <span class="Apple-tab-span" style="white-space:pre"> </span>for (int x = 0; x<500000000; x++) {</div><div> <span class="Apple-tab-span" style="white-space:pre"> </span>executor_worker.submit(new MyRunnable(x, pool));</div>
<div> <span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div> }</div><div>}</div><div><br></div><div>/*</div><div> //this is a version that creates its own connection per channel</div><div> class ConnectionPoolableObjectFactory</div>
<div> extends BasePoolableObjectFactory<Channel> {</div><div><br></div><div> private static final ConnectionFactory factory = newConnectionFactory();</div><div><br></div><div> private static ConnectionFactory newConnectionFactory() {</div>
<div> ConnectionFactory factory = new ConnectionFactory();</div><div> factory.setHost("localhost");</div><div> return factory;</div><div> }</div><div><br></div><div> @Override</div><div>
public Channel makeObject() throws Exception {</div><div> System.out.println("new channel");</div><div> <span class="Apple-tab-span" style="white-space:pre"> </span>return factory.newConnection().createChannel();</div>
<div> </div><div> }</div><div><br></div><div> @Override</div><div> public boolean validateObject(Channel channel) {</div><div> return channel.isOpen();</div><div> }</div><div><br></div><div> @Override</div>
<div> public void destroyObject(Channel channel) throws Exception {</div><div> <span class="Apple-tab-span" style="white-space:pre"> </span>System.out.println("closing");</div><div> channel.close();</div>
<div> }</div><div><br></div><div> @Override</div><div> public void passivateObject(Channel channel) throws Exception {</div><div> //System.out.println("sent back to queue");</div><div> }</div>
<div>}</div><div>*/</div><div><br></div><div>//this version pools channels </div><div>class ConnectionPoolableObjectFactory extends BasePoolableObjectFactory<Channel> {</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>private final Connection connection;</div>
<div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>ConnectionPoolableObjectFactory() throws IOException {</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>ConnectionFactory factory = new ConnectionFactory();</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>factory.setHost("localhost");</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>connection = factory.newConnection();</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>@Override</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>public Channel makeObject() throws Exception {</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>return connection.createChannel();</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>@Override</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>public boolean validateObject(Channel channel) {</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>return channel.isOpen();</div><div>
<span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>@Override</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>public void destroyObject(Channel channel) throws Exception {</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>channel.close();</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>@Override</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>public void passivateObject(Channel channel) throws Exception {</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>//do nothing</div><div>
<span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div>}</div><div><br></div><div><br></div><div>class MyRunnable implements Runnable{ </div><div><span class="Apple-tab-span" style="white-space:pre"> </span>protected int x = 0;</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>protected ObjectPool<Channel> pool;</div><div><span class="Apple-tab-span" style="white-space:pre"> </span></div><div> public MyRunnable(int x, ObjectPool<Channel> pool) {</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>// TODO Auto-generated constructor stub</div><div> <span class="Apple-tab-span" style="white-space:pre"> </span>this.x = x;</div><div> <span class="Apple-tab-span" style="white-space:pre"> </span>this.pool = pool;</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div><br></div><div><span class="Apple-tab-span" style="white-space:pre"> </span>public void run(){</div><div> try {</div><div> Channel channel = pool.borrowObject();</div>
<div> <span class="Apple-tab-span" style="white-space:pre"> </span>String message = Integer.toString(x);</div><div> <span class="Apple-tab-span" style="white-space:pre"> </span>channel.basicPublish( "", "task_queue", </div>
<div> MessageProperties.PERSISTENT_TEXT_PLAIN,</div><div> message.getBytes());</div><div> <span class="Apple-tab-span" style="white-space:pre"> </span>pool.returnObject(channel);</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>} catch (NoSuchElementException e) {</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>// TODO Auto-generated catch block</div><div>
<span class="Apple-tab-span" style="white-space:pre"> </span>e.printStackTrace();</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>} catch (IllegalStateException e) {</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>// TODO Auto-generated catch block</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>e.printStackTrace();</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>} catch (Exception e) {</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>// TODO Auto-generated catch block</div>
<div><span class="Apple-tab-span" style="white-space:pre"> </span>e.printStackTrace();</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>} </div><div> }</div><div>}</div><div><br></div></div>
_______________________________________________<br>rabbitmq-discuss mailing list<br><a href="mailto:rabbitmq-discuss@lists.rabbitmq.com">rabbitmq-discuss@lists.rabbitmq.com</a><br><a href="https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss">https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss</a><br></blockquote></div><br></div></body></html>