[rabbitmq-discuss] RabbitMQ Performance Tests and max of ~4000 msg/sec

[McIntosh Jason]

So an interesting addendum:

I tossed a slight modification of my main method:

		List<Thread> threadsStarted = new ArrayList<Thread>();
		for (int i = 0; i < 5; i++) {
			Thread t = new Thread(new RabbitMqTester("TestExchange", "Test", channel));
			threadsStarted.add(t);
			t.start();
		}
		for (Thread t : threadsStarted) {
			t.join();
		}
And started seeing 16k messages per second.  I'll be doing some more testing to see where my block might be.  Could be as simple as how I'm handling threads...
Thanks!
Jason


On Oct 7, 2012, at 8:33 PM, McIntosh Jason wrote:

> So I've been trying to design a rabbit configuration to eke the most performance possible out of a system.  However, despite a lot of repeated attempts, I'm not able to get more than 4-5k messages/sec out of any rabbit server.  Here are the results of some tests:
> 
> Single connection, Single Exchange, Single queue:  ~4000 msg/sec
> Single connection, Single Exchange, Two queues  :  ~4000 msg/sec
> Single connection, Two Exchanges, Single queue  :  ~4000 msg/sec
> Single connections, Two Exchanges, Two queues   :  ~4000 msg/sec
> Two connections, Two Exchanges, Two queues      :  ~3300 msg/sec (could be a fluke, haven't repeated this test)
> 
> The message rates I've NOT computationally verified - just reading off summary of what rabbit says is going through it's exchanges/queues.  I'll probably work on some more accurate metrics later, but right now, I'm just trying to figure out what to tweak.
> 
> While the test was running, I saw at most a 2mb/sec data write rate, and only about 50-60% CPU use.  Rabbit overview in the admin interface:
> Name	File descriptors (?)
> (used / available)
> Socket descriptors (?)
> (used / available)
> Erlang processes
> (used / available)
> Memory	Disk space	Uptime	Type
> rabbit at jasons-mac-pro
> 35 / 1024
> 3 / 829
> 204 / 1048576
> 496.3MB
> 2.2GB high watermark
> 3.0TB
> 953.7MB low watermark
> 1h 18m	Disc Stats *
> 
> At this point, I'm not sure what the heck to do to get more performance out of Rabbit and could use some advice.  Below is the setup I've got, sample code, etc.  Any advice would be welcome! 
> 
> Thanks!
> Jason
> 
> 
> Here's the setup:
> Two Exchanges, TestExchange and TestExchange2
> Two Queues, Test and Test2
> Bindings for each exchange to the queue with the routing key = Queue name.  I've got the queues setup as direct and durable, as are the queues (durable).
> 
> Here's the java code I'm testing with:
> import java.io.IOException;
> 
> import org.apache.commons.lang.RandomStringUtils;
> 
> import com.rabbitmq.client.Channel;
> import com.rabbitmq.client.Connection;
> import com.rabbitmq.client.ConnectionFactory;
> import com.rabbitmq.client.MessageProperties;
> 
> public class TestRabbitConnection {
> 
> 	public static void main(String[] args) throws Exception {
> 		ConnectionFactory factory = new ConnectionFactory();
> 		factory.setHost("localhost");
> 		factory.setUsername("guest");
> 		factory.setPassword("guest");
> 		Connection conn = factory.newConnection();
> 		Channel channel = conn.createChannel();
> 		Thread t = new Thread(new RabbitMqTester("TestExchange", "Test", channel));
> //		Thread t2 = new Thread(new RabbitMqTester("TestExchange2", "Test2", channel));
> 		t.start();
> //		t2.start();
> 	
> 		t.join();
> //		t2.join();
> 	}
> 	
> 	private static class RabbitMqTester implements Runnable {
> 		private final Channel channel;
> 		private final String exchange;
> 		private final String routingKey;
> 		private final byte[] contents = RandomStringUtils.randomAlphabetic(20).getBytes();
> 
> 		public RabbitMqTester(String exchange, String routingKey, Channel channel) {
> 			this.exchange = exchange;
> 			this.routingKey = routingKey;
> 			this.channel = channel;
> 		}
> 
> 		@Override
> 		public void run() {
> 			try {
> 				while(true) {
> 					channel.basicPublish(exchange, routingKey, MessageProperties.MINIMAL_PERSISTENT_BASIC, contents);
> 				}
> 			} catch (IOException e) {
> 				throw new RuntimeException(e);
> 			}
> 		}
> 		
> 	}
> }
> 
> 
> I'm testing on a MacPro with the following plugins/Erlang:
> 11 plugins activated:
> * amqp_client-2.8.7
> * erlando-2.8.7
> * mochiweb-2.3.1-rmq2.8.7-gitd541e9a
> * rabbitmq_federation-2.8.7
> * rabbitmq_federation_management-2.8.7
> * rabbitmq_management-2.8.7
> * rabbitmq_management_agent-2.8.7
> * rabbitmq_mochiweb-2.8.7
> * rabbitmq_shovel-2.8.7
> * rabbitmq_shovel_management-2.8.7
> * webmachine-1.9.1-rmq2.8.7-git52e62bc
> 
> erlang version : 5.9.2
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20121007/1f710332/attachment.htm>