[rabbitmq-discuss] Subscribers per Channel

[Ilya Volodarsky]

We run an API where each API call ends in RabbitMQ. Each project has its 
own queue with the idea that some customers are much bigger than others so 
 we can eventually have more workers dequeuing the queues of the largest 
customers. The end result is that we have 4,500 queues, where the majority 
are inactive while about a thousand are active and about 20 are very 
heavily used. 

We're using the node.js client node-amqp, and it currently creates a 
channel per queue (so a channel per subscription). That means each 
ingestion worker subscribes to 4.5k queues, and the RabbitMQ server has to 
handle 17k channels and subscriptions. 

Because of node-amqp, an issue is that we can't use prefetchCount since its 
on a subscription level rather on a worker level. The other issue is we're 
seeing a very high memory use on the server, even when all messages are 
happily going through the queues and being ack'd: 5GB memory without any 
active messages. It looks like having so many channels and queues are 
eating up all the memory: 

A few questions:

1. What is a recommended amount of subscriptions per channel?
2. Should we avoid having so many queues? (one queue per project design)

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20130411/7463ccf9/attachment.htm>