[rabbitmq-discuss] Subscribers per Channel
ilya at segment.io
Fri Apr 12 00:16:00 BST 2013
We run an API where each API call ends in RabbitMQ. Each project has its
own queue with the idea that some customers are much bigger than others so
we can eventually have more workers dequeuing the queues of the largest
customers. The end result is that we have 4,500 queues, where the majority
are inactive while about a thousand are active and about 20 are very
We're using the node.js client node-amqp, and it currently creates a
channel per queue (so a channel per subscription). That means each
ingestion worker subscribes to 4.5k queues, and the RabbitMQ server has to
handle 17k channels and subscriptions.
Because of node-amqp, an issue is that we can't use prefetchCount since its
on a subscription level rather on a worker level. The other issue is we're
seeing a very high memory use on the server, even when all messages are
happily going through the queues and being ack'd: 5GB memory without any
active messages. It looks like having so many channels and queues are
eating up all the memory:
A few questions:
1. What is a recommended amount of subscriptions per channel?
2. Should we avoid having so many queues? (one queue per project design)
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the rabbitmq-discuss