[rabbitmq-discuss] Connection and Channel Management

[Marek Majkowski]

On Thu, Mar 8, 2012 at 23:51, Glen Wagley <glen.wagley at gmail.com> wrote:
> I'm experimenting with rabbitmq to replace our current message queuing
> implementation. Currently our test system is comprised of a REST web
> service, a 3-node rabbitmq cluster, and a windows service. The REST service
> acts as a front-end for our clients to send data to which then gets
> published from the REST service to our rabbitmq cluster. The windows service
> has multiple subscriber threads that consume the messages from the cluster
> and do things like insert data from the message into a relational database,
> send an email containing the message contents, etc.
>
> Right now I am trying to figure out if I am limiting throughput to the
> cluster because each instance of the REST service and each instance of the
> windows service only has one connection to the cluster BUT multiple
> channels. We support about two dozen message types right now and I've
> created a channel for each one. Some channels have a lot of messages flowing
> through while some have little to none depending on the day. Is there a
> recommended approach to number of connections vs channels? We are starting
> to load test the system and I am trying to understand what could be tweaked
> to improve performance/throughput. Thanks.

Glen,

There aren't any specific guidelines about connections vs channels.
Depending on the application, it may be more convenient to use
multiple connections or channels. Use what is more convenient.

Of course, if possible you should reduce number of
connections - connections consume constrained resources
(file descriptors).


Having said that there is one particular case when you could
benefit from having multiple connections.

In a situation when RabbitMQ runs of memory (high memory
watermark), it will stop accepting new messages. But due to the
limitations of AMQP protocol, it can't  be done on per-channel
basis - a full connection will be stopped in such case.

To rephrase - when rabbit runs out of memory the connections
on which messages are being published will be temporarily
throttled. You won't be able to do anything on that connection
until rabbit recovers - that includes consuming messages.

So, if you wish to be shielded against this situation, you
may want to keep publishers on a separate connection
than consumers. That will ensure that even in the
out-of-memory case the consumers will be able to do their
job (and thus help rabbitmq free memory faster)

Hope that helps,
  Marek