[rabbitmq-discuss] Managing channels, esp wrt short lived publishers/consumers

[Marek Majkowski]

James,

On Fri, Mar 18, 2011 at 18:52, James Aimonetti
<james.aimonetti at gmail.com> wrote:
> I'm trying to figure out the best way to manage channel creation in my app.
> Currently, any process that needs to publish or consume gets a channel and I
> realize it is probably horribly inefficient, but as a prototype I wasn't too
> concerned. Now that we're moving forward, I want to get this cleaned up
> before I get too much further along.
>
> I have a setup something like:
>
> [App, App, App] <---> [AMQP] <--> Engine <--> Client
>
> So a client makes a request to the Engine, which creates two processes, one
> to publish events from the client, and one to consume commands from the Apps
> and relay them to the client.
>
> [App, App, App] <--> [AMQP] <-- ReqEvts <-- Client
> [App, App, App] <--> [AMQP] --> ReqCtl --> Client
>
> Ideally we'll be handling several hundreds of requests per second from
> clients. So the ReqEvts process pumps 10 or so messages in rapid succession
> at the beginning of a request, heartbeats every 20 or so seconds, then pumps
> another 10 or so messages at the end of the request. Requests may last 1
> second (so setup and tear down bursts happen almost on top of each other) or
> may last minutes or hours. Meanwhile, the ReqCtl receives commands from a
> targeted exchange from the Apps to relay to the Client.
>
> Each App has a queue that it binds to a targeted exchange and one or more
> topic exchanges. In general it receives most messages off the topic
> exchanges and publishes to the targeted exchange.
>
> So, hopefully that gives a broad overview. The Apps and the Engine pieces
> are all on different servers.
>
> So right now each server has an instance of rabbitmq running, and is
> clustered to all the other servers (Apps and Engine, currently four total
> servers but expandable). What I'm not sure about is how to manage each
> server's connection and channels. Each server has a gen_server managing the
> connection to rabbit, as well as the channels. So when a pid requests to
> publish or consume or declare a queue, the gen_server checks if the pid is
> known and uses that channel, or creates a new channel and stores the
> association. With all of these potentially short-lived consumers and
> publishers, I see a LOT of channel creation/destruction going on and wonder
> if there's a better way? Should I not have hidden channels behind a
> gen_server?

Having a pool of channels is quite a common practice.

> My initial thought is to have a dedicated channel for all publishers on the
> server to send over, a dedicated channel for any queue declarations, and a
> channel for each process that is consuming from a queue. Am I serializing
> too much with this setup?

That should be fine. Some people use a single channel for all the stuff.

Channel is a weird abstraction, it's mainly responsible for:
 - scoping errors (ie: channel is closed on error)
 - message ordering (messages sent through a channel will be delivered in order)
 - also, some resources may be associated with channel,
   and its state may change when the channel is closed (autodelete queues,
   unacknowledged messages)
 - parallelization of synchronous actions (like queue.declare)

Basically, if you don't need this features, you can just use a single channel.
In other case you must use multiple channels, but managing the lifetime
of a channel is your decision.

For example you may want to destroy channels often to
isolate errors (ie: if one channel fails for some reason, you
want it to affect only a single 'request' in your app)

In your case, it looks like you only need the 'parallelization' feature,
so using a pool of channels should do.

Cheers,
  Marek