[rabbitmq-discuss] Managing channels, esp wrt short lived publishers/consumers

Fri Mar 25 15:51:11 GMT 2011

Marek,

Thanks for the feedback. I was uncertain about the role channels played; 
nice to have some clarification. I've been busy implementing the pool of 
channels and have been encouraged by initial tests.

Looking forward to the RabbitMQ talk at the Erlang Factory today.

James

On 03/25/2011 07:14 AM, Marek Majkowski wrote:
> James,
>
> On Fri, Mar 18, 2011 at 18:52, James Aimonetti
> <james.aimonetti at gmail.com>  wrote:
>> I'm trying to figure out the best way to manage channel creation in my app.
>> Currently, any process that needs to publish or consume gets a channel and I
>> realize it is probably horribly inefficient, but as a prototype I wasn't too
>> concerned. Now that we're moving forward, I want to get this cleaned up
>> before I get too much further along.
>>
>> I have a setup something like:
>>
>> [App, App, App]<--->  [AMQP]<-->  Engine<-->  Client
>>
>> So a client makes a request to the Engine, which creates two processes, one
>> to publish events from the client, and one to consume commands from the Apps
>> and relay them to the client.
>>
>> [App, App, App]<-->  [AMQP]<-- ReqEvts<-- Client
>> [App, App, App]<-->  [AMQP] -->  ReqCtl -->  Client
>>
>> Ideally we'll be handling several hundreds of requests per second from
>> clients. So the ReqEvts process pumps 10 or so messages in rapid succession
>> at the beginning of a request, heartbeats every 20 or so seconds, then pumps
>> another 10 or so messages at the end of the request. Requests may last 1
>> second (so setup and tear down bursts happen almost on top of each other) or
>> may last minutes or hours. Meanwhile, the ReqCtl receives commands from a
>> targeted exchange from the Apps to relay to the Client.
>>
>> Each App has a queue that it binds to a targeted exchange and one or more
>> topic exchanges. In general it receives most messages off the topic
>> exchanges and publishes to the targeted exchange.
>>
>> So, hopefully that gives a broad overview. The Apps and the Engine pieces
>> are all on different servers.
>>
>> So right now each server has an instance of rabbitmq running, and is
>> clustered to all the other servers (Apps and Engine, currently four total
>> servers but expandable). What I'm not sure about is how to manage each
>> server's connection and channels. Each server has a gen_server managing the
>> connection to rabbit, as well as the channels. So when a pid requests to
>> publish or consume or declare a queue, the gen_server checks if the pid is
>> known and uses that channel, or creates a new channel and stores the
>> association. With all of these potentially short-lived consumers and
>> publishers, I see a LOT of channel creation/destruction going on and wonder
>> if there's a better way? Should I not have hidden channels behind a
>> gen_server?
> Having a pool of channels is quite a common practice.
>
>> My initial thought is to have a dedicated channel for all publishers on the
>> server to send over, a dedicated channel for any queue declarations, and a
>> channel for each process that is consuming from a queue. Am I serializing
>> too much with this setup?
> That should be fine. Some people use a single channel for all the stuff.
>
> Channel is a weird abstraction, it's mainly responsible for:
>   - scoping errors (ie: channel is closed on error)
>   - message ordering (messages sent through a channel will be delivered in order)
>   - also, some resources may be associated with channel,
>     and its state may change when the channel is closed (autodelete queues,
>     unacknowledged messages)
>   - parallelization of synchronous actions (like queue.declare)
>
> Basically, if you don't need this features, you can just use a single channel.
> In other case you must use multiple channels, but managing the lifetime
> of a channel is your decision.
>
> For example you may want to destroy channels often to
> isolate errors (ie: if one channel fails for some reason, you
> want it to affect only a single 'request' in your app)
>
> In your case, it looks like you only need the 'parallelization' feature,
> so using a pool of channels should do.
>
> Cheers,
>    Marek

-- 
James Aimonetti
Distributed Systems Engineer / DJ MC_

2600hz | http://2600hz.com
sip:james at 2600hz.com
tel: 415.886.7905