[rabbitmq-discuss] RabbitMQ scaling with OpenStack

[Ask Solem]

On Jan 23, 2013, at 2:04 AM, Ray Pekowski <pekowski at gmail.com> wrote:

> On Jan 22, 2013, at 1:58 PM, Ask Solem <ask at rabbitmq.com> wrote:
> 
> > I could help you review the code as I have lots of experience optimizing
> > this pattern and Python consumers.
> 
> It would be great if you could review the code.  Here is where the Kombu interface is implemented in OpenStack
> 
> https://github.com/openstack/nova/blob/master/nova/openstack/common/rpc/impl_kombu.py
> 
> Roughly, the RPC call is sent by TopicPublisher (caller), then received by TopicConsumers (callee).  The response is sent by DirectPublisher (callee) and received by DirectConsumer (caller).  In real time, the consumes happen before the publishes.
> 
> The RPC calls to impl_kombu.py come from amqp.py. which is simply a common higher level set of methods that either calls methods from impl_kombu.py or impl_qpid.py depending upon which implementation is configured.
> 
> Ray
> 

I've spent some time reviewing the code, but it's really hard to follow, and it's
reinventing a lot of things that Kombu already supports.   Having two sets of
implementations means that we both have to debug and maintain it, this code could
definitely be reduced by a lot.

One thing that I'm unable to understand is whether it's creating a channel or not
(other than the Connection.default_channel), and if so if it creates a channel for
every request.  This would be bad since creating channels is very expensive (at least in amqplib/py-amqp).

Also, it seems that the reply messages are persistent, do they have to be?