[rabbitmq-discuss] Will this work?

[Tony Garnock-Jones]

Hi Dinabandhu,

Dinabandhu Mitra wrote:
> 1. What do you mean by full Connection? Should I create a completely
> separate socket per thread? Or does it mean AMQP level connection?

You should create a separate socket per thread, with an AMQP-level
connection per socket.

> 2. If it is AMQP level connection, I have tried that before but it
> did not seem to work. I can try again if you feel it should work.

I understand. You will need to have a one-to-one relationship between
sockets and AMQP-level connections, and you should be extremely careful
about sharing a connection or its channels between threads, since
librabbitmq is written for event-driven, not threaded, use.

> 3. If it is completely separate socket, that is separate socket for
> each consumer and producer I seem to land in interaction between
> nagle and delayed ack which is playing havoc with response times.
> Disabling nagle does not help.

It is not necessarily a separate socket for each consumer: you can have
multiple consumers (in the sense of basic.consume) active on a single
channel. Your code would have a thread responsible for retrieving
messages from the connection and dispatching them out to any waiting
threads, which would not themselves manage and AMQP connection.

I'm not sure what the problem is with your response times. Could you
remind me which operating system you are using, please? If you have
special code for measuring latencies, I'd appreciate it if you could
send it through so I can try it myself to try to find out what the
problem is.

> 4. If I share a (single socket + AMQP connection) between one
> producer and one consumer each on a different thread in same
> application using different channels seem to give best results. But
> in this configuration total message throughput that I can achieve
> with a reasonable latency (sub millisecond) on 1 KB message size is
> around 1000/sec. This throughput is somewhat restrictive.

As you've noticed, it's difficult to simultaneously optimise for good
latency and good throughput. I'm glad you're running these experiments,
please do keep posting your findings to the list (and if you can post
the code at the same time so others can repeat the experiments, then
that's even better).

> 5. Of course I can use multiple "pairs" of such consumers and
> producers to scale up but from the application point of view
> producers and consumers are not really paired, that is one
> application instance will have m producers and n consumers.

What would be nice here would be a thin layer on top of librabbitmq
(perhaps part of it) that was aware of threads, and could manage a
connection on behalf of other threads. It would need to integrate with
some event source (perhaps blocking, perhaps libevent, etc etc), and
respond to message from other threads requesting services like exchange
declaration or message publication, and handle deliveries from the
server, passing out received messages to other threads and collecting
the resulting message acknowledgements to pass back to the server.

Regards,
  Tony