[rabbitmq-discuss] Multithreaded application, separation of connections and channels

Thu Jun 25 09:59:15 BST 2009

Hi,

I'm currently in the design phase of a new system and we are going to
experiment with AMQP as it allows us to model a few things much better
than JMS than we used before. Perhaps this would better belong on a
AMQP general list, but since I was planning on using RabbitMQ I
figured I could try here, there might be some implementation details
that are relevant for example.

Anyway, at the simplest level, there will be an application that will
need to consume and produce messages in multiple threads. For say,
producing, one channel will be given for each thread that is
responsible for writing data as they are not thread safe, but I was
wondering if there are any best practices regarding the connections
that the channels are created from.

The way I see it, there are 3 possible combinations.

1) Each thread has its own connection from which it creates one channel
2) One connection is shared for the whole application, multiple
channels are created and given to threads
3) A hybrid variant where X channels is created from every connection,
and depending on the number of threads, a number of connections are
created

The application needs to be fail-safe and be able to restablish
connections to the broker in case of network problems, broker goes
down etc. For that reason, [1] looks easier to implement as each
thread can handle all by itself in isolation. For [2] it would be more
tricky, as the failure of the connection would be reported in one part
of the application, who would then need to produce new channels and
give to the worker threads. [3] would be basically the same as [2]

I was also curious about performance. If a large amount of messages
are sent, it seems like it would be more efficient to send those over
a set of sockets ([1] or [3] above), instead of forcing them all
through the only one available ([2] above).

Any input would be greatly appreciated.

Thanks!

Matthias