[rabbitmq-discuss] Process N messages / VM concurrently

[Ben Hood]

Joern,

On Wed, Jul 2, 2008 at 7:58 PM, Joern <opendev at gmail.com> wrote:
> I'm new to AMQP and RabbitMQ and like to solve the following problem.
> I am going to deploy a couple of JVMs on different nodes. Depending on
> certain characteristics of these nodes they will be capable of
> handling a node-specific N of messages concurrently. For each message
> processed a worker Thread is used (probably using Java's Executor
> framework). A message specific to the outcome of the work done in the
> thread is put into the queue after the job is done and the transaction
> is finished / the original message is acknowledged (I'm still a bit
> unsure of the appropriate semantics here).
>
> Is there a recommended strategy how to combine the Executor framework
> of Java 5+ with the BlockingConsumer of Rabbit? I'm wiring the
> application using Spring and use the latest version of RabbitMQ.

I'm not sure about the Executor framework but I think one thing that
you may want to consider is binding one AMQP channel to each worker
thread. The connection is thread safe and hence can be shared, but the
channel is designed to be the smallest unit of parallelism.
Furthermore, they are not threadsafe for this very reason.

Whilst it is possible to spawn a new thread in your own consumer for
each message, but going down the multi-channel route is more in line
with the AMQP model.

HTH,

Ben