[rabbitmq-discuss] Sequential message processing guarantee

[Yogesh Ketkar]

> Well, having more threads will help if same channel is going to be used to
> consume messages from other queues.
Actually more appropriate would be, having more threads will help if
same Connection is going to be used
with different channels.

On Jan 24, 10:29 am, Yogesh Ketkar <yogim... at gmail.com> wrote:
> Hello Steve/Alex,
>
> Thanks a lot for the help. It's pretty clear now.
> So basically for a queue, as long as there is going to be only one
> consumer, it hardly matters if ThreadPool with
> com.rabbitmq.client.Connection underneath the channel has 1 thread or many
> threads.
> Well, having more threads will help if same channel is going to be used to
> consume messages from other queues.
>
> regards, Yogesh
>
> 2012/1/23 Steve Powell <st... at rabbitmq.com>
>
>
>
>
>
>
>
> > Hi Yogesh,
>
> > What Alex says is perfectly correct.
>
> > The executor service is set on the connection, and is there to allow
> > multiple
> > channels to execute consumers simultaneously, and without blocking the
> > channel
> > thread. By default you get an executor service which has five threads.
>
> > From the point of view of a single channel, the consumers are driven
> > serially,
> > one delivery at a time, upon one of the executor service threads. The same
> > thread is not guaranteed to be used each time for each consumer call for a
> > channel (so don't try to use ThreadLocal objects).
>
> > The reason we provide this, is so that clients with many, many channels
> > are not
> > forced to have a Consumer thread per channel -- something that is quite
> > expensive -- but have the option of allocating as many threads as they
> > wish if
> > they need it. At the same time we guarantee that no Consumer calls can
> > 'overtake' each other on any one channel, and that Consumer processing
> > runs on a
> > distinct thread from the channel and the app, so that channel calls in the
> > Consumer code will not deadlock.
>
> > I hope this clarifies rather than obscures the situation,
>
> > Regards,
> > Steve Powell  (*a hoppy bunny*)
> > *----------some more definitions from the SPD----------*
> > *vermin* (*v.*) Treating the dachshund for roundworm.
> > *chinchilla* (*n.*) Cooling device for the lower jaw.
> > *socialcast* (*n.*) Someone to whom everyone is speaking but nobody likes.
>
> > On 23 Jan 2012, at 15:22, Alexandru Scvorţov wrote:
>
> > Is my understanding correct?
>
> > Almost.  Deliveries will happen serially on a channel, so, as long as you
> > only have one consumer, everything should be fine.
>
> > On Mon, Jan 23, 2012 at 06:54:15AM -0800, Yogesh Ketkar wrote:
>
> > Hello Alex,
>
> > I thought, using default Connection con = factory.newConnection(),
>
> > ThreadPool which gets created will have more than one thread and
>
> > calling basicConsume will result in processing of messages on multiple
>
> > threads.
>
> > But if we explicitly provide Executors.newSingleThreadExecutor() to
>
> > newConnection method, only single thread will come into play with
>
> > basicConsume.
>
> > Is my understanding correct?
>
> > regards, Yogesh
>
> > On Jan 23, 7:48 pm, Alexandru Scvorţov <alexan... at rabbitmq.com> wrote:
>
> > Hi Yogesh,
>
> > using channel.basicConsume on the channel which is created like this
>
> > ExecutorService es = Executors.newSingleThreadExecutor();
>
> > Connection connection = factory.newConnection(es);
>
> > final Channel channel = connection.createChannel();
>
> > is probably a better approach.
>
> > Any comments on this?
>
> > I don't quite see the point of the ExecutorService.
>
> > Just doing this:
>
> > Connection connection = factory.newConnection(es);
>
> > final Channel channel = connection.createChannel();
>
> > Then, calling basicConsume with your own subclass of DefaultConsumer
>
> > should be fine.
>
> > Cheers,
>
> > Alex
>
> > On Mon, Jan 23, 2012 at 08:08:37PM +0530, Yogesh Ketkar wrote:
>
> > Thanks Alex.
>
> > I think, for sequential message processing, rather than this pattern
>
> > while(true) {
>
> >     GetResponse res = channel.basicGet(QUEUE_NAME, false);
>
> >     if(res != null) {
>
> >         // process and ack message
>
> >    }
>
> > }
>
> > using channel.basicConsume on the channel which is created like this
>
> > ExecutorService es = Executors.newSingleThreadExecutor();
>
> > Connection connection = factory.newConnection(es);
>
> > final Channel channel = connection.createChannel();
>
> > is probably a better approach.
>
> > Any comments on this?
>
> > regards, Yogesh
>
> > 2012/1/23 Alexandru Scvorţov <alexan... at rabbitmq.com>
>
> > Hi Yogesh,
>
> > In the absence of consumer failures, RabbitMQ will deliver messages from
>
> > a queue in order.
>
> > So, if messages 1, 2, 3 reach a queue in order, RabbitMQ will deliver
>
> > them to consumers in the same order (1, 2, 3).
>
> > But, if consumers fail before acknowledging the messages (or if they
>
> > reject the messages), those messages will requeued at the end of the
>
> > queue.  So, if the consumer that got message 1 rejects it, the new
>
> > order of messages will be 2, 3, 1.
>
> > This all deals with message *delivery*.
>
> > There are multiple consumer threads consuming the messages.
>
> > Though multiple threads are consuming the messages, I observed that
>
> > messages still get processed sequentially.
>
> > Is that the case and if NOT, what is the way to guarantee sequential
>
> > processing of messages on one queue?
>
> > The library makes no guarantees about the order in which you process
>
> > messages, only about the order in which they're delivered.
>
> > If you want to process all the messages on a queue in order, only
>
> > consume from one thread (doing basic.get from one thread like you
>
> > suggested in the other email would work, but would also be highly
>
> > inefficient).
>
> > Does this answer your question?
>
> > Cheers,
>
> > Alex
>
> > On Sun, Jan 22, 2012 at 09:38:40AM -0800, Yogesh Ketkar wrote:
>
> > Running the code below, gives
>
> > channel.basicConsume(QUEUE_NAME, autoAck, CONSUMER_TAG,
>
> >       new DefaultConsumer(channel)  {
>
> >         @Override
>
> >         public void handleDelivery(String consumerTag, Envelope
>
> > envelope, BasicProperties properties, byte[] body) {
>
> >                 System.out.println(Thread.currentThread().getName());
>
> >          }
>
> > }
>
> > o/p like
>
> > pool-1-thread-1
>
> > pool-1-thread-2
>
> > pool-1-thread-3
>
> > etc
>
> > There are multiple consumer threads consuming the messages.
>
> > Though multiple threads are consuming the messages, I observed that
>
> > messages still get processed sequentially.
>
> > Is that the case and if NOT, what is the way to guarantee sequential
>
> > processing of messages on one queue?
>
> > regards, Yogesh
>
> > _______________________________________________
>
> > rabbitmq-discuss mailing list
>
> > rabbitmq-disc... at lists.rabbitmq.com
>
> >https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
> > _______________________________________________
>
> > rabbitmq-discuss mailing list
>
> > rabbitmq-disc... at lists.rabbitmq.comhttps://
> > lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
> > _______________________________________________
>
> > rabbitmq-discuss mailing list
>
> > rabbitmq-disc... at lists.rabbitmq.com
>
> >https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
> > _______________________________________________
> > rabbitmq-discuss mailing list
> > rabbitmq-disc... at lists.rabbitmq.com
> >https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-disc... at lists.rabbitmq.comhttps://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss