[rabbitmq-discuss] Sequential message processing guarantee

Yogesh Ketkar yogimogi at gmail.com
Tue Jan 24 05:29:01 GMT 2012


Hello Steve/Alex,

Thanks a lot for the help. It's pretty clear now.
So basically for a queue, as long as there is going to be only one
consumer, it hardly matters if ThreadPool with
com.rabbitmq.client.Connection underneath the channel has 1 thread or many
threads.
Well, having more threads will help if same channel is going to be used to
consume messages from other queues.

regards, Yogesh

2012/1/23 Steve Powell <steve at rabbitmq.com>

> Hi Yogesh,
>
> What Alex says is perfectly correct.
>
> The executor service is set on the connection, and is there to allow
> multiple
> channels to execute consumers simultaneously, and without blocking the
> channel
> thread. By default you get an executor service which has five threads.
>
> From the point of view of a single channel, the consumers are driven
> serially,
> one delivery at a time, upon one of the executor service threads. The same
> thread is not guaranteed to be used each time for each consumer call for a
> channel (so don't try to use ThreadLocal objects).
>
> The reason we provide this, is so that clients with many, many channels
> are not
> forced to have a Consumer thread per channel -- something that is quite
> expensive -- but have the option of allocating as many threads as they
> wish if
> they need it. At the same time we guarantee that no Consumer calls can
> 'overtake' each other on any one channel, and that Consumer processing
> runs on a
> distinct thread from the channel and the app, so that channel calls in the
> Consumer code will not deadlock.
>
> I hope this clarifies rather than obscures the situation,
>
> Regards,
> Steve Powell  (*a hoppy bunny*)
> *----------some more definitions from the SPD----------*
> *vermin* (*v.*) Treating the dachshund for roundworm.
> *chinchilla* (*n.*) Cooling device for the lower jaw.
> *socialcast* (*n.*) Someone to whom everyone is speaking but nobody likes.
>
> On 23 Jan 2012, at 15:22, Alexandru Scvorţov wrote:
>
> Is my understanding correct?
>
>
> Almost.  Deliveries will happen serially on a channel, so, as long as you
> only have one consumer, everything should be fine.
>
> On Mon, Jan 23, 2012 at 06:54:15AM -0800, Yogesh Ketkar wrote:
>
> Hello Alex,
>
>
> I thought, using default Connection con = factory.newConnection(),
>
> ThreadPool which gets created will have more than one thread and
>
> calling basicConsume will result in processing of messages on multiple
>
> threads.
>
>
> But if we explicitly provide Executors.newSingleThreadExecutor() to
>
> newConnection method, only single thread will come into play with
>
> basicConsume.
>
>
> Is my understanding correct?
>
>
> regards, Yogesh
>
>
>
> On Jan 23, 7:48 pm, Alexandru Scvorţov <alexan... at rabbitmq.com> wrote:
>
> Hi Yogesh,
>
>
> using channel.basicConsume on the channel which is created like this
>
> ExecutorService es = Executors.newSingleThreadExecutor();
>
> Connection connection = factory.newConnection(es);
>
> final Channel channel = connection.createChannel();
>
>
> is probably a better approach.
>
> Any comments on this?
>
>
> I don't quite see the point of the ExecutorService.
>
>
> Just doing this:
>
>
> Connection connection = factory.newConnection(es);
>
> final Channel channel = connection.createChannel();
>
>
> Then, calling basicConsume with your own subclass of DefaultConsumer
>
> should be fine.
>
>
> Cheers,
>
> Alex
>
>
>
>
>
>
>
>
>
>
> On Mon, Jan 23, 2012 at 08:08:37PM +0530, Yogesh Ketkar wrote:
>
> Thanks Alex.
>
>
> I think, for sequential message processing, rather than this pattern
>
> while(true) {
>
>     GetResponse res = channel.basicGet(QUEUE_NAME, false);
>
>     if(res != null) {
>
>         // process and ack message
>
>    }
>
> }
>
>
> using channel.basicConsume on the channel which is created like this
>
> ExecutorService es = Executors.newSingleThreadExecutor();
>
> Connection connection = factory.newConnection(es);
>
> final Channel channel = connection.createChannel();
>
>
> is probably a better approach.
>
> Any comments on this?
>
>
> regards, Yogesh
>
>
> 2012/1/23 Alexandru Scvorţov <alexan... at rabbitmq.com>
>
>
> Hi Yogesh,
>
>
> In the absence of consumer failures, RabbitMQ will deliver messages from
>
> a queue in order.
>
>
> So, if messages 1, 2, 3 reach a queue in order, RabbitMQ will deliver
>
> them to consumers in the same order (1, 2, 3).
>
>
> But, if consumers fail before acknowledging the messages (or if they
>
> reject the messages), those messages will requeued at the end of the
>
> queue.  So, if the consumer that got message 1 rejects it, the new
>
> order of messages will be 2, 3, 1.
>
>
> This all deals with message *delivery*.
>
>
> There are multiple consumer threads consuming the messages.
>
> Though multiple threads are consuming the messages, I observed that
>
> messages still get processed sequentially.
>
> Is that the case and if NOT, what is the way to guarantee sequential
>
> processing of messages on one queue?
>
>
> The library makes no guarantees about the order in which you process
>
> messages, only about the order in which they're delivered.
>
>
> If you want to process all the messages on a queue in order, only
>
> consume from one thread (doing basic.get from one thread like you
>
> suggested in the other email would work, but would also be highly
>
> inefficient).
>
>
> Does this answer your question?
>
>
> Cheers,
>
> Alex
>
>
> On Sun, Jan 22, 2012 at 09:38:40AM -0800, Yogesh Ketkar wrote:
>
> Running the code below, gives
>
> channel.basicConsume(QUEUE_NAME, autoAck, CONSUMER_TAG,
>
>       new DefaultConsumer(channel)  {
>
>         @Override
>
>         public void handleDelivery(String consumerTag, Envelope
>
> envelope, BasicProperties properties, byte[] body) {
>
>                 System.out.println(Thread.currentThread().getName());
>
>          }
>
> }
>
>
> o/p like
>
> pool-1-thread-1
>
> pool-1-thread-2
>
> pool-1-thread-3
>
> etc
>
>
> There are multiple consumer threads consuming the messages.
>
> Though multiple threads are consuming the messages, I observed that
>
> messages still get processed sequentially.
>
> Is that the case and if NOT, what is the way to guarantee sequential
>
> processing of messages on one queue?
>
>
> regards, Yogesh
>
>
> _______________________________________________
>
> rabbitmq-discuss mailing list
>
> rabbitmq-disc... at lists.rabbitmq.com
>
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
> _______________________________________________
>
> rabbitmq-discuss mailing list
>
> rabbitmq-disc... at lists.rabbitmq.comhttps://
> lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
> _______________________________________________
>
> rabbitmq-discuss mailing list
>
> rabbitmq-discuss at lists.rabbitmq.com
>
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
> _______________________________________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.rabbitmq.com
> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20120124/59f58372/attachment.htm>


More information about the rabbitmq-discuss mailing list