[rabbitmq-discuss] question on message ordering guarantees

[Alexis Richardson]

See the long discussion on ordering posted in the depths of the thread here:

http://www.quora.com/RabbitMQ/RabbitMQ-vs-Kafka-which-one-for-durable-messaging-with-good-query-features/

Note that Kafka has at most one consumer per 'partition' which is
equivalent to the property Tim described.


On Thu, Oct 18, 2012 at 12:03 PM, Alexis Richardson <alexis at rabbitmq.com> wrote:
> If you use the consistent hashing exchange, then you can make sure
> that two messages with the same key are sent to the same queue.  Then,
> provided you do not allow parallel processing of those messages due to
> how they are consumed, you can "process in order" as required.
>
>
>
>
>
> On Thu, Oct 18, 2012 at 11:43 AM, Tim Watson <tim at rabbitmq.com> wrote:
>> Hi
>>
>>
>> On 10/18/2012 10:30 AM, Artur Ejsmont wrote:
>>
>> Hi there everyone,
>>
>> I am working on a rabbitmq based messaging system and i was wondering if you
>> could help me out :)
>>
>> I was wondering how could i enforce events being processed in the order they
>> are published?
>>
>>
>> So from http://www.rabbitmq.com/semantics.html, we can see:
>>
>> "Section 4.7 of the AMQP 0-9-1 core specification explains the conditions
>> under which ordering is guaranteed: messages published in one channel,
>> passing through one exchange and one queue and one outgoing channel will be
>> received in the same order that they were sent. RabbitMQ offers stronger
>> guarantees since release 2.7.0.
>>
>> Messages can be returned to the queue using AMQP methods that feature a
>> requeue parameter (basic.recover, basic.reject and basic.nack), or due to a
>> channel closing while holding unacknowledged messages. Any of these
>> scenarios caused messages to be requeued at the back of the queue for
>> RabbitMQ releases earlier than 2.7.0. From RabbitMQ release 2.7.0, messages
>> are always held in the queue in publication order, even in the presence of
>> requeueing or channel closure.
>>
>> With release 2.7.0 and later it is still possible for individual consumers
>> to observe messages out of order if the queue has multiple subscribers. This
>> is due to the actions of other subscribers who may requeue messages. From
>> the perspective of the queue the messages are always held in the publication
>> order."
>>
>> So you've got to think about it in those terms. It's exactly the same with
>> ActiveMQ message groups, because they are based on the Exclusive Consumer
>> feature, which states: "The broker will pick a single MessageConsumer to get
>> all the messages for a queue to ensure ordering. If that consumer fails, the
>> broker will auto failover and choose another consumer."
>> (http://activemq.apache.org/exclusive-consumer.html) except that groups are
>> parallel, so: "... logically Message Groups are kinda like a parallel
>> Exclusive Consumer. Rather than all messages going to a single consumer, the
>> standard JMS header JMSXGroupID is used to define which message group the
>> message belongs to. The Message Group feature then ensures that all messages
>> for the same message group will be sent to the same JMS consumer - while
>> that consumer stays alive. As soon as the consumer dies another will be
>> chosen." (http://activemq.apache.org/message-groups.html).
>>
>> You might be able kludge something similar using the consistent hash
>> exchange plugin
>> (http://hg.rabbitmq.com/rabbitmq-consistent-hash-exchange/file/rabbitmq_v2_8_7/README.md)
>> but you will *not* get the same kind of guarantees.
>>
>> We have been asked about this feature before and will probably implement
>> something like it in a future release.
>>
>> Cheers,
>> Tim
>>
>> _______________________________________________
>> rabbitmq-discuss mailing list
>> rabbitmq-discuss at lists.rabbitmq.com
>> https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>