[rabbitmq-discuss] [ANNOUNCEMENT] Introducing Lyra: A High Availability RabbitMQ Client

[Jonathan Halterman]

Hi Matthias - Comments below,

On Thu, Oct 31, 2013 at 12:27 AM, Matthias Radestock
<matthias at rabbitmq.com>wrote:

> Jonathan,
>
>
> On 31/10/13 02:49, Jonathan Halterman wrote:
>
>> We could cache delivery tags for channels internally, but I'm
>> hesitant to go down that road.
>>
>
> It wouldn't help anyway, since, as a mentioned, the same delivery_tag
> can appear on both the old and new incarnation of the channel.
> delivery_tags are just counters, starting from 1 on a fresh channel.
>
>
>  While Lyra certainly makes it easier for this scenario to pop up,
>> rare as it might be,
>>
>
> It's not rare at all. You are guaranteed to end up in that scenario
> anytime a recovery takes place while a consumer is processing a message.
>
>
>  the same problem could occur without Lyra when swapping a consumer
>> over to a new channel after the consumer is unexpectedly cancelled
>> and/or the channel is unexpectedly closed.
>>
>
> Absolutely. Don't do that :)
>
>
>  That said, no good solutions come to mind just yet. Do you have any
>> ideas?
>>
>
> The scenario is impossible to address without changing the API. Bunny does
> that with the VersionedDeliveryTag. The Messaging Patterns libraries we
> wrote some years ago do it by making ack() take the entire Message data
> structure, and have that include the channel version. See
> http://hg.rabbitmq.com/**rabbitmq-java-messagepatterns/**
> file/21df7ca85b96/src/main/**java/com/rabbitmq/**messagepatterns/unicast/*
> *Receiver.java#l19<http://hg.rabbitmq.com/rabbitmq-java-messagepatterns/file/21df7ca85b96/src/main/java/com/rabbitmq/messagepatterns/unicast/Receiver.java#l19>


One of my goals for Lyra was to minimize the amount of new APIs and allow
users to interact mostly with the amqp-client API while still gaining the
benefits of transparent recovery, but it seems like that might not be
possible.

The ReceivedMessage approach would work as long as you don't need to store
the ReceivedMessage off for later use, something that I imagine is not
uncommon. In that case, something like Bunny's VersionedDeliveryTag would
work well, tracking the channel version rather than an unserializable
reference to the channel itself.

I don't suppose you're open to enhancing the amqp-client API to include
something like this? :) One possible change would be the inclusion of a
VersionedDeliveryTag like object in the Envelope along with some
corresponding basic ack/nack/reject(VersionedDeliveryTag) methods on the
Channel interface. The channel's "version" would just be another piece of
meta information that the user could specify. Short of any amqp-client API
changes, I'll have to think about how to best introduce new Lyra APIs to
achieve what's needed here while staying out of the way of the existing
amqp-client API as much as possible.

One other idea - Brett Cameron suggested tracking the last delivery tag for
each channel and dropping any ack/nack/reject requests for delivery tags
greater than that. If a channel is recovered, the last delivery tag for the
new channel will most likely be less than any delivery tags for messages
consumed on the previously closed channel. This is obviously not foolproof,
but it's not a bad start.

Cheers,
Jonathan




>
>
> Regards,
>
> Matthias.
> ______________________________**_________________
> rabbitmq-discuss mailing list
> rabbitmq-discuss at lists.**rabbitmq.com<rabbitmq-discuss at lists.rabbitmq.com>
> https://lists.rabbitmq.com/**cgi-bin/mailman/listinfo/**rabbitmq-discuss<https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/attachments/20131031/52b410be/attachment.htm>