[rabbitmq-discuss] request for help!

Tue May 11 11:51:24 BST 2010

On 10/05/10 18:25, Rafael Schloming wrote:
> Tim Fox wrote:
>> On 10/05/10 16:18, Robert Godfrey wrote:
>>> Just to be clear, while this behaviour is permitted under the spec, it
>>> is not mandated that every message exchange follows this pattern.
>>> Firstly the protocol will support different reliability guarantees
>>> agreed at the link level (at most once, at least once, exactly once,
>>> etc) which will allow simpler patterns where the extra guarantees are
>>> not required.
>> I believe you can use a simpler pattern, but still retain the once 
>> and only once guarantee.
>>
>> This is how we do it in HornetQ to give us once and only once when 
>> bridging messages from one server A to another B.
>>
>> The sender doesn't maintain any delivery map at all. Each message has 
>> a de-duplication-id which can either be set by the client when the 
>> message was originally sent, or can be set by the server and derived 
>> from the message and node id of the sender.
>>
>> Since the id can be derived from the message, or is already persisted 
>> as part of the message there's no need to persist it separately or 
>> maintain a map on the client side.
>
> The specification doesn't require you to persist the delivery-tag 
> separately. It's a perfectly valid (and expected) choice to populate 
> it from the message itself as you describe. When you do this the set 
> of unacked messages has all the necessary information to construct the 
> protocol primitives defined in the spec without maintaining a separate 
> map.
I'm a bit confused then. Section 6.10 of the spec states:

"The application upon initiating a transfer will supply the sending link 
endpoint (Sender) with the
message data and its associated delivery-tag. ** The Sender will create 
an entry in its unsettled map **"

This implies to me that the implementor needs to maintain a settled map.

But if I read your latest comment correctly, you're saying that actually 
I don't have to maintain a settled map, and the set of unacked messages 
will do fine.

Reading this chapter, it seems that it's mandating a way to implement 
once and only once.

If instead, this is just an illustration of a possible implementation, 
and in fact, implementors are free to implement once and only once in a 
simpler way and still remain AMQP compliant, I think it would be a good 
idea to make that clear in the spec.

>
>> The receiving node B simply maintains a circular cache of received 
>> ids (which is optionally persisted). If B sees the same 
>> de-duplication-id more than once it simply ignores the message.
>
> As I believe Rob already pointed out, this is all the information 
> necessary to fulfill the protocol contracts from the Receiver's side.
>
>> After failure, the sender can just carry on sending any unacked 
>> messages as normal and the server will reject any dups.
>>
>> B sends acks back to A asynchronously in a different stream (so 
>> everything is pipelined for performance). When A receives the ack, 
>> then the message can be removed from storage.
>>
>> This gives us once and only once without having the c) interaction 
>> above.
>
> How does B know when A has received the ack?
>
> If this fact is communicated on the wire, then it is equivalent to the 
> c) interaction. If this fact isn't communicated on the wire, then B is 
> making an assumption when it purges entries from its cache, and is 
> exposing itself to duplicate messages, i.e. this is only providing a 
> probabilistic exactly-once guarantee.
>
>> I believe it this gives us one less transfer on the wire, but more 
>> importantly one less write to storage - on the sender side.
>
> The c) interaction has almost no overhead since it is trivially 
> inferred from the window of unsettled deliveries which is piggybacked 
> on message transfers, however both the Sender and Receiver can short 
> circuit the full transfer interaction by indicating that a delivery is 
> settled.
>
> This can be done on the initial transfer if fire and 
> forget/at-most-once messaging is desired, or on the initial ack if 
> at-least-once (or probabilistic exactly-once) messaging is desired.
>
>> I can see that your scheme accomplishes the same goal, but it seems 
>> somewhat more complex, unless I am missing something (quite possible 
>> ;) )
>
> I think perhaps the key point is that the spec is defining the 
> protocol primitives and minimal semantics required for interop, but 
> not actually attempting to prescribe or proscribe a particular 
> implementation strategy. So the intention is very much to permit the 
> sort of things you're describing, but not to constrain implementations 
> to choose only those strategies.
>
> As I mentioned to Rob in another post, we could probably be a bit 
> clearer on this in our descriptions and examples.
>
> --Rafael

-- 
Sent from my BBC Micro Model B

Tim Fox
JBoss

HornetQ - putting the buzz in messaging http://hornetq.org
http://hornetq.blogspot.com/
http://twitter.com/hornetq
irc://irc.freenode.net:6667#hornetq	
fox at redhat.com