[rabbitmq-discuss] Fwd: question on the faq

Mon Jan 5 16:30:06 GMT 2009

Tim,

On Mon, Jan 5, 2009 at 3:11 PM, Tim Coote <tim+rabbitmq.com at coote.org> wrote:
> Hullo

Hi :-)

> Remember, I'm new to messaging technology. I'm coming at this area from a
> more general perspective.

Understood and I would not wish to dissuade anyone from asking about
this stuff.  It is an essential part of the community process around
AMQP.  Ask away and comment freely!

If you follow the google cloud computing group, you will see a similar
discussion, this time about eventual consistency.  My point being -
here be dragons - people should discuss it and discuss it until it is
clear and all good implementations work.

> Starting with Gray and Reuter's definition of a
> Transaction (an ACID unit of work). My guess is that the term's used in a
> different way in the messaging community, so I'm likely to make mistakes.
>  My background is mostly about understanding how to control technology and
> its use, so I focus on understanding the limits first.

For sure; and your point about making it easier for end users (eg app
dev) to avoid mistakes, with minimal code, was spot on.

In fact one can take two different views of transactions in the sense
of AMQP TX:

1. Operations are operations on *resources* - so more like databases
(Jim Gray et al.).

This would be fine if it made sense for messaging but arguably it does
not fit messaging in every case.  It makes some sense in the MQ-style
idempotent delivery case; and when HA is well defined in combination
with that; and in some DTX cases.  It would also make sense if all
operations on resources in AMQP (when that makes sense) were
explicitly formalised as such and grouping laws were similarly
defined.

2. Operations are about control of a *stream*.

We like this view as being consistent with AMQP in its full
generality.  Also, as Carl implied, there is a case for seeing TX as
an "acknowledgement pattern" if you take the interpretation Matthias
and I described for 0-8/9 TX.

> Atomic means either everything happens or nothing happens as far as the
> system using a transactional interface is concerned.

At the risk of being Clintonian, it depends what you mean by
'happens'.  Routing to a queue is acceptable streaming behaviour.  So
if the queue fails after being routed to, then it is not unreasonable
to treat something as having 'happened'.

> At minimum, I'd expect
> that if my app sends a message to n recipients, they either all get it or
> none of them do and my app is reliably informed what the outcome was.

We should discuss this some more.  In my own view some of such cases
are better handled using idemptotent delivery; or even DTX.

> I
> interpreted Gordon's comment to mean that there were side effects that could
> be abused by programmers to mean that the message had got through.

I don't think so - but I may be wrong.  I think Gordon's point shows
how TX in 0-8/9 leaves room for debate in some cases.  We would LOVE
to reach a more precise definition acceptable to the 0-8/9 user base,
pending 1.0.

> My
> concern would be that some programmer would use a side effect and come to
> the wrong conclusion.  I probably need to read the specs before I make more
> of a fool of myself.  I'll try to read them before I make any comment after
> this.

Please DO let us know what you think of 0-9-1.  How can we improve it?

> I know some of the guys in JPMC at that time.  They knew what they were
> doing.

Yes they did.  Maximum respect.

> Of course, they could have been different people from the ones that
> were working on what became RabbitMQ, although one did tell me that he was
> sponsoring some open source messaging work.

Yes - RabbitMQ implementation began after 0-8 was made public.  Since
then we have had discussions with past and new implementors through
the magic of the AMQP WG.

alexis

>
> Tim
>
>
> On 5 Jan 2009, at 12:55, Alexis Richardson wrote:
>
>>
>> Tim,
>>
>> On Mon, Jan 5, 2009 at 12:36 PM, Tim Coote <tim+rabbitmq.com at coote.org>
>> wrote:
>>>
>>> Mostly the value comes from
>>> not having application programmers try to fix errors that they
>>> misunderstand, leading to increased cost and decreased quality and
>>> performance.
>>
>> I agree this is a key benefit of ACID in a system with side effects.
>> But it is also important to be clear about what sorts of consistency
>> (etc) are promised.
>>
>>
>>
>>> your description implies to me that RabbitMQ message delivery cannot
>>> be Atomic, even at the single message level,
>>
>> I don't understand how you can draw this conclusion from what I said
>> (note that the comment below is from Gordon).
>>
>> My statement included the assertion that at the single message level
>> RabbitMQ delivery *is* atomic.  Without single message atomicity it
>> would be very hard to reason about consistency *at all*.
>>
>>
>>
>>> let alone defining units
>>> of work that span messages.
>>
>> RabbitMQ provides a correct implementation of the TX class which
>> enables work to be grouped.  It is the TX class to which Gordon refers
>> in his email below.  The problem is that the definition of TX is open
>> to *some* interpretation in edge cases, and is implemented slightly
>> differently by the various brokers.  When we discussed the differences
>> in the working group we agreed that it would be more useful to clarify
>> how each 'correct' interpretation actually works for pre 1.0 specs,
>> pending AMQP 1.0.
>>
>> We would welcome a discussion of the subtleties of TX on this list.
>> If you can find holes in our implementation we would be delighted to
>> make it better!
>>
>>
>>> Shame.  Promising that it's even been discussed tho'.
>>
>> TX was defined in the period 2004-2006 before AMQP became a public
>> spec, when it was in incubation at JPMorgan Chase with the assistance
>> of several groups of software professionals.  But it's not yet
>> perfect.
>>
>> The complexity of TX arises due to the fact that messaging is about
>> routing and delivery as much as it is about enforcing various forms of
>> seriality in the presence of change.  For example consider the case
>> where N messages are sent in a TX transaction, over multiple queues,
>> and during this operation one queue disappears (eg due to failure).  A
>> proper treatment of this requires a proper treatment of multiple
>> failure cases, and a clear definition of ordering.
>>
>>
>>> I'll see if I
>>> can find the spec that you refer to.
>>
>> Please do take a look and let us know what you think.
>>
>> Cheers
>>
>> alexis
>>
>>
>>
>>>
>>> Tim
>>>
>>> On 5 Jan 2009, at 11:35, Gordon Sim wrote:
>>>
>>>>
>>>> Alexis Richardson wrote:
>>>>>
>>>>> On Wed, Dec 31, 2008 at 12:17 PM, Tim Coote <tim+rabbitmq.com at coote.org
>>>>>>
>>>>>> wrote:
>>>>>> A neophyte's question:
>>>>>>
>>>>>> The  faq includes 'is rabbitmq transactional?'
>>>>>>
>>>>>> The answer is 'yes' and then says that it has atomicity and
>>>>>> durability
>>>>>> guarantees. What about Isolation and consistency?
>>>>>
>>>>> There are senses in which AMQP, and RabbitMQ, behaviour can be
>>>>> described as ACID, following the wikipedia definition:
>>>>> http://en.wikipedia.org/wiki/ACID
>>>>
>>>> In recent discussions on improving the text for the tx 'class' in
>>>> the 0-9-1 spec there appeared to be some disagreement around the
>>>> atomicity guarantee.
>>>>
>>>> In particular, I understood from comments by members of the RabbitMQ
>>>> team that though effects of all contained (publish and ack)
>>>> operations are visible if the the commit is successful, RabbitMQ
>>>> does not guarantee that none of them would be visible in the event
>>>> of a failure.
>>>>
>>>
>>> Tim Coote
>>>
>>>
>>>
>>> _______________________________________________
>>> rabbitmq-discuss mailing list
>>> rabbitmq-discuss at lists.rabbitmq.com
>>> http://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
>>>
>>
>
> Tim Coote
> tim at coote.org
> vincit veritas
>
>
>
>